Abstract

In Firth (1993, Biometrika) it was shown how the leading term in the asymptotic bias of the maximum likelihood estimator is removed by adjusting the score vector, and that in canonical-link generalized linear models the method is equivalent to maximizing a penalized likelihood that is easily implemented via iterative adjustment of the data. Here a more general family of bias-reducing adjustments is developed for a broad class of univariate and multivariate generalized nonlinear models. The resulting formulae for the adjusted score vector are computationally convenient, and in univariate models they directly suggest implementation through an iterative scheme of data adjustment. For generalized linear models a necessary and sufficient condition is given for the existence of a penalized likelihood interpretation of the method. An illustrative application to the Goodman row-column association model shows how the computational simplicity and statistical benefits of bias reduction extend beyond generalized linear models.

References

Bull
S. B.
Lewinger
J. B.
Lee
S. S. F
Confidence intervals for multinomial logistic regression in sparse data
Statist. Med.
 , 
2007
, vol. 
26
 (pg. 
903
-
18
)
Bull
S. B.
Mak
C.
Greenwood
C
A modified score function estimator for multinomial logistic regression in small samples
Comp. Statist. Data Anal.
 , 
2002
, vol. 
39
 (pg. 
57
-
74
)
Cook
R. D.
Tsai
C.-L.
Wei
B. C
Bias in nonlinear regression
Biometrika
 , 
1986
, vol. 
73
 (pg. 
615
-
23
)
Cordeiro
G. M.
McCullagh
P
Bias correction in generalized linear models
J. R. Statist. Soc. B
 , 
1991
, vol. 
53
 (pg. 
629
-
43
)
Cramér
H
Mathematical Methods of Statistics
 , 
1946
Princeton, NJ
Princeton University Press
Fahrmeir
L.
Tutz
G
Multivariate Statistical Modelling Based on Generalized Linear Models
 , 
2001
New York
Springer
Firth
D
Fahrmeir
L.
Francis
B.
Gilchrist
R.
Tutz
G.
Bias reduction, the Jeffreys prior and GLIM
Advances in GLIM and Statistical Modelling: Proc. GLIM 92 Conf.
 , 
1992
New York
Springer
(pg. 
91
-
100
)
Firth
D
Dodge
Y.
Whittaker
J.
Generalized linear models and Jeffreys priors: an iterative generalized least-squares approach
Computational Statistics I
 , 
1992
Heidelberg
Physica
(pg. 
553
-
7
)
Firth
D
Bias reduction of maximum likelihood estimates
Biometrika
 , 
1993
, vol. 
80
 (pg. 
27
-
38
)
Goodman
L. A
Simple models for the analysis of association in cross-classifications having ordered categories
J. Am. Statist. Assoc.
 , 
1979
, vol. 
74
 (pg. 
537
-
52
)
Goodman
L. A
Association models and canonical correlation in the analysis of cross-classifications having ordered categories
J. Am. Statist. Assoc.
 , 
1981
, vol. 
76
 (pg. 
320
-
34
)
Goodman
L. A
The analysis of cross-classified data having ordered and/or unordered categories: association models, correlation models, and asymmetry models for contingency tables with or without missing entries
Ann. Statist.
 , 
1985
, vol. 
13
 (pg. 
10
-
69
)
Heinze
G.
Schemper
M
A solution to the problem of separation in logistic regression
Statist. Med.
 , 
2002
, vol. 
21
 (pg. 
2409
-
19
)
Heinze
G.
Schemper
M
A solution to the problem of monotone likelihood in Cox regression
Biometrics
 , 
2004
, vol. 
57
 (pg. 
114
-
9
)
Jeffreys
H
An invariant form for the prior probability in estimation problems
Proc. R. Soc. Lond.
 , 
1946
, vol. 
186
 (pg. 
453
-
61
)
Magnus
J. R.
Neudecker
H
Matrix Differential Calculus with Applications in Statistics and Econometrics
 , 
1999
Chichester
Wiley
McCullagh
P.
Nelder
J
Generalized Linear Models
 , 
1989
2nd ed. London
Chapman and Hall
Mehrabi
Y.
Matthews
J. N. S
Likelihood-based methods for bias reduction in limiting dilution assays
Biometrics
 , 
1995
, vol. 
51
 (pg. 
1543
-
49
)
Pettitt
A. N.
Kelly
J. M.
Gao
J. T
Bias correction for censored data with exponential lifetimes
Statist. Sinica
 , 
1998
, vol. 
8
 (pg. 
941
-
64
)
Sartori
N
Bias prevention of maximum likelihood estimates for scalar skew normal and skew t distributions
J. Statist. Plan. Infer.
 , 
2006
, vol. 
136
 (pg. 
4259
-
75
)
Wei
B
Exponential Family Nonlinear Models
 , 
1997
New York
Springer
Zorn
C
A solution to separation in binary response models
Polit. Anal.
 , 
2005
, vol. 
13
 (pg. 
157
-
70
)