Summary

A broadly applicable algorithm for computing maximum likelihood estimates from incomplete data is presented at various levels of generality. Theory showing the monotone behaviour of the likelihood and convergence of the algorithm is derived. Many examples are sketched, including missing value situations, applications to grouped, censored or truncated data, finite mixture models, variance component estimation, hyperparameter estimation, iteratively reweighted least squares and factor analysis.

References

1

Andrews
,
D. F.
,
Bickel
,
P. J.
,
Hampel
,
F.
,
Huber
,
P. J.
,
Rogers
,
W. H.
and
Tukey
,
J. W.
(
1972
).
Robust Estimates of Location.
Princeton, N.J.
:
Princeton University Press
.

2

Baum
,
L. E.
(
1971
).
An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes
. In
Inequalities, III: Proceedings of a Symposium.
(
Shisha
,
Qved
ed.).
New York
:
Academic Press
.

3

Baum
,
L. E.
and
Eagon
,
J. A.
(
1967
).
An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology
.
Bull. Amer. Math. Soc.
,
73
,
360
363
.

4

Baum
,
L. E.
,
Petrie
,
T.
,
Soules
,
G.
and
Weiss
,
N.
(
1970
).
A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains
.
Ann. Math. Statists.
41
,
164
171
.

5

Beale
,
E. M. L.
and
Little
,
R. J. A.
(
1975
).
Missing values in multivariate analysis
.
J. R. Statist. Soc., B
,
37
,
129
145
.

6

Blight
,
B. J. N.
(
1970
).
Estimation from a censored sample for the exponential family
.
Biometrika
,
57
,
389
395
.

7

Brown
,
M. L.
(
1974
).
Identification of the sources of significance in two-way tables
.
Appl. Statist.
,
23
,
405
413
.

8

Carter
,
W. H.
, Jr and
Myers
,
R. H.
(
1973
).
Maximum likelihood estimation from linear combinations of discrete probability functions
.
J. Amer. Statist. Assoc
,
68
,
203
206
.

9

Ceppellini
,
R.
,
Siniscalco
,
M.
and
Smith
,
C. A. B.
(
1955
).
The estimation of gene frequencies in a random-mating population
.
Ann. Hum. Genet.
,
20
,
97
115
.

10

Chen
,
T.
and
Fienberg
,
S.
(
1976
).
The analysis of contingency tables with incompletely classified data
.
Biometrics
,
32
,
133
144
.

11

Corbeil
,
R. R.
and
Searle
,
S. R.
(
1976
).
Restricted maximum likelihood (REML) estimation of variance components in the mixed model
.
Technometrics
,
18
,
31
38
.

12

Day
,
N. E.
(
1969
).
Estimating the components of a mixture of normal distributions
.
Biometrika
,
56
,
463
474
.

13

Dempster
,
A. P.
(
1972
).
Covariance selection
.
Biometrics
,
28
,
157
175
.

14

Efron
,
B.
(
1967
).
The two-sample problem with censored data
.
Proc. 5th Berkeley Symposium on Math. Statist. and Prob.
,
4
,
831
853
.

15

Efron
,
B.
and
Morris
,
C.
(
1975
).
Data analysis using Stein's estimator and its generalizations
.
J. Amer. Statist. Assoc
,
70
,
311
319
.

16

Good
,
I. J.
(
1965
)
The Estimation of Probabilities: An Essay on Modern Bayesian Methods.
Cambridge, Mass.
:
M.I.T. Press
.

17

Good
,
I. J.
(
1956
).
On the estimation of small frequencies in contingency tables
.
J. R. Statist. Soc., B
,
18
,
113
124
.

18

Grundy
,
P. M.
(
1952
).
The fitting of grouped truncated and grouped censored normal distributions
.
Biometrika
,
39
,
252
259
.

19

Haberman
,
S. J.
(
1976
).
Iterative scaling procedures for log-linear models for frequency tables derived by indirect observation
.
Proc. Amer. Statist. Assoc. (Statist. Comp. Sect. 1975)
, pp.
45
50
.

20

Hartley
,
H. O.
(
1958
).
Maximum likelihood estimation from incomplete data
.
Biometrics
,
14
,
174
194
.

21

Hartley
,
H. O.
and
Hocking
,
R. R.
(
1971
).
The analysis of incomplete data
.
Biometrics
,
27
,
783
808
.

22

Hartley
,
H. O.
and
Rao
,
J. N. K.
(
1967
).
Maximum likelihood estimation for the mixed analysis of variance model
.
Biometrika
,
54
,
93
108
.

23

Harville
,
D. A.
(
1977
).
Maximum likelihood approaches to variance component estimation and to related problems
.
J. Amer. Statist. Assoc
,
72
, to appear.

24

Hasselblad
,
V.
(
1966
).
Estimation of parameters for a mixture of normal distributions
.
Technometrics
,
8
,
431
444
.

25

Hasselblad
,
V.
(
1969
).
Estimation of finite mixtures of distributions from the exponential family
.
J. Amer. Statist. Assoc
,
64
,
1459
1471
.

26

Healy
,
M.
and
Westmacott
,
M.
(
1956
).
Missing values in experiments analysed on automatic computers
.
Appl. Statist.
5
,
203
206
.

27

Hosmer
,
D. W.
Jr (
1973
).
On the MLE of the parameters of a mixture of two normal distributions when the sample size is small
.
Comm. Statist.
,
1
,
217
227
.

28

Hosmer
,
D. W.
Jr (
1973
).
A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample
.
Biometrics
,
29
,
761
770
.

29

Huber
,
P. J.
(
1964
).
Robust estimation of a location parameter
.
Ann. Math. Statist.
,
35
,
73
101
.

30

Irwin
,
J. O.
(
1959
).
On the estimation of the mean of a Poisson distribution with the zero class missing
.
Biometrics
,
15
,
324
326
.

31

Irwin
,
J. O.
(
1963
).
The place of mathematics in medical and biological statistics
.
J. R. Statist. Soc., A
,
126
,
1
45
.

32

Jöreskog
,
K. G.
(
1969
).
A general approach to confirmatory maximum likelihood factor analysis
.
Psychometrika
,
34
,
183
202
.

33

McKendrick
,
A. G.
(
1926
).
Applications of mathematics to medical problems
.
Proc. Edin. Math. Soc.
,
44
,
98
130
.

34

Mantel
,
N.
and
Greenhouse
,
S. W.
(
1967
).
Note: Equivalence of maximum likelihood and the method of moments in probit analysis
.
Biometrics
,
23
,
154
157
.

35

Maritz
,
J. S.
(
1970
).
Empirical Bayes Methods.
London
:
Methuen
.

36

Martin
,
J. K.
and
McDonald
,
R. P.
(
1975
).
Bayesian estimation in unrestricted factor analysis: A treatment for Heywood cases
.
Psychometrika
,
40
,
505
517
.

37

Mosteller
,
F.
and
Wallace
,
D. L.
(
1964
).
Inference and Disputed Authorship: The Federalist.
Reading, Mass.
:
Addison-Wesley
.

38

Orchard
,
T.
and
Woodbury
,
M. A.
(
1972
).
A missing information principle: theory and applications
.
Proc. 6th Berkeley Symposium on Math. Statist. and Prob.
1
,
697
715
.

39

Patterson
,
H. D.
and
Thompson
,
R.
(
1971
).
Recovery of inter-block information when block sizes are unequal
.
Biometrika
,
58
,
545
554
.

40

Raiffa
,
H.
and
Schlaifer
,
R.
(
1961
).
Applied Statistical Decision Theory.
Cambridge, Mass.
:
Harvard Business School
.

41

Rao
,
C. R.
(
1965
).
Linear Statistical Inference and its Applications.
New York
:
Wiley
.

42

Rubin
,
D. B.
(
1974
).
Characterizing the estimation of parameters in incomplete-data problems
.
J. Amer. Statist. Assoc
,
69
,
467
474
.

43

Rubin
,
D. B.
(
1976
).
Inference and missing data
.
Biometrika
,
63
,
581
592
.

44

Sundberg
,
R.
(
1974
).
Maximum likelihood theory for incomplete data from an exponential family
.
Scand. J. Statist.
,
1
,
49
58
.

45

Sundberg
,
R.
(
1976
).
An iterative method for solution of the likelihood equations for incomplete data from exponential families
.
Comm. Statist–Simula. Computa.
,
B5
(
1
),
55
64
.

46

Turnbull
,
B. W.
(
1974
).
Nonparametric estimation of a survivorship function with doubly censored data
.
J. Amer. Statist. Assoc
,
69
,
169
173
.

47

Turnbull
,
B. W.
(
1976
).
The empirical distribution function with arbitrarily grouped, censored and truncated data
.
J. R. Statist. Soc., B
,
38
,
290
295
.

48

Wolfe
,
J. H.
(
1970
).
Pattern clustering by multivariate mixture analysis
.
Multivariate Behavioral Research
,
5
,
329
350
.

49

Woodbury
,
M. A.
(
1971
).
Discussion of paper by Hartley and Hocking
.
Biometrics
,
27
,
808
817
.

This content is only available as a PDF.
This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)