3. 4. Parameter estimation

Consider a vector of random variables, X = (X₁,X₂,...,X_n), representing any of the types of data described above. Assume that X is drawn from some distribution whose form is known but parameters unknown - if X is continuous, if X is discrete. Parameter estimation is the process of choosing a set of parameters, , such that the model is as consistent as possible with a vector of observations of the random variables, x = (x₁,x₂,...,x_n). While a wide variety of methods exist for estimating parameters, I have employed two techniques: generalized least squares, and maximum likelihood estimation.

generalized least squares

Least squares parameter estimation is commonly used in regression analyses (Draper and Smith, 1981; Neter, et al. 1985; Seber and Wild, 1989). I have also used it in applications where the model is applied to frequency data. The model takes the form

, (3.3)

where n_i is the observed number of individuals in the ith class, N is the total number observed in all classes, p_i is the probability (under the model) of an individual falling in the ith class, and is the error term. Generalized least squares is often used when there are unequal variances among the error terms (Draper and Smith, 1981; Seber and Wild, 1989). With generalized least squares, the following equation is minimized with respect to the parameter vector, :

, (3.4)

where k is the number of classes, and w_i is the weight associated with that class. To account for unequal variances, the weighting function w_i = 1/v_i is often used, where v_i is the variance of the ith class.

maximum likelihood

Maximum likelihood estimation proceeds by maximizing the likelihood function, , with respect to the parameters. X can be either continuous or discrete. The likelihood function is defined as (Mood, et al., 1974; Bickel and Doksum, 1977):

(3.5)

for continuous functions. For discrete models, is substituted for . Maximum likelihood estimation involves selecting the parameter vector, , which is "most likely" to have produced the data. In other words,

. (3.6)

If is differentiable with respect to the _i's, then it can be maximized by setting

, (3.7)

where p is the number of parameters being estimated. Otherwise, can be maximized numerically.

It is generally easier to work with the log of the likelihood function,

. (3.8)

With discrete data, based on the multinomial distribution becomes:

, (3.9)

where c is a combinatorial constant that is unaffected by the choice of parameters.

performance of parameter estimates

In comparing competing parameter estimation methods, the most commonly used criteria for assessing the performance are the bias and the precision of the parameter estimates (Bickel and Doksum, 1977). Bias is defined as , where is the true value of the parameter and is the estimated value. Obviously, as small a bias as possible is desirable. A common definition of precision is the mean squared error (MSE), given by . MSE is equal to the variance of the parameter estimate plus the bias squared (Bickel and Doksum, 1977), so if the estimate is unbiased, MSE is equal to the variance. In many cases, it is possible to determine these values directly; in cases where this is not possible, simulations can be used. In the last section of this chapter, I discuss simulation procedures.