Re: testing fixed effects
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: testing fixed effects

> Unfortunately I am in the position of having to test the significance of my
> fixed effects. 

This is not unfortunate, just that this is generally ignored - models are
determined using just a fixed effects only model before estimating variance
components, etc.

>I am aware that this is not a simple task.
Actually, it is close to an impossible task if your variance components are

I would strongly advise EVERYONE to read the following:
Henderson's 1984 Book Chapter 8
Kennedy (1991) J. Dairy Science 74:4067-4081 <- excellent paper!
	(NB Henderson did not like the likelihood ratio test in this regard)
Boik et al. (1993) J. Animal Science 71:51-56 
There are older references that are referenced in these but these are the key
ones. Also the SAS proc mixed documentation has some key points.
I know there are other techniques for specific designs.
There is the Bayesian aspect as well... 

Without knowing the data structure, and model :-( , so hopefully, you can find
something useful below.

My suggestion is to:
1) Estimate the variance components
2) Use the F-test described in the first two references assuming the
non-residual variance components are fixed.  For a contrast you can use the
approach of McLean and Sanders (1988 - see the SAS proc mixed documentation).
3) Apply sensitivity analysis to verify your conclusions.

Some points to note:
1) Kacker and Harville (1981 Comm in Stat A10:1249) show the estimates of the
solutions remain unbiased provided the estimates of the variance components are
translation invariant and are even functions of the data.
2) You can substitute estimates since the test statistic distribution should
approach the same one as if the variance components were known (Henderson, 1984;
Kennedy, 1991).  
3) The hypotheses for the fixed effects appear relatively insensitive to
moderate errors in the estimation process (Henderson, 1984; Kennedy, 1991).
4) Robinson (1991, Statistical Science 6:15-51) suggests to use a conservative
interpretation of the results.
5) Note that assuming known variance components, you have a weighted least
squares problem.  
6) If possible, you can make your tests more conservative.  For example in a
repeatability model fitting 'animal' as independent with no permanent
environment effect is more conservative than the correct repeatability model.

My experience with an animal model and a repeatability model is that point
three generally holds if there is sufficient information in the data.  I have
generally observed that p-values in the range 1 to 10% will change sufficiently
to change your conclusion.  So these are the ones to watch, mainly when there
are few observations.

Other points:
1) The correct degrees of freedom may be an issue.
2) A potential problem with the Likelihood ratio test is the influence
of the variance components.  If you are estimating these and they change then
is the difference likelihood due to the change in variance components, the
fixed effects not included or both?  

> I have been warned
> that the Wald statistic is only asymptotically distributed as a Chi square
> if all the variance components except the error is know (a situation that I
> am not in). 
Actually this is an F statistic if you estimate the residual.  For KNOWN
residual variance, it is an Chi-squared.  In the SAS manual, using a
Chi-squared rather than a F makes it more liberal since it effectively assumes
an infinite denominator degrees of freedom.

>The Questions chapter in the ASREML manual briefly addresses
> this question but with no great conviction. Welhamn and Thompson (1997) have
> also discussed this problem and describe a likelihood ratio test for fixed
> effects but I am unsure whether I can manipulate ASREML to do this.
> So are there better ways than the Wald statistic to test for the
> significance of fixed effects within a mixed model setting while still being
> able to use ASREML?
> Cheers,
> Andrew
> --
> Asreml mailinglist archive:

Asreml mailinglist archive: