Re: Using Poisson data

Hi,
Since this email got rejected due to the size, I have removed the data.

Regards
Bruce

---- Original message ----
>Date: Thu, 28 Apr 2005 08:58:20 -0500
>From: Bruce Southey <southey_at_uiuc.edu>
>Subject: Re: Using Poisson data
>To: Rob Brooks <rob.brooks_at_unsw.edu.au>, ASREML-L_at_AGRIC.NSW.GOV.AU
>
>Hi,
>It really helps to examine the data before deciding on the distribution!
>
>The problem is that the data is not Poisson as the mean of the variable
'first'
>is 3397.31 but ranges from zero to 31710! The Poisson parameter should be
close
>to the mean so the distribution would be fairly normal centered around this
>mean value (look at a Poisson distribution with a mean of 3397). However if
you
>look at the frequencies of the variable 'first' you will see that the most
>frequent values occur between zero and about 28 (this is only 1/3 of the data)
>with the rest being rather continuous (occurring once per value).
>
>You really need to look at the data and model before decide what
transformation
>or distribution is appropriate. A gamma/weibull distribution may also be
>appropriate. Natural log and square root are not appropriate transformations
>with the model you fitted. You have some heterogeneity of variance with the
>sires so perhaps the model is missing some important term.
>
>Given the mating structure you probably should be fitting sire, dam and the
>interaction between sire and dam as in the NCII design. Unless you have clones
>or plants, you probably need to address the fact that a sire (dam) can not
have
>two dams (sires) at the same time.
>
>Regards
>Bruce
>
>---- Original message ----
>>Date: Thu, 28 Apr 2005 12:47:22 +1000
>>From: Rob Brooks <rob.brooks_at_unsw.edu.au>
>>Subject: Re: Using Poisson data
>>To: southey_at_UIUC.EDU, ASREML-L_at_AGRIC.NSW.GOV.AU
>>
>>Dear Bruce,
>>
>>Thanks for your response. I would be ecstatic if I could assume normality.
>>I am attaching the .asr file, and, in case you want to see what we have
>>done, the data file. The commands in the .as file are copied below.
>>
>>Many thanks,
>>
>>Rob
>>
>>
>>
>>The job was
>>
>>sire 53 !I
>> dam 9 !I
>> first second third thirdage
>> avg 1st max log1st log2nd log3rd
>>cbd.txt !skip1
>>first !pois !disp ~ mu !r sire sire.dam
>>
>>
>>At 08:43 PM 27/04/2005 -0500, Bruce Southey wrote:
>>>Hi,
>>>There is probably a good chance that you can assume normality. You probably
>>>should at least do so. You may need to first use a square root or natural
log
>>>transformation to stabilize the variance.
>>>
>>>This should provide some basis for using the variances and model checking.
>>>Your
>>>results suggest the the model (including distribution) may not be correct.
>>>
>>>Can you at least provide the relevant part of the .asr file?
>>>
>>>Regards
>>>Bruce
>>>
>>>---- Original message ----
>>> >Date: Thu, 28 Apr 2005 10:41:32 +1000
>>> >From: Rob Brooks <rob.brooks_at_UNSW.EDU.AU>
>>> >Subject: Using Poisson data
>>> >To: ASREML-L_at_AGRIC.NSW.GOV.AU
>>> >
>>> >Dear ASREMLers
>>> >
>>> >I have recently used !POIS to estimate variances of poisson distributed
>>> >data from a half-sib design. The dire and dam within sire components are
>>> >significantly greater than zero but appear impossibly small relative to
the
>>> >within groups variance. I gather this is an issue to do with the scaling
of
>>> >poisson variances, and it seems from searching back through the ASREML
>>> >discussions that there is no appropriate way to use these variance
>>> >estimates to calculate heritabilities.
>>> >
>>> >Likewise, I gather that there is no way to estimate covariances where one
>>> >or more traits follows a poisson distribution.
>>> >
>>> >Am I right about these points or am I missing something?
>>> >
>>> >Further, would it be appropriate to use the sire breeding values for each
>>> >trait given in the .sln file to estimate the correlation among breeding
>>> >values as a rough estimate of the genetic correlation?
>>> >
>>> >Your thoughts on these issues would be much appreciated.
>>> >
>>> >Rob
>>> >
>>> >
>>>
>>............................................................................
>>> >............................
>>> >School of Biological, Earth and Environmental Sciences
>>> >The University of New South Wales
>>> >Kensington, Sydney 2052
>>> >NSW, Australia
>>> >
>>> >http://www.bees.unsw.edu.au/research/groups/brookslab/brookslab.html
>>________________

>>________________
>> ASREML [25 May 1999] analysing herit of Felix's callbox data
>> 28 Apr 2005 12:44:53.921 8.00 Mbyte h2
>> QUALIFIERS: !skip1
>> Reading cbd.txt FREE FORMAT skipping 1 lines
>> Univariate analysis of first
>> Using 785 records [of 785 read from 785 lines of
>cbd.txt ]
>> Model term Size Type COL Minimum Mean Maximum #zero
>#miss
>> 1 sire 53 Factor 1 1 25.8510 52 0
>0
>> 2 dam 9 Factor 2 1 4.1732 9 0
>0
>> 3 first 1 Variate 3 1.000 3397. 0.3171E+05 115
>0
>> 4 second 1 Covariat 4 1.000 0.1287E+05 0.4463E+05 85
>119
>> 5 third 1 Covariat 5 1.000 0.1356E+05 0.4180E+05 43
>553
>> 6 thirdage 1 Covariat 6 1.000 1.025 2.000 0
>0
>> 7 avg 1 Covariat 7 0.5000 7790. 0.3068E+05 41
>0
>> 8 max 1 Covariat 8 1.000 1.167 3.000 0
>0
>> 9 log1st 1 Covariat 9 0.6931 7.697 10.71 41
>0
>> 10 log2nd 1 Covariat 10 0.6931 4.803 10.36 115
>0
>> 11 log3rd 1 Covariat 11 0.6931 7.279 10.71 85
>119
>> 12 mu 1 Constant Term
>> 13 sire.dam 477 Interaction 1 sire : 53 2 dam :
>9
>> Forming 532 equations: 2 dense
>> Initial updates will be shrunk by factor 0.010
>> Data distribution and link: Poisson; Log Mu=exp(XB)
>> Deviance adjust= -6453748.50
>> Deviance adjust= -20843224.0
>> NOTICE: 175 (more) singularities,
>> LogL=-.104197E+08 S2= 1880.8 784 df 0.10000 0.10000 1.0000
>>
>> Deviance adjust= -7808407.50
>> Deviance adjust= -3843321.25
>> LogL=-.191972E+07 S2= 2799.7 784 df 0.10000E-010.10000E-01 1.0000
>>
>> Deviance adjust= -2797413.50
>> Deviance adjust= -2824858.50
>> LogL=-.141017E+07 S2= 3326.1 784 df 0.10000E-020.10000E-02 1.0000
>>
>> Deviance adjust= -2719273.75
>> Deviance adjust= -3044960.25
>> LogL=-.152009E+07 S2= 4039.8 784 df 0.10000E-030.32401E-03 1.0000
>>
>> Deviance adjust= -2984056.75
>> Deviance adjust= -3299057.25
>> LogL=-.164711E+07 S2= 4678.4 784 df 0.77055E-040.15373E-03 1.0000
>>
>> Deviance adjust= -3281172.75
>> Deviance adjust= -3356524.50
>> LogL=-.167585E+07 S2= 4862.3 784 df 0.63442E-040.13519E-03 1.0000
>>
>> Deviance adjust= -3355855.50
>> Deviance adjust= -3374219.75
>> LogL=-.168469E+07 S2= 4898.1 784 df 0.61806E-040.13088E-03 1.0000
>>
>> Deviance adjust= -3374192.00
>> Deviance adjust= -3377668.25
>> LogL=-.168642E+07 S2= 4904.6 784 df 0.61478E-040.13009E-03 1.0000
>>
>> Deviance adjust= -3377667.25
>> Deviance adjust= -3378367.75
>> LogL=-.168677E+07 S2= 4905.9 784 df 0.61414E-040.12993E-03 1.0000
>>
>> Deviance adjust= -3378367.75
>> Deviance adjust= -3378509.25
>> LogL=-.168684E+07 S2= 4906.1 784 df 0.61401E-040.12990E-03 1.0000
>>
>> Final parameter values 0.61398E-040.12990E-03 1.0000
>>
>> Variance heterogenity factor [Deviance/DF] 4309.32
>>
>> Source Model terms Gamma Component Compnt/StndErr
>%C
>> sire 53 52 0.613984E-04 0.301228 2.64 0 P
>> sire.dam 477 303 0.129896E-03 0.637287 5.41 0 P
>> Variance 785 784 1.00000 4906.12 16.28 0 U
>>
>> Analysis of Variance DF F-incr F-adj StndErrDiff
>> 12 mu 1 5648.08 5648.08
>>
>> Solution Standard Error T-value T-prev
>> 12 mu
>> 1 7.83539 0.104258 75.15
>> 1 sire 52 effects fitted
>> 13 sire.dam 303 effects fitted
>> Finished: 28 Apr 2005 12:44:55.875 LogL Converged
>>________________
>>............................................................................
>>............................
>>School of Biological, Earth and Environmental Sciences
>>The University of New South Wales
>>Kensington, Sydney 2052
>>NSW, Australia
>>
>>http://www.bees.unsw.edu.au/research/groups/brookslab/brookslab.html
>
>
Received on Wed Apr 28 2005 - 09:04:14 EST