Re: problem size limits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: problem size limits



Hi,

Are you even able to fit a simple animal model?

You need more memory (many many gigabytes - actually a 64 bit processor as
well since on x86 chips, 32 bit processors, a single application is
limited to 3-4GB with some very slow work arounds.  This was a topic on
Linux kernal mailing lists http://kt.zork.net/kernel-traffic/.). An out
of memory error will occur when forming the equations.

You should also reduce your pedigree size as most animals don't have
records (30%?).  So I would try three to four generations only from the
animals with records.  Then it is a matter how good ASREML deals with
memory - 2GB is about 2E9 bytes, assuming 16 bytes (space required by a
double) that leaves space for about 11,180 animals if you had to form the
dense equations.  That is assuming my calculations are correct but under X
you will have around 1.5GB free ( use the 'free' and cat /proc/memory
commands).

Or change to a Gibbs sampling model or different model.

Note that you can claim back around 30MB by NOT running X and a
Window manager (change into runlevel 3 - /sbin/telinit 3 AFTER closing
all applications).  On my RH7.1 17MB is X and another 10 is KDE.

I, for one, would be very interested in the largest model that you can
run.

Regards
Bruce

On Thu, 30 Aug 2001, Denny Crews wrote:

> ASREML'ers:
>
> I was wondering if anyone has got a good idea of how large a problem ASREML might successfully complete.  For example, I have a single trait genetic (variance components) analysis, with n~280,000 and approximately 850,000 animals in a-inverse.
>
> I am running on a Linux (RedHat 7.1) workstation with two Xeon 933 processors, 2GB RDRAM and about 4 GB swap space.  Using the above data, the model specifies 2 random effects (direct and maternal genetic) in addition to the residual.  Repeated attempts to run this job result in an error message: Too BIG! use -s option.  The job was invoked using, for example, the command:
>
> $ asreml -s8 weight
>
> Apparently, ASREML forms and writes ainverse.bin without a problem, so the error occurs either while forming the equations or during iteration.  The appropriate .asr file is attached.
>
> Any thoughts or comments would be appreciated.
>
> DENNY
>
>
> *************************************************************
> D. H. "Denny" Crews, Jr.
> Research Scientist and Leader
> Beef Cattle Genetics
> Agriculture and Agri-Food Canada Research Centre
> 5403 1st Avenue South
> Lethbridge, Alberta T1J 4B1 Canada
> 403-317-2288 voice
> 403-382-3156 fax
> dcrews@em.agr.ca
> *************************************************************
> A wise man will hear, and will increase learning,
> and a man of understanding shall attain unto
> wise counsels.                                     Proverbs 1:5
> *************************************************************
>
>


--
Asreml mailinglist archive: http://www.chiswick.anprod.csiro.au/lists/asreml