Re: problem size limits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: problem size limits


Are you even able to fit a simple animal model?

You need more memory (many many gigabytes - actually a 64 bit processor as
well since on x86 chips, 32 bit processors, a single application is
limited to 3-4GB with some very slow work arounds.  This was a topic on
Linux kernal mailing lists An out
of memory error will occur when forming the equations.

You should also reduce your pedigree size as most animals don't have
records (30%?).  So I would try three to four generations only from the
animals with records.  Then it is a matter how good ASREML deals with
memory - 2GB is about 2E9 bytes, assuming 16 bytes (space required by a
double) that leaves space for about 11,180 animals if you had to form the
dense equations.  That is assuming my calculations are correct but under X
you will have around 1.5GB free ( use the 'free' and cat /proc/memory

Or change to a Gibbs sampling model or different model.

Note that you can claim back around 30MB by NOT running X and a
Window manager (change into runlevel 3 - /sbin/telinit 3 AFTER closing
all applications).  On my RH7.1 17MB is X and another 10 is KDE.

I, for one, would be very interested in the largest model that you can


On Thu, 30 Aug 2001, Denny Crews wrote:

> ASREML'ers:
> I was wondering if anyone has got a good idea of how large a problem ASREML might successfully complete.  For example, I have a single trait genetic (variance components) analysis, with n~280,000 and approximately 850,000 animals in a-inverse.
> I am running on a Linux (RedHat 7.1) workstation with two Xeon 933 processors, 2GB RDRAM and about 4 GB swap space.  Using the above data, the model specifies 2 random effects (direct and maternal genetic) in addition to the residual.  Repeated attempts to run this job result in an error message: Too BIG! use -s option.  The job was invoked using, for example, the command:
> $ asreml -s8 weight
> Apparently, ASREML forms and writes ainverse.bin without a problem, so the error occurs either while forming the equations or during iteration.  The appropriate .asr file is attached.
> Any thoughts or comments would be appreciated.
> *************************************************************
> D. H. "Denny" Crews, Jr.
> Research Scientist and Leader
> Beef Cattle Genetics
> Agriculture and Agri-Food Canada Research Centre
> 5403 1st Avenue South
> Lethbridge, Alberta T1J 4B1 Canada
> 403-317-2288 voice
> 403-382-3156 fax
> *************************************************************
> A wise man will hear, and will increase learning,
> and a man of understanding shall attain unto
> wise counsels.                                     Proverbs 1:5
> *************************************************************

Asreml mailinglist archive: