Projects Use Too Much VM

Message boards : BOINC Manager : Projects Use Too Much VM
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11209 - Posted: 25 Jun 2007, 9:04:29 UTC
Last modified: 25 Jun 2007, 9:17:48 UTC

One of my machines is a lo-spec machine running BOINC ver = 5.10.7, CPU = skt A Athlon 2400+, WIn 2K SP4, 768MB real memory, 1150MB swap file, participating in Rosetta & Seti.

Rosetta appears to be ignoring the maximum memory limits that I have set, and sometimes does not unload suspended projects. Current memory limits are 25% of real memory (in use and idle) and 5% of the swap file.
PROBLEM 1 - even though I have set the max swap file use to 5% of 1150MB = 57.5MB, Rosetta repeatedly uses way too much VM - currently a single instance of Rosetta is using 246MB of VM (as reported by Taks Manager).

I have been on the Rosetta forum and was told that memory allocation is done by BOINC, see
Rosetta Forum Thread

PROBLEM 2 = Projects have not always unloaded when suspended - this recently left 3 instances of Rosetta in memory once while Seti was running, each instance of Rosetta was using over 150MB of VM, so approx 500MB of VM was being used when I would have expected no more than 57.5MB to be used.

Can anyone help?

ID: 11209 · Report as offensive
clownius

Send message
Joined: 17 Feb 07
Posts: 35
Australia
Message 11210 - Posted: 25 Jun 2007, 15:50:49 UTC

Do you have a local preferences file? It could be that causing it. Someone else will expand on this for sure.

It should be called global_prefs_overide.xml or similar
ID: 11210 · Report as offensive
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11211 - Posted: 25 Jun 2007, 17:20:07 UTC - in response to Message 11210.  
Last modified: 25 Jun 2007, 17:20:23 UTC

Do you have a local preferences file? It could be that causing it. Someone else will expand on this for sure.

It should be called global_prefs_overide.xml or similar


I do have a file called global_prefs.xml and it has a entry vm_max_used_pct with a value of 5. I can't see any other file starting with "global....". The 5 must mean 5%, so I presume that it confirms that the setting is 5%, as expected.

Anyone know if there are any other "overide" files that may be causing the problem?
ID: 11211 · Report as offensive
j2satx

Send message
Joined: 31 Aug 05
Posts: 51
United States
Message 11212 - Posted: 25 Jun 2007, 17:52:41 UTC - in response to Message 11209.  

One of my machines is a lo-spec machine running BOINC ver = 5.10.7, CPU = skt A Athlon 2400+, WIn 2K SP4, 768MB real memory, 1150MB swap file, participating in Rosetta & Seti.

Rosetta appears to be ignoring the maximum memory limits that I have set, and sometimes does not unload suspended projects. Current memory limits are 25% of real memory (in use and idle) and 5% of the swap file.
PROBLEM 1 - even though I have set the max swap file use to 5% of 1150MB = 57.5MB, Rosetta repeatedly uses way too much VM - currently a single instance of Rosetta is using 246MB of VM (as reported by Taks Manager).

I have been on the Rosetta forum and was told that memory allocation is done by BOINC, see
Rosetta Forum Thread

PROBLEM 2 = Projects have not always unloaded when suspended - this recently left 3 instances of Rosetta in memory once while Seti was running, each instance of Rosetta was using over 150MB of VM, so approx 500MB of VM was being used when I would have expected no more than 57.5MB to be used.

Can anyone help?


Do you use this machine for other purposes, that you need 1Gb of vm reserved?


ID: 11212 · Report as offensive
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11252 - Posted: 26 Jun 2007, 8:15:04 UTC - in response to Message 11213.  
Last modified: 26 Jun 2007, 8:16:01 UTC



The only preference override file is global_prefs_overide.xml. If you don't have one then your preferences will be determined by the preferences specified on the project web pages.

I believe (though I am not certain) the reason Rosetta/BOINC is ignoring your 57.5MB limit is that it simply cannot run in 57.5MB. If that is the case then you would think BOINC would generate a message stating such and then not let Rosetta run. That would be the nice way to handle the situation but we don't always get nice from BOINC. It's better than it used to be but still there are a few unfriendly things. Again, that's just my theory and I may be wrong.

As for 3 Rosetta work units staying in memory while suspended.... are you sure that preference is set the way you think it is? Sometimes they get changed accidentally so maybe check it again? If it really is set to not suspend to memory then it appears you have found a bug. The good news is that 5.10.8 has been released. Maybe they've fixed all your problems in 5.10.8? Yes, it is a beta release but it behaves well here though I must admit I have not tested whether or not it respects VM settings and suspends out of memory properly.


The Rosetta W/Us being left in memory appears to have been triggered by me setting the max memory to be used (while working & idle) to 10% of 768MB, & it's sensible to assume that Rosetta cannot work with so little memory as doing this now causes Rosetta to abort & error. OK, the moral is don't set the allowed real memory so low I have just installed BOINC 5.10.8 & now this problem has gone away. I have now set the allowed real memory to 50% as the recent "slow downs" happened only when about 0.5GB of swap file was being used by Rosetta.

But I still have a Rosetta W/U using way more than the amount of VM I've specified. Even if BOINC calculates the allowed VM based on 5%*(real mem + swap file), it still gives a limit of 95.9MB and I have 167MB currently in use.

Is this a BOINC or a Rosetta problem ?

ID: 11252 · Report as offensive
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11262 - Posted: 26 Jun 2007, 11:08:13 UTC
Last modified: 26 Jun 2007, 11:08:43 UTC

To confirm,
"Leave applications in memory while suspended" is set to off,
and it is set to,
"run based on preferenced",
and the swap file is set to 1150MB-1150MB in an attempts to reduce HDD fragmentation (OK, this is a bit futile but having a fixed swap file seems to make a slight difference). (but I do appreciate the ideas)

I admit that I have not allocated a lot of memory, but I leave the PC on 24/7 for remote work access reasons, so it seems a waste not to put the CPU cycles to use but it has to be done in a way that does not overly affect PC performance. If I can't sort out the Rosetta VM problem I may need to pick another project that has low requirements!

At least the 3 instances of Rosetta being left using memory & VM has not not happened for a couple of days - which I knew which thing that I changed made the difference.

So I am still back at wondering why 5% of 1150MB=57.5MB of VM is being exceeded.

ID: 11262 · Report as offensive
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11314 - Posted: 27 Jun 2007, 10:57:14 UTC

Notwithstanding the fact that the specific VM limits are still being exceeded by Rosetta Work units, the PC performance is adequate and BOINC is not currently affecting the remote access feature that lets me work from home, so my conclusions are:-
1 - I have set limits of 50% of real memory (because it seems sensible) and 5% of swap file (because of the small fork tailed guy sitting on one shoulder) and it seems to work.
2 - Why is it exceeding the VM limit, "Nobody knows. Nobody will ever know." and life is too short for me to worry about it.
3 - Rosetta will still be run alongside Seti but I have verbally threatened it so it knows it will be removed if it acts up again :-)

Thanks for the help peoples.
ID: 11314 · Report as offensive
Profile mzrider

Send message
Joined: 25 Jun 07
Posts: 10
Message 11325 - Posted: 27 Jun 2007, 19:59:01 UTC - in response to Message 11321.  

If running Windows then another way to squeeze out a little more speed is to reduce/eliminate page file fragmentation by putting pagefile.sys all by itself on a spare partition.

Totally agree. I have a 2nd HDD on all my home PCs that I use for the swap file (& Linux) so the swap file is on a separate drive to all the Windows files.


If no spare partition then another way to reduce/eliminate page file fragmentation is to turn off paging, reboot, delete pagefile.sys, defrag, specify a minimum page file size larger than you will ever use, turn on paging. It seemed to help here a while back when running a rather nasty app that consumed huge amounts of VM on a low spec machine. I'm not sure what Microsoft says about it but I've heard from a reliable source that Windows does not defrag pagefile.sys when defragmenting the disk, hence the procedure above.

Just a few ideas you may already be aware of in which case they might help someone else.

Again I totally agree. The swap file defrag routine you mention is part of my normal Windows installation routine. I also believe that there are 3rd party defrag programs out there that will defrag the swap file before Windows boots up and prevents a defrag being possible.

The main point for me is that the problem of 3 Rosetta work units all being in memory at once and between them using about 500MB of swap file has not happened again - but I will be making sure that the "tweaks" mentioned above are present on all my PCs. :-)

Hope these tweaks help others.

ID: 11325 · Report as offensive

Message boards : BOINC Manager : Projects Use Too Much VM

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.