Here is another quandary:

I checked BOINC messages today, and saw a red message that a Spinhenge WU was late, running High Priority and advising me to think about aborting it because I might not get credit for it.

First, let me say that I am not in this for credit.

This is a relatively short WU. I do not know why it would be late.

As I said, I rarely check messages or look at results. So, I visited the web site of each of my projects and checked my results. I saw only a rare failure here and there, so, I figure everything is O.K.

Is there a way for you all or for me to determine if I am running too many projects?

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

The number of projects is most likely not the problem. I have a hunch you have your cache (additional work) set too high or have been micro-managing BOINC. The cache need be no higher than .1 if you have a reliable ISP, allow BOINC access to the net 24/7 and are connected 24/7.

Many people think that if they are attached to 20 projects then there is something wrong if they don't have a task from each of those projects in their cache at all times. That is wrong thinking and if you are micro-managing BOINC to force it to have a task for each project at all times then you'll get into trouble.

Hey, thanks for the reply.

I am not micro-managing BOINC, everything is running at the default settings. I just bwent looking for teh cache in BOINC Manager and could not find it mto check it. That tells you how good I am, not.

But, I will be happy to check it if someone will tell me how to find it.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

BOINC Manager->(Advanced view if in Simple GUI mode->)Tasks.

---

Micro-managing refers to suspending some tasks/projects for various reasons such as forcing tasks to download for other projects. It does not refer to altering the defaults.

Now about the default settings. The scheduler does a very good job of making sure your computer doesn't download tasks it cannot complete on time, no matter how many projects you run. However, it's obvious that in order to do it's job properly, the scheduler needs accurate estimates of how long tasks will take. That estimate is provided by the projects themselves. Some projects give very inaccurate estimates of task completion time and if you are attached to one of those projects then it's wise to set your cache to .1 or less rather than the default .25. In fact, even if you're not attached to one of those projects, setting to .1 or less is a good idea just in case a project's estimates change from being accurate to inaccurate. It happens.

If you're running BOINC on a laptop that you want to load up with lots of tasks to crunch while you're on the road and disconnected from the network then you'll want to set your cache differently. I am assuming your computer is a desktop that is connected 24/7.

I am running two PIII's and two Core 2 Duos. In both cases, one is a desktop and one is a laptop. the two PIII's run 24/7. I re-boot them every day just in case of memory leakage. Of the Core 2 Duos, my home desktop runs about 140 hrs/wk. My work laptop runs about 162 hrs/wk (it is only down while I commute). The PIII's have Rosetta, Spinhenge, Proteins@home, and Rice from WCG, because they all have WU's that those machines can complete. The Core 2 Duos both run Proteins, Rosetta, Einstein, LHC, QMC, and WCG (Rice and HCC). One also runs Cosmology, the other runs MilkyWay.

I still cannot find that cache setting in either Simple View or Advanced View.
I would like to find it, just to see what you are talking about.

But I have not changed anything.

If you can give me any more direction, I would certainly appreciate it.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

I still cannot find that cache setting in either Simple View or Advanced View. I would like to find it, just to see what you are talking about.

Having fixed a typo (whole word) in my last post, let's try to type correctly this time. ;-)

cache settings, aka how much work you can get in;
Available in Advanced view only.
Advanced (from the top menu bar)->Preferences->Network usage->general options.

Settings for Connect every and Additional work buffer.

---

I still cannot find that cache setting in either Simple View or Advanced View. I would like to find it, just to see what you are talking about.

Having fixed a typo (whole word) in my last post, let's try to type correctly this time. ;-)

cache settings, aka how much work you can get in;
Available in Advanced view only.
Advanced (from the top menu bar)->Preferences->Network usage->general options.

Settings for Connect every and Additional work buffer.

Great!! I'm there. Do they both get .1? The connect every is at 0.3000, the Additional is 0.00.

So much thanks!!

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Great!! I'm there. Do they both get .1? The connect every is at 0.3000, the Additional is 0.00.

If you connect via dialup modem and it's on your voice line then you can use the connect every to limit connections so it interferes less with voice. With 0.30 it will connect to projects every 0.30 days and maintain enough work in the cache to keep your computer busy between connects.

The additional tells BOINC to cache that much work in addition to whatever it caches with the connect every setting. The units are days, not hours.

If you have a 24/7 connection or your dialup modem is on it's own line then I would set connect every to 0.00 which tells BOINC to connect to projects whenever it wants. I would set the additional to .1 or less.

Thanks so much for your help and your patience. I will make the adjustments and see what occurs.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Thanks so much for your help and your patience. I will make the adjustments and see what occurs.

>>RSM

You're welcome. BOINC sometimes takes a while to adjust to changes so give it a week or 2.

Also, I was looking at the computers you have attached at Spinhenge and noticed that some are running older versions of BOINC. The scheduler has seen a lot of improvements in the past year or so. Some of the older versions were not the best. I suggest updating your hosts to BOINC version 5.10.45.

I may have forgotten 1 important point. Several versions back the scheduler used to sum all the time to completion estimates for all the tasks in the cache and use that sum to decide if it should download more work. I think that may have changed recently. I think perhaps now the scheduler applies the additional setting to each project individually. In other words, if you set additional to .1 days then the scheduler will cache .1 days work for each and every project. If you have 10 projects attached and additional = .1 then the scheduler would cache a total of 1 day of work, not just .1 day. Comments anyone? Do I understand correctly? (Sorry, I can't deduce the answer from my own hosts because i run primarily ABC and CPDN which have wierd time to completion estimates.)

Thanks so much. I have really learned a lot. You know, I am five miles wide and one millimeter thick when it comes to this work. I am not an I/T person, not a scientist, and, on the computer, not a very good drillmeister. I am in this work because I love it. I am not in it for credits. It provides me with a way to give back to society, which has given me a lot. It's sort of like donating blood- a passive contribution that does not ask me for money or too much time. I know that at WCG, the projects are well vetted, and at BOINC, I can see also from where the projects originate, so, I know everyone is a good guy.

I have taken as my own personal responsibilities a.)to follow the journalism on crunching and go after journalists to get BOINC and WCG mentioned in their pieces; and b.) to generally proselytize. I started a team in my home location, I just love this work.

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

BOINCview still shows totals of all project to equal to the total cache i.e. it's caching proportional to resource share / project weight (that what was chosen to be cached + that what comes from the 'connect' frequency), proportionate to processors and % processing allowed and active_frac (for all that is the old "Read the Manual" adagio applicable).

For permanent connected clients, best is I think a very low value like 0.01 days and use the AD/Cache to steer the buffering entirely.

Difference of old and new scheduler is that the old would not count the suspended task thus allowing complete overloading on queued tasks way into the permanent Earliest Deadline First / Too Late computing realm even running 24/7 at 100% (Once the Suspended tasks are released). The latest versions include all time to go on running tasks, plus anything not started, in whatever state.

Boy, this is getting pretty deep for a guy like me. I translate that i should set Connect to 0.01, and leave Add at 0.00? Is that a correct interpretation?

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Sekerob advises set Connect to 0.01 if you are on a permanent connection. For Additional he doesn't mention a specific number, he just says use Additional to steer the cache which means set it anywhere from 0 to 10 depending on your needs and computing habits. I see nothing wrong with that advice.

I set the Connect to 0.00 simply because when I hover the mouse pointer over the Connect box the tooltip hint advises to set it at 0.00 if always connected. To me that suggests that if it's 0.00 then BOINC is free to use the connection whenever it wants/needs to. I like that sort of unrestricted freedom. If it's not 0.00 then it seems to me BOINC is subject to a restriction I see no need for because I have ADSL not dialup. 0.00 or 0.01... I don't think it makes much difference. Experiment with both values, it can't hurt.

The Additional setting is, as Sekerob said, used to steer the cache. I set it no higher than .1 because some projects send tasks estimated to require 2 hours but they end up taking 36 hours. If I had Additional set to 1.00 then there is a chance I might cache several 36 hour tasks which would make 1 or more tasks late. If I set Additional = 0.00 then I find that I don't have enough work cached to keep the computer busy when my ISP goes offline.

If you set Additional to 0.00 and Connect to 0.01 then you won't cache much work. If your connection goes down due to ISP maintenance, storm or whatever, your computer may not have enough work to last until your connection comes back up. Only you know how reliable your ISP is so it's your decision.

[/quote]


Hey, thanks I am appreciative of your time and the specificity of your suggestions.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Boy, this is getting pretty deep for a guy like me. I translate that i should set Connect to 0.01, and leave Add at 0.00? Is that a correct interpretation?

>>RSM

Sekerob advises set Connect to 0.01 if you are on a permanent connection. For Additional he doesn't mention a specific number, he just says use Additional to steer the cache which means set it anywhere from 0 to 10 depending on your needs and computing habits. I see nothing wrong with that advice.

I set the Connect to 0.00 simply because when I hover the mouse pointer over the Connect box the tooltip hint advises to set it at 0.00 if always connected. To me that suggests that if it's 0.00 then BOINC is free to use the connection whenever it wants/needs to. I like that sort of unrestricted freedom. If it's not 0.00 then it seems to me BOINC is subject to a restriction I see no need for because I have ADSL not dialup. 0.00 or 0.01... I don't think it makes much difference. Experiment with both values, it can't hurt.

The Additional setting is, as Sekerob said, used to steer the cache. I set it no higher than .1 because some projects send tasks estimated to require 2 hours but they end up taking 36 hours. If I had Additional set to 1.00 then there is a chance I might cache several 36 hour tasks which would make 1 or more tasks late. If I set Additional = 0.00 then I find that I don't have enough work cached to keep the computer busy when my ISP goes offline.

If you set Additional to 0.00 and Connect to 0.01 then you won't cache much work. If your connection goes down due to ISP maintenance, storm or whatever, your computer may not have enough work to last until your connection comes back up. Only you know how reliable your ISP is so it's your decision.

To keep it simple:

• cache_size = nCPUs*(Connect+Additional), BOINC will try to maintain enough work in cache to keep computer busy for (Connect+Additional) days long (to overcome project outages, ISP maintenance intervals etc.),
  
• according to the source code, Connect every is low-bound to 0.00001, any lower value (like 0.00) si simply set to 0.00001 (a bit less than 1 second),
  
• Connect every value should be correctly used by projects to not send tasks with a deadline shorter than Connect every days, because your client is suggesting it might not connect the project any sooner (but some projects do not obey it), the value is also used by scheduler to decide, if any task(s) is(are) to be late and should be run with priority (in EDF mode),
  
• if a project sets wrong task estimates (like 2 instead of 36 hours), the project has to be blamed :-) If all their tasks behave like this, then BOINC tries to accomodate the rDCF factor accordingly - this may take up to a few tens of crunched results to settle.
  

Peter

[/quote]
To keep it simple:

• cache_size = nCPUs*(Connect+Additional), BOINC will try to maintain enough work in cache to keep computer busy for (Connect+Additional) days long (to overcome project outages, ISP maintenance intervals etc.),
  
• according to the source code, Connect every is low-bound to 0.00001, any lower value (like 0.00) si simply set to 0.00001 (a bit less than 1 second),
  
• Connect every value should be correctly used by projects to not send tasks with a deadline shorter than Connect every days, because your client is suggesting it might not connect the project any sooner (but some projects do not obey it), the value is also used by scheduler to decide, if any task(s) is(are) to be late and should be run with priority (in EDF mode),
  
• if a project sets wrong task estimates (like 2 instead of 36 hours), the project has to be blamed :-) If all their tasks behave like this, then BOINC tries to accomodate the rDCF factor accordingly - this may take up to a few tens of crunched results to settle.
  

Peter

[/quote]

Peter, thanks.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Boy, this is getting pretty deep for a guy like me. I translate that i should set Connect to 0.01, and leave Add at 0.00? Is that a correct interpretation?

>>RSM

Sekerob advises set Connect to 0.01 if you are on a permanent connection. For Additional he doesn't mention a specific number, he just says use Additional to steer the cache which means set it anywhere from 0 to 10 depending on your needs and computing habits. I see nothing wrong with that advice.

I set the Connect to 0.00 simply because when I hover the mouse pointer over the Connect box the tooltip hint advises to set it at 0.00 if always connected. To me that suggests that if it's 0.00 then BOINC is free to use the connection whenever it wants/needs to. I like that sort of unrestricted freedom. If it's not 0.00 then it seems to me BOINC is subject to a restriction I see no need for because I have ADSL not dialup. 0.00 or 0.01... I don't think it makes much difference. Experiment with both values, it can't hurt.

The Additional setting is, as Sekerob said, used to steer the cache. I set it no higher than .1 because some projects send tasks estimated to require 2 hours but they end up taking 36 hours. If I had Additional set to 1.00 then there is a chance I might cache several 36 hour tasks which would make 1 or more tasks late. If I set Additional = 0.00 then I find that I don't have enough work cached to keep the computer busy when my ISP goes offline.

If you set Additional to 0.00 and Connect to 0.01 then you won't cache much work. If your connection goes down due to ISP maintenance, storm or whatever, your computer may not have enough work to last until your connection comes back up. Only you know how reliable your ISP is so it's your decision.

To keep it simple:

• cache_size = nCPUs*(Connect+Additional), BOINC will try to maintain enough work in cache to keep computer busy for (Connect+Additional) days long (to overcome project outages, ISP maintenance intervals etc.),
  
• according to the source code, Connect every is low-bound to 0.00001, any lower value (like 0.00) si simply set to 0.00001 (a bit less than 1 second),
  
• Connect every value should be correctly used by projects to not send tasks with a deadline shorter than Connect every days, because your client is suggesting it might not connect the project any sooner (but some projects do not obey it), the value is also used by scheduler to decide, if any task(s) is(are) to be late and should be run with priority (in EDF mode),
  
• if a project sets wrong task estimates (like 2 instead of 36 hours), the project has to be blamed :-) If all their tasks behave like this, then BOINC tries to accomodate the rDCF factor accordingly - this may take up to a few tens of crunched results to settle.
  

Peter

The total of connect every X and extra work is low bounded at one second (in the code there is a function to retrieve the total, and it includes the statement if (total < 1) total = 1; (or something equivalent - I wrote it a while ago). Both can be set to 0 in 5.10.45. In some earlier versions connect every X had to be set to a positive number to avoid a crash.

---

BOINC WIKI

Great!! I'm there. Do they both get .1? The connect every is at 0.3000, the Additional is 0.00.

If you connect via dialup modem and it's on your voice line then you can use the connect every to limit connections so it interferes less with voice. With 0.30 it will connect to projects every 0.30 days and maintain enough work in the cache to keep your computer busy between connects.

The additional tells BOINC to cache that much work in addition to whatever it caches with the connect every setting. The units are days, not hours.

If you have a 24/7 connection or your dialup modem is on it's own line then I would set connect every to 0.00 which tells BOINC to connect to projects whenever it wants. I would set the additional to .1 or less.

Connect every X does not itself limit connections. You have to do that with either a schedule or by hand. Connect every X is a promise by the user to allow BOINC to connect to the projects at least that often. It does 2 things. First, and most obviously, it caches enough work to bridge that connection gap. Second, it attempts to finish all work earlier than the report deadline - (connect every X + task switch time + safety margin). If the task if finished before the report deadline, but after the last connection before the report deadline, it is going to be late anyway.

---

BOINC WIKI

That much I understood. And I understand the safety margin in the above formula is 24 hours, yes?

It depends on the version of your client, some it is the connect setting, some it is 24 hours. There is also an 80% fudge factor in some of the calculations.

If the task if finished before the report deadline, but after the last connection before the report deadline, it is going to be late anyway.

Maybe I don't understand this last sentence but that doesn't sound very user friendly... to have a finished task sitting there, the result presumably uploaded, needing only to be reported, and BOINC just sits there and allows the deadline to expire without reporting the task. I mean if the connect every X does not restrict the number or frequency of connections BOINC can make then why would BOINC not send the report before the deadline? Have I misunderstood what you're saying in that last sentence?

Remember the assumption with the connect setting is that the client will go at least this long between connections. So it will not be able to report in this case because there will not be a connection in time.

---

BOINC WIKI

BOINCing since 2002/12/8

Remember the assumption with the connect setting is that the client will go at least this long between connections.

You seem to be contradicting what JM7 said in post #17759 in this thread. He said, "Connect every X does not itself limit connections." You're saying it will go at least that long which means connections are limited by the setting. Sorry, I'm not trying to be argumentative, just trying to clarify. Maybe I'm taking JM7's words too literally?

The setting does not itself limit connections. However there is an assuption made that your computer will not be connected to the internet for the period in between, think dial-up or travelling laptops.

---

BOINC WIKI

BOINCing since 2002/12/8

I started this thread and I am glad it has gotten so much interest.

I made the change to .1, but I am not sure that this is the whole answer.

My interest was piqued when I was running two very long WU's on this Core 2 Duo machines. Slowly my queue of WU's from other projects emptied out and nothing was added by other projects. It is as if, and I am not speaking techically, because I do not know the language, it seems as if BOINC says, O.K., you have enough work to keep you busy, so you are not going to get anything new for a while.

I had a SETI WU of about 39 hours, and a QMC WU of somewhere of 30 hours.

But, then, after maybe 9 or 10 hours, the SETI WU had a computational error and quit. This was the second time that SETI had this problem. So, I was left with work for one CPU. BOINC did not wake up to the fact that I was running out of work for quite some time.

I have detached from SETI, as I do not want any more to waste time.

I had been set at .3, which appears to be the default setting, and I had no problems until this coincidence of two very long WU's.

Now I have 15 hours left on the QMC WU and BOINC has added a WU from each of four other projects.

I am probably going to let things sit for a while; but I may go back to the .3 setting as I really see no reason to alter BOINC's defaults. I am on two very fast ISP's, not dial up.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

Thanks. I am glad you responded. I am sure that it is somewhere else in this thread, but I now have it clearly banged into place that the .3 setting is based upon the weakest link, so to speak, which is dial up.

As I said, I am going to stick with the .1 for a while and see what happens.

The two SETI incidents just burned my britches. I wasted about 20 hrs between the two events. For me, it is not credits, it is being part of important research and getting work done. I have great respect for Dave Anderson and the SETI@home team; but I am just not going to waste my resources like that.

Thanks again, everyone has been so patient.

>>RSM

---

http://sciencesprings.wordpress.com
http://facebook.com/sciencesprings

My interest was piqued when I was running two very long WU's on this Core 2 Duo machines. Slowly my queue of WU's from other projects emptied out and nothing was added by other projects. [...] I had a SETI WU of about 39 hours, and a QMC WU of somewhere of 30 hours.

But, then, after maybe 9 or 10 hours, the SETI WU had a computational error and quit. This was the second time that SETI had this problem. So, I was left with work for one CPU. BOINC did not wake up to the fact that I was running out of work for quite some time.

Probably BOINC was observing the connect every X setting and not connecting until X time had passed but it depends what you mean by "quite some time"... was it 1 hour? 2 hours? 10 hours?

No. In the case a 2 CPU host was left with one WU, BOINC should have immediately ask for additional work. Not doing so is a client error.

I have detached from SETI, as I do not want any more to waste time.

That's a sad coincidence, as I've had no SETI errors for a pretty long time (months? years?) and such occasional error could happen to any project. (BTW, 39 hours on C2D - that must have been SETI beta?)

Now I have 15 hours left on the QMC WU and BOINC has added a WU from each of four other projects.

When did it actually happen?

Peter

Probably BOINC was observing the connect every X setting and not connecting until X time had passed but it depends what you mean by "quite some time"... was it 1 hour? 2 hours? 10 hours?

No. In the case a 2 CPU host was left with one WU, BOINC should have immediately ask for additional work. Not doing so is a client error.

There we have another facet of the confusion I've been trying to clarify. Keck and McLeod say BOINC assumes it cannot connect more often than the period set in connect every X, not even to report a task that has been completed and uploaded and in danger of exceeding the deadline. But it will ignore that assumption to get more work? Huh? If it can ignore the assumption to get more work then why can it not ignore the assumption to report a completed task on time. Seems reporting completed work on time is just as important as getting new work.

No, Dagorath, you possibly misunderstood, what they said. BOINC hopes it will be able to connect at least that often (because the user promised it). But any task considered being in deadline danger will be reported imediately after being crunched. If e.g. your connect interval is set to say 5 days, the result will be reported even more than these 5 days prior deadline. And any idle core will be immediately supplied with work. (Except that you (well, mitrichr) have revealed a bug in the client.)

Peter