Posts by Raistmer

1) Message boards : Projects : Open-source projects (Message 104414)
Posted 19 May 2021 by Raistmer
Post:
I was wondering which BOINC projects are open-source? Thanks


This is a list of open souce projects

Definitely needs update.

2) Message boards : Questions and problems : What is "virtual memory" in Boinc? (Message 103954)
Posted 12 Apr 2021 by Raistmer
Post:
I think BOINC just lists corresponding OS counters for those values, nothing more.
And to have them listed is useful for debugging, managing what tasks fit for what PC.
In Windows all "usual" app memory is virtual. So working set size always should be less than "virtual, or swap" size. Even if in reality HDD swap was not used.
In task manager this value called "commit size".

Boinc doesn't manage it. Just as with RAM itself. But, if threshold exceeded, it can suspend task (for example, user could use this as guard against excessive HDD usage with swapping).
3) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103692)
Posted 25 Mar 2021 by Raistmer
Post:
Set CPU usage to 1 as FGRP does and you get full core reserved - no probs.
But having ability not to reserve full core by specifying less than 1 is useful.
The choice always better than no choice (and tendency to program in a manner " I know better" usually result in less useful apps, compare BOINC for Android and NativeBOINC when it was supported on OS).
4) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103677)
Posted 23 Mar 2021 by Raistmer
Post:
0.9 = 0 (not 0.9)
0.9+0.9 = 1 (not 1.8)
0.9+0.9+0.9 = 2 (not 2.7)
That's exactly correct. BOINC 'overcommits' the CPU by 'not more than one complete core'.

Actually it's quite OK behavior.
The alternative is to leave core almost idle if any GPU task in progress.
If GPU app really need full core for support it just provides 1.
5) Message boards : Questions and problems : GPU tasks skipped after scheduler overcommits CPU cores (Message 103672)
Posted 22 Mar 2021 by Raistmer
Post:
Yep, the choice between over-fetch and idle cores - the sad choice...

BOINC many years suffers from wrong "atomic entity" definition, I would say.
First time I said that 15 (?) years ago, at first approach of GPU computing, it was in mail list those times, not only on forums...
There are many ad-hoc additions since those times but no real re-write with re-design. And it's just as needed as before.
The "atomic entity" here is app_version/plan class. Not the project. And still we suffer from project-centric initial approach.
It's just everywhere. From per-project server requests to per-project shares.
Initially there was only 1 app per project (and just single project - SETI ). And "atomic project" was == "atomic app" (it appeared so cause no other apps were there).
But then AstroPulse added... and whole Credit system gone mad. Then GPGPU emerged - wow, 2 different devices in single host just uncomparable by computing abilities... And so on...
BOINC manages not projects, it manages tasks for particular specific apps as atomic entities.
And those apps then grouping into different groups by resource usage and by project owning....

What I mean - all that can be done between 2 apps belonging to 2 different projects should be able to do between 2 apps belonging to single project.
And it's definitely not the case still...
6) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103666)
Posted 22 Mar 2021 by Raistmer
Post:
would that solve the problem?

If MW can provide as many tasks (task takes 10 minutes, if it has hardwired limit on tasks in fly per host this will not work) as needed, then yes.

I'll try such config and will see if it will solve GPU part of problem.
7) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103664)
Posted 22 Mar 2021 by Raistmer
Post:
"0.9CPU"

as far as I am aware, this value is only used for BOINC bookkeeping on used resources. this value cannot influence/limit/dictact what is ACTUALLY being used. you could change this value to 0.01, and the actual use for this task will remain unchanged, however BOINC will think you have more free CPU resources than you actually do, leading to the possibility to over-commit CPU resources depending on other CPU projects and CPU use settings you have.


Fully agree. But if app can work w/o reserved CPU core (under "can work" I mean not technical ability that of course exist always as far as we speak about not real-time processing but rather "work w/o big noticeable slowdown") project usually configures it as demanding less than 1 CPU. That allows full CPU cores load by CPU apps in BOINC.
Compare GW app (0.9CPU (maybe incorrectly)+ 1GPU), FGRP (1CPU+1GPU) and MW app (0.83CPU+1GPU, correctly, no noticeable slowdown while all cores busy, ~99% GPU load through all task processing).
Hence, looking on 0.9CPU setting, I came to (perhaps wrong) conclusion.
8) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103661)
Posted 22 Mar 2021 by Raistmer
Post:
He said in the previous thread that he wanted Einstein to run on one of the {CPU|NV} devices - I forget which - but not the other. That rules out RS zero.

The previous thread also wrote:
1 GPU MW task that doesn't need full CPU core!
What makes you think that MW has cracked the busy-wait OpenCL problem on NVidia?


I guess I don't understand the intended behavior or what's trying to be accomplished. if he doesn't want one device to run a certain project, you can just use the <exclude_gpu> flag in cc_config, and either exclude by the project as a whole, or exclude by the application.


I'll try to describe "what &why":
0) Both CPU and GPU: due to my router hangs time to time I prefer to have some local cache of work for both CPU and GPU.
1) CPU part
I prefer (from personal scientific declination) to participate mostly in GW search (also I consider it more resource-demanding so not all hosts can do it and need help from those who can), but host in question can't deal with 4 CPU GW tasks at once (low memory issue). Hence I attempt to limit number of GW tasks simultaneously in fly not disabling them completely.
So I attempted to use max_concurrent tag in app_config for GW app. Result - failure. BOINC inadequately implements this mode of operation [that's topic for another thread actually]

2) GPU part.
GW NV unsupported on my GPU, FGRP NV has strong effects on sound quality and host response. So I crunch more "easy" app on GPU - MW NV.
Cause MW time to time has no work I attempt to configure E@h as "backup" project for GPU part. But setting it backup in BOINC sense (0 share) will contradict with 0).
Result - failure. Cache slowly (not too slowly actually) fills with E@h NV tasks so only them are crunching on GPU eventually causing disturbances in my own work more often I'm ready to tolerate.
9) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103660)
Posted 22 Mar 2021 by Raistmer
Post:

The previous thread also wrote:
1 GPU MW task that doesn't need full CPU core!
What makes you think that MW has cracked the busy-wait OpenCL problem on NVidia?

Hm, my SETI NV OCL build didn't need full CPU too as far as I could recall. It's the question of kernel size and parameters of the call.
Moreover, E@h GW NV build doesn't require busy-loop also [accordingly its 0.9CPU config]. Hardly it's CUDA one, E@h strongly prefer OpenCL...
Only FGRP search currently requires full CPU core.


nvidia GPU app for GW does indeed use a full CPU core. in most cases it actually requires MORE than 1 full core.

pull up top or htop while it's running and you'll see more than 100% of 1 core being used, i've seen up to 150%. since the GPU job is offloading some work to the CPU.


Well, I didn't study GW NV app work by myself so far (it runs in another host, my "on desk" one can't process it). So I based exclusively on its configuration, 0.9CPU+1GPU.
Maybe wrong config from server deceived me, will not argue here.

Nevertheless the main idea remains - it's possible to work w/o busy-wait loop. Will it bring benefit or not strongly depends on size of used kernels inside app.
10) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103655)
Posted 22 Mar 2021 by Raistmer
Post:
BTW, there is some peculiarity in MW server configuration adding complexity. As soon as I suspended all E@h NV tasks cache was immediately filled with MW tasks. Coincidence?
At least it means that MW has work quite often and if BOINC would ask for it....

3/22/2021 19:38:54 PM | Milkyway@Home | checking NVIDIA GPU
3/22/2021 19:38:54 PM | Milkyway@Home | [work_fetch] set_request() for NVIDIA GPU: ninst 1 nused_total 0.00 nidle_now 1.00 fetch share 1.00 req_inst 1.00 req_secs 138240.00
3/22/2021 19:38:54 PM | Milkyway@Home | NVIDIA GPU set_request: 138240.000000
3/22/2021 19:38:54 PM | Milkyway@Home | [work_fetch] request: CPU (0.00 sec, 0.00 inst) NVIDIA GPU (138240.00 sec, 1.00 inst)
3/22/2021 19:38:54 PM | Milkyway@Home | Sending scheduler request: To fetch work.
3/22/2021 19:38:54 PM | Milkyway@Home | Requesting new tasks for NVIDIA GPU
3/22/2021 19:38:55 PM | | [work_fetch] Request work fetch: application exited
3/22/2021 19:38:57 PM | Milkyway@Home | Scheduler request completed: got 206 new tasks
3/22/2021 19:38:57 PM | Milkyway@Home | Project requested delay of 91 seconds
3/22/2021 19:38:57 PM | | [work_fetch] Request work fetch: RPC complete
11) Message boards : Questions and problems : GPU tasks skipped after scheduler overcommits CPU cores (Message 103654)
Posted 22 Mar 2021 by Raistmer
Post:
Currently (as was expected) 2 GW CPU tasks + 1 NV FGRP task. 1 core sits completely idle.
And no MW tasks on host....
Seems I have no other way as to set E@h as backup (zero share) project again and crunch w/o cache on my second-power host...
Accounting for instability of my current router this almost definitely means idle host time :/

Richard, if no more info about modded build required I prefer to return to stock one cause it will ask for work when CPU is idle.
12) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103653)
Posted 22 Mar 2021 by Raistmer
Post:

The previous thread also wrote:
1 GPU MW task that doesn't need full CPU core!
What makes you think that MW has cracked the busy-wait OpenCL problem on NVidia?

Hm, my SETI NV OCL build didn't need full CPU too as far as I could recall. It's the question of kernel size and parameters of the call.
Moreover, E@h GW NV build doesn't require busy-loop also [accordingly its 0.9CPU config]. Hardly it's CUDA one, E@h strongly prefer OpenCL...
Only FGRP search currently requires full CPU core.
13) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103652)
Posted 22 Mar 2021 by Raistmer
Post:
why not set Einstein to 0 instead of 1. then it will act as a true backup and only request work when MW is out of work.

That way I will go w/o CPU cache at all. Cause E@h the only who does CPU work on that host. MW configured as GPU-only (old NV GPU barely cope with E@h tasks causing noticeable host slowdown, no such effects on MW tasks).
14) Message boards : Questions and problems : Black screen / Win Update? BOINC not sharing nicely. (Message 103638)
Posted 21 Mar 2021 by Raistmer
Post:

Apart from me creating a Python script to set BOINC Activity to Suspend whenever WinUpdate is triggered, is there something I can do?


You could check how much memory your current BOINC projects use.
Especially with multi-core host and especially with Einstein@home project.
I found their current GW search impossible to support with quad CPU and 8GB RAM, time to time OS will crash due to low memory.
BOINC has an option to limit RAM in use so try to use it (Windows update tends to eat lot of memory too while running).
15) Message boards : Android : BOINC & Android TV (Message 103636)
Posted 21 Mar 2021 by Raistmer
Post:
Hm, not quite understand the core of issue.
BOINC for Android shouldn't behave so differently than BOINC for let say Windows.
And for example E@h has preferences on site to enable or disable GPU work for particular venue per particular app type...

And of course before release in wild alpha then beta testing should be carried out.
To evaluate heating amongst other possible issues too.
Not every phone can support heat dissipation from CPU-only processing, but nevertheless BOINC moved to mobiles... Don't see too big difference with GPU.
Ultimately user decides to use BOINC or not. And (just as E@h project example shows) project that pioneered GPU mobile processing could make it "opt-in" instead of "opt-out" (agreed that to wake some good morning and to find phone being fried because of unattended GPU app install isn't good move ). BOINC not M$ to throw bad updates on innocent peoples...
But for those who willing to try such option would be interesting IMO....
16) Message boards : Android : BOINC & Android TV (Message 103634)
Posted 21 Mar 2021 by Raistmer
Post:
But modern heavy-graphics mobile games provide comparable to crunching load...
17) Message boards : Questions and problems : BOINC unable to honor project shares (at all, not only in short run) (Message 103633)
Posted 21 Mar 2021 by Raistmer
Post:
Example:

Project shares:

Einstein@home: 1
MilkyWay: 10000

What could be expected (I started this discussion with PM exchange with Richard and he noticed that project share should be considered (in BOINC current paradigma) not as processing time share but rather credit share, but BOTH metrics can't be honored!):
MW runs almost always on GPU and E@h receives tasks very rare, when MW has prolonged "no work" stages.

What I see in reality:
I prepared "initial state" for this experiment as most easy to follow:
No E@h GPU tasks in queue; queue filled with MW GPU tasks.
[work_fetch] target work buffer: 129600.00 + 8640.00 sec

After a week:
15 E@h GPU tasks in queue.
RAC (note, that E@h host RAC includes CPU, so situation even worse that pictured):



Absolutely not 1:10 000. And will not be in near future (cause those 15 E@h tasks in cache worth >~150 MW tasks and they should be processed once downloaded!).
And I state that requested share never will be and can't be honored in current work fetch design.

When work is needed BOINC asks for work from MW first (as should be). But MW not as reliable in work providing as E@h is.
So time to time no work given. Then BOINC immediately (and that's NOT OK !) asks for work from E@h and get it.
Situation repeated when cache is lowering again.

Hence, only possible share will be the ratio of MW having work ready to send. It has absolute no connection not with processing time share nor with credit/RAC share.

What should be changed to fix this situation: there are 2 parts of work buffer: "main" and "additional".
Currently BOINC asks for work only when "main" in shortage and asks shortage + "additional". It's wrong behavior (!).
It should ask when some % of "additional" in shortage. And ask ONLY from high priority project until there is no shortage in "main". Then it should ask from all projects (in priority order of course).

Such sequence will give high priority project a chance to provide work accordingly its share. In current state all that "sophisticated" work fetch simulation just can't provide adequate results.
18) Message boards : Questions and problems : GPU tasks skipped after scheduler overcommits CPU cores (Message 103629)
Posted 21 Mar 2021 by Raistmer
Post:
Running Richard's bugfixed build with work fetch through last week.
Initial bug definitely fixed - there is no overfetch.
Unfortunately, the signs of other issue I mentioned earlier getting stronger:
Currently host has only GW tasks in cache + 1 running FGRP task. Cause only 2 GW tasks allowed at once and host has 4 cores there will be idle ones soon...

And GPU part suffers from inability to honor project shares. But this issue worth separate thread.


EDIT: Unfortunately, it's not "signs", it 's happened already...



2 GW tasks, 1 FGRP task... and 1 GPU MW task that doesn't need full CPU core! So, 1 CPU core already idle. When FGRP task finishes there will be 2 cores with high probability....

So, this bugfix isn't enough to use max_concurrent as expected.

EDIT2: And BOINC doesn't react on idle device (CPU):

3/21/2021 22:48:01 PM | | [work_fetch] ------- start work fetch state -------
3/21/2021 22:48:01 PM | | [work_fetch] target work buffer: 129600.00 + 8640.00 sec
3/21/2021 22:48:01 PM | | [work_fetch] --- project states ---
3/21/2021 22:48:01 PM | Einstein@Home | [work_fetch] REC 19077.814 prio -6075.662 can request work
3/21/2021 22:48:01 PM | Milkyway@Home | [work_fetch] REC 12328.567 prio -0.513 can request work
3/21/2021 22:48:01 PM | SETI@home Beta Test | [work_fetch] REC 0.000 prio 0.000 can't request work: suspended via Manager
3/21/2021 22:48:01 PM | | [work_fetch] --- state for CPU ---
3/21/2021 22:48:01 PM | | [work_fetch] shortfall 0.00 nidle 0.00 saturated 153096.94 busy 0.00
3/21/2021 22:48:01 PM | Einstein@Home | [work_fetch] share 1.000
3/21/2021 22:48:01 PM | Milkyway@Home | [work_fetch] share 0.000 blocked by project preferences
3/21/2021 22:48:01 PM | SETI@home Beta Test | [work_fetch] share 0.000
3/21/2021 22:48:01 PM | | [work_fetch] --- state for NVIDIA GPU ---
3/21/2021 22:48:01 PM | | [work_fetch] shortfall 0.00 nidle 0.00 saturated 139709.31 busy 0.00
3/21/2021 22:48:01 PM | Einstein@Home | [work_fetch] share 0.000
3/21/2021 22:48:01 PM | Milkyway@Home | [work_fetch] share 1.000
3/21/2021 22:48:01 PM | SETI@home Beta Test | [work_fetch] share 0.000
3/21/2021 22:48:01 PM | | [work_fetch] ------- end work fetch state -------

Manual update didn't help (as expected):

3/21/2021 22:50:17 PM | Einstein@Home | piggyback: resource CPU
3/21/2021 22:50:17 PM | Einstein@Home | piggyback: don't need CPU
3/21/2021 22:50:17 PM | Einstein@Home | piggyback: resource NVIDIA GPU
3/21/2021 22:50:17 PM | Einstein@Home | piggyback: don't need NVIDIA GPU
3/21/2021 22:50:17 PM | Einstein@Home | [work_fetch] request: CPU (0.00 sec, 0.00 inst) NVIDIA GPU (0.00 sec, 0.00 inst)
3/21/2021 22:50:17 PM | Einstein@Home | Sending scheduler request: Requested by user.
3/21/2021 22:50:17 PM | Einstein@Home | Not requesting tasks: don't need (CPU: job cache full; NVIDIA GPU: job cache full)
3/21/2021 22:50:18 PM | Einstein@Home | Scheduler request completed
3/21/2021 22:50:18 PM | Einstein@Home | Project requested delay of 60 seconds
3/21/2021 22:50:18 PM | | [work_fetch] Request work fetch: RPC complete
3/21/2021 22:50:23 PM | | choose_project(): 1616356223.263348
3/21/2021 22:50:23 PM | | [work_fetch] ------- start work fetch state -------
3/21/2021 22:50:23 PM | | [work_fetch] target work buffer: 129600.00 + 8640.00 sec
3/21/2021 22:50:23 PM | | [work_fetch] --- project states ---
3/21/2021 22:50:23 PM | Einstein@Home | [work_fetch] REC 19076.165 prio -15117.019 can't request work: scheduler RPC backoff (54.94 sec)
3/21/2021 22:50:23 PM | Milkyway@Home | [work_fetch] REC 12330.310 prio -1.120 can request work
3/21/2021 22:50:23 PM | SETI@home Beta Test | [work_fetch] REC 0.000 prio 0.000 can't request work: suspended via Manager
3/21/2021 22:50:23 PM | | [work_fetch] --- state for CPU ---
3/21/2021 22:50:23 PM | | [work_fetch] shortfall 0.00 nidle 0.00 saturated 153031.73 busy 0.00
3/21/2021 22:50:23 PM | Einstein@Home | [work_fetch] share 0.000
3/21/2021 22:50:23 PM | Milkyway@Home | [work_fetch] share 0.000 blocked by project preferences
3/21/2021 22:50:23 PM | SETI@home Beta Test | [work_fetch] share 0.000
3/21/2021 22:50:23 PM | | [work_fetch] --- state for NVIDIA GPU ---
3/21/2021 22:50:23 PM | | [work_fetch] shortfall 0.00 nidle 0.00 saturated 139559.98 busy 0.00
3/21/2021 22:50:23 PM | Einstein@Home | [work_fetch] share 0.000
3/21/2021 22:50:23 PM | Milkyway@Home | [work_fetch] share 1.000
3/21/2021 22:50:23 PM | SETI@home Beta Test | [work_fetch] share 0.000
3/21/2021 22:50:23 PM | | [work_fetch] ------- end work fetch state -------
19) Message boards : Android : BOINC & Android TV (Message 103628)
Posted 21 Mar 2021 by Raistmer
Post:
I see strong contradiction between reality (BOINC for Android isn't set and forget one) and its interface design.
BOINC for Android interface designed to provide as little tuning options as possible. Needed for just to run it options hide into "advanced" options set.
I have no probs with software that needs tuning and admits it, providing as many tunable params as it can. Actually it can even provide some fun.
But pretending to be "set and forget" with minimalistic interface....

Well, at least it runs eventually.
So BOINC can run on AndroidTV platform. These TV boxes, having 4-core ARM processors, provide definitely not less computing power than average phone.

Next step would be to use their GPU part... About year ago the answer on the question "is any project uses GPU under Android" was "no". Any changes from that time?....
20) Message boards : Android : BOINC & Android TV (Message 103624)
Posted 21 Mar 2021 by Raistmer
Post:
Well, question is quite rhetoric cause TV box has no battery at all just as stated.

So BOINC reads some temp sensor (or no sensor at all!) and ERRONEOUSLY links it to battery temp sensor.

Message appears IMMEDIATELY after run, not even after task started, but BEFORE task started.
So consider it just as BUG REPORT for BOINC for Android 7.16.16 (there will be no operation at all unless operator takes some actions).

Currently I circumvented this bug by setting allowed temperature unimpossible high. Computations started so there is some hope BOINC looked at some real temp sensor, not just got max_int from non-existent one.


Next 20

Copyright © 2022 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.