Account Managers and Project URLs

Message boards : Questions and problems : Account Managers and Project URLs
Message board moderation

To post messages, you must log in.

AuthorMessage
Gary Roberts

Send message
Joined: 7 Sep 05
Posts: 130
Australia
Message 88990 - Posted: 22 Nov 2018, 22:30:34 UTC

The background to this is contained in this thread at Einstein. Briefly, the essence of the problem was explained by Christian Beer in this message from that thread. I'm not a programmer and I don't claim to fully understand the problem, or why it seems so intractable. However, I can't imagine the suggestion that the use of a different 'project_config' type URL entered manually to get at the correct master URL, is going to fly with the average volunteer. I'm not even really sure if that was exactly what Christian was suggesting.

I'm asking for help on this because of this recent thread where yet another volunteer is being troubled by this issue - or so it seems. I'm not 100% sure because the person concerned is obsessed with his particular impression of the problem and is fixated on his need to "delete a duplicate (old) account" and can't even say for sure if he used an account manager to install BOINC on his newest machine.

I have zero experience with account managers so my inability to help this person is probably just a case of 'the blind trying to lead the blind'. The Einstein admins must be fully aware of this so I really don't understand why something can't be done about it to fix it permanently, somehow. It could easily be that I have totally misunderstood the issue. If so, and if there is any kind soul who knows what to do to stop the notices that are troubling this volunteer, I'd certainly appreciate the advice.
Cheers,
Gary.
ID: 88990 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 88991 - Posted: 22 Nov 2018, 22:45:16 UTC - in response to Message 88990.  

Christian wrote an even fuller explanation of the problem in #2642. I suspect the user problem is not helped (though without checking it at this time of night) by BOINC using the error message "using an 'old' url" when it simply means 'different'.
ID: 88991 · Report as offensive
Gary Roberts

Send message
Joined: 7 Sep 05
Posts: 130
Australia
Message 88992 - Posted: 22 Nov 2018, 23:41:13 UTC - in response to Message 88991.  

Richard,
Thanks very much for your response. There's no particular urgency so please get your sleep.

Thanks for the github link. I have a basic understanding of the issue as explained there. The snippet "(because it is not possible to just change this without all users re-attaching to the project)" helped a lot :-).

What's unclear is whether or not any changes made within the BOINC stable will fix this problem for people using boinctasks/account managers on top of BOINC. Are we just supposed to tell people either to (a) ignore the notices until a future release stops them coming, or (b) stop using account managers/boinctasks/boinccmd to attach and just use the standard wizard instead?
Cheers,
Gary.
ID: 88992 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89004 - Posted: 23 Nov 2018, 18:40:45 UTC

OK, I'm slowly going to get back up to speed after a period of distraction. In no particular order:

1) Confirmed that the message says 'old' (for url), when it really means 'different' or 'unexpected'.
https://github.com/BOINC/boinc/blob/f14d96d2be2e5a290177f7b5391b0802fbd3756c/client/cs_scheduler.cpp#L617

2) Despite the urgency (red letter alert) of the message, you can ignore it, and things will just burble along as usual. I've got one like that.

3) For people who have got themselves into a mess and want to sort themselves out, the 'correct' way to identify the 'correct' url is to stick 'get_project_config.php' into the address bar of a browser after the url of the project website. Thus,
https://einsteinathome.org/get_project_config.php
returns (among other guff)
<master_url>http://einstein.phys.uwm.edu/</master_url>
That one's a keeper.

After another sleep, I'll have a go at

4) Can I prove that some AM out there is attaching using a url that doesn't comply with (3) above?
5) If so, can I recreate the 'attached twice to the same project' error?
6) And if I succeed, can I work out a procedure for cleaning up the mess?
ID: 89004 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89006 - Posted: 23 Nov 2018, 20:22:57 UTC

OK, since Account Managers aren't well understood by the helpdesk regulars (certainly not by me), I'll park these here for reference.

https://boinc.berkeley.edu/trac/wiki/AccountManagement
https://boinc.berkeley.edu/wiki/Account_managers
ID: 89006 · Report as offensive
Gary Roberts

Send message
Joined: 7 Sep 05
Posts: 130
Australia
Message 89008 - Posted: 24 Nov 2018, 4:47:37 UTC - in response to Message 89004.  

Please don't waste time on this on my account. Your second point suits my purposes very well.

2) Despite the urgency (red letter alert) of the message, you can ignore it, and things will just burble along as usual. I've got one like that.

I will simply advise the volunteer that he shouldn't be distressed about a frightening message as it is nothing to do with any type of 'old' account that he needs to delete. I'll just point him to the above quote and tell him once again just to ignore it. As his affected computer has yet another new ID and seems to be returning work OK at the moment, I suspect he has moved on from the problem anyway.

3) For people who have got themselves into a mess and want to sort themselves out, the 'correct' way to identify the 'correct' url is to stick 'get_project_config.php' into the address bar of a browser after the url of the project website. Thus,
https://einsteinathome.org/get_project_config.php
returns (among other guff)
<master_url>http://einstein.phys.uwm.edu/</master_url>
That one's a keeper.

I've been trying to make sure I properly understand what Christian wrote in #2642. I'll have a go at paraphrasing what it means to me. The critical bit seems to be that if the 'attach to project' operation is used from BOINC Manager, it uses the website URL as a base for getting the contents of get_project_config.php which then allows the correct master URL to be determined from what is returned (just as you've done above).

If boinccmd --project_attach URL account_key is used, the URL in that command is the website URL and boinccmd passes that same URL to the client as the master URL - which, in the case of Einstein, it is not. In other words, for any project like Einstein with two different URLs, boinccmd needs to do what the attach wizard does and get the correct master URL from get_project_config.php and then pass that master URL to the client so that the "scheduler_url can be scraped from the HTML" as Christian described it.

Hopefully, I've got that correctly understood. The extra bit (which is more conjectural) is that external entities like AMs, etc., may just be doing the same as boinccmd - assuming the URL is both project URL and master URL and not allowing for the possibility that a project might have two different URLs. In that case, I'd be very interested in what you find if you do continue on with 4), 5), and 6). That seems like a very good approach to putting this all behind us once and for all. My impression (from something someone said a while ago about BAM - I think) was that AMs only dealt with one URL for both, so tough luck. I hope it's not too messy in finding a suitable compromise that suits all parties :-).
Cheers,
Gary.
ID: 89008 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89016 - Posted: 24 Nov 2018, 18:20:19 UTC

That's OK - I'm using my own time, as a follow-up to things I partially learned while acting as release manager for v7.10

And here's a nice little log snippet:

24/11/2018 17:34:00 |  | Contacting account manager at http://www.gridrepublic.org/
24/11/2018 17:34:02 |  | Account manager contact succeeded
24/11/2018 17:34:02 |  | Attaching to http://climateprediction.net/
24/11/2018 17:34:04 | http://climateprediction.net/ | sched RPC pending: Project initialization
24/11/2018 17:34:04 | http://climateprediction.net/ | [sched_op] Starting scheduler request
24/11/2018 17:34:04 | http://climateprediction.net/ | [sched_op] Fetching master file
24/11/2018 17:34:09 | http://climateprediction.net/ | [sched_op] Got master file; parsing
24/11/2018 17:34:09 | http://climateprediction.net/ | [sched_op] Found 1 scheduler URLs in master file
24/11/2018 17:34:09 | http://climateprediction.net/ | Master file download succeeded
24/11/2018 17:34:14 | http://climateprediction.net/ | sched RPC pending: Project initialization
24/11/2018 17:34:14 | http://climateprediction.net/ | [sched_op] Starting scheduler request
24/11/2018 17:34:14 | http://climateprediction.net/ | Sending scheduler request: Project initialization.
24/11/2018 17:34:14 | http://climateprediction.net/ | Requesting new tasks for CPU and NVIDIA GPU
24/11/2018 17:34:14 | http://climateprediction.net/ | [sched_op] CPU work request: 1.00 seconds; 0.00 devices
24/11/2018 17:34:14 | http://climateprediction.net/ | [sched_op] NVIDIA GPU work request: 1.00 seconds; 0.00 devices
24/11/2018 17:34:16 | climateprediction.net | Scheduler request completed: got 0 new tasks
24/11/2018 17:34:16 | climateprediction.net | [sched_op] Server version 707
24/11/2018 17:34:16 | climateprediction.net | This project is using an old URL.  When convenient, remove the project, then add https://climateprediction.net/

https://www.climateprediction.net/get_project_config.php returns
<master_url>https://climateprediction.net/</master_url>
but acct_mgr_reply.xml contains
<url>http://climateprediction.net/</url>

That little difference between http and https is enough to trigger the error message - and in this case, the epithet 'old' is probably correct.

Trying a manual attach (not under AM control) produced

24/11/2018 17:49:50 | climateprediction.net | Already attached to a project named climateprediction.net (possibly with wrong URL)
The 'manual attach' account (with the correct url) can be removed through BOINC Manager, but the GR account (with the bad url) can't - 'Remove' button is disabled. I think this is the original Einstein state that prompted this thread.

Shutting down BOINC and restarting it reduced the project count to 1, but as often as not it deleted the wrong account. And 'Synchronising' with the Account Manager (that's a specific reserved word for AM interactions) re-creates the account with the bad url.

For an account created via an AM, client_state.xml contains the line
<attached_via_acct_mgr/>
(about 40 lines into the <project> specification, in this case). Removing this line - something NOT recommended for inexperienced users - allows the project to be removed using the standard BOINC Manager button. And 'Stop using Grid Republic..." (i.e. doing away with AM operation, and returning to the native BOINC mode of operation we're familiar with) should stop it coming back.

------------------------------------
I've tested BAM! as well, and that sent out the correct Master URL for both Climate Prediction and Einstein. Grid Republic (the website) couldn't contact Einstein today (probably used a bad url :P), so wouldn't allow me to test attachment to Einstein. Now for another break, and then I'll see how Science United and Gridcoin respond to the same AM stress tests.
ID: 89016 · Report as offensive
Gary Roberts

Send message
Joined: 7 Sep 05
Posts: 130
Australia
Message 89022 - Posted: 25 Nov 2018, 3:33:11 UTC - in response to Message 89016.  

Richard, you deserve a huge pay rise!! :-).

It was really helpful to now know exactly why the EAH volunteer sees the 'greyed out' button and how it could be rectified by deleting the line in the state file. If it were my host I'd have no hesitation but I don't think it would be wise to push it for this particular case. If anyone else comes along with the same issue, I might offer the advice but with obligatory warnings about potential damage from incorrect editing, etc.

Thank you so much for the detailed explanation. I'd just like to clarify my understanding of why BAM was able to correctly handle both CPDN and Einstein, but GR wasn't.

For the case of GR, (and despite using http://climateprediction.net/ rather than https) the master file was successfully fetched and the scheduler URL extracted - or so it seems. Presumably that URL would have been correct. It's not clear to me exactly why there was a 'no work' outcome. Was it anything to do with http/https mismatch or was it just expected from the 0.00 devices? Irrespective of that question, exactly what triggered the "This project is using an old URL" message? If (as I presumed) the correct scheduler URL had been parsed, something else must have noted the two different versions of the master URL?? Perhaps I'm just unnecessarily overthinking all this.

I imagine you may have followed a similar procedure when testing how BAM would behave. I even imagine you may have a similar log snippet where the first line reads, "Contacting account manager at https://bam.boincstats.com". If so, was the only difference just the correct use of https at all stages with no complaint line at the end?

For Einstein, can we assume that BAM now succeeds because it parses get_project_config.php to find the correct master URL and is able to handle the project URL being different to the master URL? In other words, does this imply that GR just needs to do something similar?
Cheers,
Gary.
ID: 89022 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89024 - Posted: 25 Nov 2018, 13:39:40 UTC

To answer those two, first read (very forensically) the developer Wiki on Account Managers I linked earlier (here it is again).

The section Account manager RPCs talks about 'a get_project_config.php file', but that's for your client to attach to the Account Manager. It's nothing to do with verifying the project url.

Lower down, in the 'action' section, read

projects: A list of currently attached projects. For each project:
url: the project's master URL
(my emphasis)
Because neither the AM itself, nor the client when told to attach to a project, does a second get_project_config to confirm the url before attaching, it is CRITICAL that this is accurate. Account manager administrators aren't warned about this, or about the precise technical meaning of a MASTER url. All the problems so far arise from outdated information in this field.

BAM, to give Willy due credit, got both CPDN and Einstein right. Because BAM supports a wider range of projects than the master list supplied by BOINC, I suspect Willy maintains his own list manually, and is very precise in his actions - a man after my own heart. Others are more slapdash.

As to CPDN not sending work - frankly, I'm glad it didn't. I'm using a test BOINC account and data folder for this, and I wouldn't want to be stuck with a 4-month simulation until the Spring! Given all the problems Oxford has simply running a stable set of servers with stable BOINC code, I expect they simply didn't have any work to send at the critical moment.

BTW, I'd avoid Grid Republic if I were you. It tries to install an ancient version of BOINC (v6.8.44, dated 22 August 2012), and doesn't give you any choice about where to install it. But it does pick up and try to re-use the folder locations of your regular BOINC installation. I'm an experienced Windows user and developer (I've even run Windows Servers with Active Directory security and user account management), and it took me four or five attempts to get the installer to run past the various error messages. And the guidance link for what to put in the account/password boxes takes you direct to Facebook.

So far, Science United seems to have the right project data for both CPDN and Einstein - though I'm slightly surprised to find Primegrid classified as Physics (or possibly Earth Sciences). And it keeps re-attaching the same projects, and over-riding my 'no new tasks' setting.
ID: 89024 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89025 - Posted: 25 Nov 2018, 16:47:28 UTC

Science United testing complete. Took longer than I expected - not only did it keep attaching new projects, but it succeeded in detaching one project (PrimeGrid), reattaching, and fetching new work - all while preserving the 'No New Work' flag. That's not nice. And if it uses a new randomised account name for each re-attach (hard to tell), that's a lot of spamming of project user tables.

Another point: while attached to an Account Manager, you're expected to use the AM to do all project attachments and similar manoeuvres via the AM website. But if you happen to discover BOINC Manager and open it, the website address isn't visible anywhere - just the normal range of project buttons. The only clues that you've joined an AM are the disabled 'Remove' button, and some changes to the Tools menu. I don't think that's enough.

I looked at Gridcoin, but they're recommending BAM! as their account manager, so I'm done here. Phew.
ID: 89025 · Report as offensive
Gary Roberts

Send message
Joined: 7 Sep 05
Posts: 130
Australia
Message 89028 - Posted: 26 Nov 2018, 6:39:50 UTC - in response to Message 89024.  

Thanks once again for your detailed responses.

To answer those two, first read (very forensically) the developer Wiki on Account Managers I linked earlier (here it is again).

I did read both the links when you first posted them earlier. I had previously read the second one (a couple of times) whilst trying to deal with the problem at Einstein, before coming here. I then read the first link, several times with increasing realisation that the content (even when read forensically) was well beyond my limited abilities. I'm not a programmer, developer, web developer, or any other sort of 'expert' and wouldn't have a clue how a lot of this stuff really works at a low level. I've always avoided AMs because they just seem like yet another layer of complexity that I really don't need. I'm extremely sorry if I've put you to extra work in trying to answer my dumb questions.

I guess I was trying to get enough of an understanding to feel comfortable to go back and recommend that the EAH user get rid of GR. I don't feel comfortable telling someone to take an action if I know that I don't fully understand the consequences myself. It's also a worry if I just think I understand. My favourite Mark Twain quote that I try to be mindful of is, "It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so." :-)

I'm very grateful that you have solved this dilemma for me. I can simply point to the following very good summary of sensible reasons to do that. I'm particularly horrified by the last bit as well.

BTW, I'd avoid Grid Republic if I were you. It tries to install an ancient version of BOINC (v6.8.44, dated 22 August 2012), and doesn't give you any choice about where to install it. But it does pick up and try to re-use the folder locations of your regular BOINC installation. I'm an experienced Windows user and developer (I've even run Windows Servers with Active Directory security and user account management), and it took me four or five attempts to get the installer to run past the various error messages. And the guidance link for what to put in the account/password boxes takes you direct to Facebook.

So, thank you very much for researching all of this. On a positive note, hopefully your work and insights will be of use to others contemplating the use of account managers.
Cheers,
Gary.
ID: 89028 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 89030 - Posted: 26 Nov 2018, 10:06:20 UTC

@Richard, are you using that data to make issue tickets out of on GitHub? Or is it a private test with the outcome being for whomever reads it and hopefully someone will pick up on it?
ID: 89030 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89031 - Posted: 26 Nov 2018, 10:32:47 UTC - in response to Message 89030.  

I'm pondering that. I certainly want to feed it in to the core BOINC development team (if such a thing exists), but just at the moment you're aware of the 'office politics' problem - I'm not sure how best to get it into the system. Look, for example, at David Anderson's comment of 22 January 2016 in #1455 - though that was before SU, of course. Suggestions for how to avoid it languishing in the backlog welcome.
ID: 89031 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 89032 - Posted: 26 Nov 2018, 10:57:12 UTC - in response to Message 89031.  

In any case, the AM protocol is pretty much set in stone at this point;
BAM! and Gridrepublic aren't going to change the way they work.
Well, why develop BOINC any further?
ID: 89032 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89033 - Posted: 26 Nov 2018, 11:19:48 UTC - in response to Message 89032.  
Last modified: 26 Nov 2018, 11:20:15 UTC

The question is - is SU going to change the way it works? :P

I think I might re-apply the clothespeg to my nose and give my SU test account another spin around the block - this time watching out for those 'random username' attach/detach events. If I get a different username every couple of hours, that would certainly be worth reporting, and I could also shoehorn it into the next developer call.
ID: 89033 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 89034 - Posted: 26 Nov 2018, 11:47:17 UTC - in response to Message 89033.  

You do know that not all BOINC developers think that way? Or at least, they didn't slightly over a month ago, things may have changed since I stopped monitoring.
ID: 89034 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89035 - Posted: 26 Nov 2018, 15:34:12 UTC

It seems that I may have over-worried about spamming project user tables. I appear to be:

Science United user b356c73a at Acoustics@home
Science United user 0c7077f2 at LHC@home
Science United user c205d69d at Amicable Numbers
Science United user 353e0849 at PrimeGrid
Science United user fd9dddbc at CPDN
and Anonymous at Einstein@Home

None of those seem to have changed since yesterday.

Now to find a clean way out, given that SU keeps overwriting my 'No new tasks' settings. (Apologies to Einstein and CPDN - I had to resort to the abort button for some of theirs)
ID: 89035 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 89071 - Posted: 28 Nov 2018, 15:42:25 UTC

I'm still picking away slowly at this one.

The original problem at Einstein - and one which I've seen over the years at other projects - is not being able to remove a project which is throwing up bad messages in the event log (bad url or duplicate attachment, for example).

The simple answer now seems to be definitive, and could be put into an FAQ.

Stop using an Account Manager

In BOINC Manager, switch to Advanced View (required step).
Open Tools menu
Select "Stop using [name of account manager]"

You'll be given a confirmation dialog explaining the consequences, but nothing frightening will happen: it simply says

If you stop using [name of account manager], you'll keep all your current projects, but you'll have to manage projects manually.
Do you want to stop using [name of account manager]?
And it works. Tasks will continue running, you can use NNT to prevent downloading new ones, and you (immediately) gain the power to remove unwanted or badly-configured projects.
ID: 89071 · Report as offensive

Message boards : Questions and problems : Account Managers and Project URLs

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.