ERR_NOT_FOUND -161

Message boards : Questions and problems : ERR_NOT_FOUND -161
Message board moderation

To post messages, you must log in.

AuthorMessage
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68924 - Posted: 11 Apr 2016, 20:03:07 UTC
Last modified: 11 Apr 2016, 20:03:45 UTC

After a system crash 2 days ago, I restored the system, now it's running again well.
With exception of one thing:
When I crunch ATLAS@home tasks, the finished WUs are uploaded, but on the ATLAS homepage, under my results, it shows "computation error".

When I look these WUs up in detail, I notice, at the bottom of a long log file, the following:

upload failure: <file_xfer_error>
<file_name>ifiKDmgPm9nn7jp7oou28CBqABFKDmABFKDmNxMKDmABFKDm7HlENm_0_ATLAS_result</file_name>
<error_code>-161 (not found)</error_code>
</file_xfer_error>


I posted this in the ATKAS forum, and they told me to look this up here:
Check this page of error codes http://boincfaq.mundayweb.com/index.php?view=77&language=1

I got this:

ERR_NOT_FOUND -161
This happens when you have an inconsistent client_state.xml file. Files aren't written to it.
Task not found would be the error message.


Do you BOINC people agree to this?
And, in case my client_state.xml file is inconsistent, what can I do?

PS: I am crunching not only ATLAS, but also several other projects. No problem with those. So, why only with ATLAS? Would this problem really have to do with this BOINC file?
ID: 68924 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68925 - Posted: 11 Apr 2016, 20:19:57 UTC - in response to Message 68924.  

Do you BOINC people agree to this?

I wrote the BOINC FAQs. So, yes, wholeheartedly. :-)

And, in case my client_state.xml file is inconsistent, what can I do?

I'd start with an elevated sfc /scannow before doing a chkdsk /f /r (if Windows at least).

PS: I am crunching not only ATLAS, but also several other projects. No problem with those. So, why only with ATLAS? Would this problem really have to do with this BOINC file?

You can have a local corruption on the hard drive in the place where the client_state.xml file is. Problem is, the client_state.xml file is the most important file in BOINC, as it stores all the information about all the projects, all the applications and all the tasks, aside from BOINC its state.

ATLAS runs with VirtualBox, right? More complications.

No, I'd do a thorough check of the file system first, then disk check second.
ID: 68925 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 68927 - Posted: 11 Apr 2016, 20:21:57 UTC - in response to Message 68924.  

Usually a project reset would clean most inconsistencies. Detach re_add Atlas would empty the project folder.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 68927 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68928 - Posted: 11 Apr 2016, 20:30:41 UTC - in response to Message 68925.  

Ageless, thanks for your comments.

sfc /scannow and chkdsk /f /r were done already, this is okay.

Yes, of course, VirtualBox could add complications. That's why I un- and re-installed it.

So, what else could I do?
ID: 68928 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68930 - Posted: 11 Apr 2016, 20:57:31 UTC - in response to Message 68928.  

What caused the system crash?
Did it give a blue screen error? Do you know off-hand what it was?
If not, check with Bluescreenview what problems you may have had the last weeks.

And don't forget to try what Sekerob says, remove and re-add ATLAS.
ID: 68930 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68935 - Posted: 12 Apr 2016, 5:36:48 UTC - in response to Message 68930.  
Last modified: 12 Apr 2016, 5:44:56 UTC

When I had such system crashes (altogether 4 within a month or so), they happened at the moment a new ATLAS task was started (but NOT each time, only occasionally, very rarely in fact).
Bluescreenview (thanks for providing the link, I did not know this tool) shows "NDProxy.SYS" (NDIS Proxy) being the cause.

Only this morning I found out that the same problem happens with VirtualLHC@Home tasks (these, too, are finished and uploaded, but in the results list it says "computation error"). It took me that long to realize this since these tasks run about 26 hours.
They show the same error message:

upload failure: <file_xfer_error>
<file_name>wu_1460104689_4762_0_r1690680919_0</file_name>
<error_code>-161 (not found)</error_code>
</file_xfer_error>


By this, it's definitely clear now that the problem is NOT with ATLAS as I originally thought (that's why removing and re-adding ATLAS did not help).
It seems that all projects which are working with the VirtualBox are affected.
So I could un- and re-install the VB once more (I think I did this already).

If the problem persists, do you think that un- and re-installing BOINC could help - since, after all, the explanation for the error message says:
ERR_NOT_FOUND -161
This happens when you have an inconsistent client_state.xml file. Files aren't written to it.
Task not found would be the error message.


Is there a chance that the client_state.xml file would no longer be "inconsistent" after re-installation of BOINC?
ID: 68935 · Report as offensive
Les Bayliss
Help desk expert

Send message
Joined: 25 Nov 05
Posts: 1654
Australia
Message 68936 - Posted: 12 Apr 2016, 7:02:24 UTC

I'm not sure if it's relevant, but ...

In climateprediction.net, there are several zip files created throughout the processing of a model.
BOINC has a list of what is expected to be created, and where to send them. (This list is in client_state.xml.)

If the processing encounters a problem, causing the model to fail before reaching the end point, then BOINC will send back a list of the errors. As a part of this, it runs through it's list of zip files, and when it finds some that haven't been sent back, (because they were never created), it gives that "161 error, can't find the file" for each zip that it can't find.
ID: 68936 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68937 - Posted: 12 Apr 2016, 7:26:35 UTC - in response to Message 68935.  

Is there a chance that the client_state.xml file would no longer be "inconsistent" after re-installation of BOINC?

Uninstalling BOINC does nothing against files in the data directory. Everything in the data directory will be left intact, as else you cannot upgrade BOINC without constantly losing everything in that directory.

Is also not what we asked, what we asked was:
e.g. ATLAS@home, open BOINC Manager->(View->Advanced view->)Projects tab->select ATLAS@home->click Remove->Acknowledge->Tools menu->Add Project->choose ATLAS@home->Next->Yes, returning user + details about account->Next/Finish.

This deletes the Atlas project from the hard drive, out of the \Projects\ folder, and out of the client_state.xml file. Adding it again will add it anew, all new entries.

Cons: All in progress work will be removed this way, any not reported work that ran fine will get deleted and you may not get credit for it.
ID: 68937 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68939 - Posted: 12 Apr 2016, 8:14:30 UTC - in response to Message 68937.  

okay, what you are saying is clear to me.

However, assuming that this -161 error does in fact indicate an inconsistent client_state.xml file, would the (only) method to get a consistent file not be to uninstall BOINC completely, i.e. also deleting all data in the Projects (and Slots) folder? In other words: to start BOINC complety from scratch?

I am aware that not-reported work would be lost without getting credit for it, but if on the other hand I finally have a well working BOINC with which I can run ATLAS, VirtualLHC and others, it would be worth it.

But, my question again is: would this step most probably eliminate my problem? Or are there major chances that the problem is located somewhere else?
ID: 68939 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68940 - Posted: 12 Apr 2016, 9:05:22 UTC - in response to Message 68939.  

However, assuming that this -161 error does in fact indicate an inconsistent client_state.xml file, would the (only) method to get a consistent file not be to uninstall BOINC completely, i.e. also deleting all data in the Projects (and Slots) folder? In other words: to start BOINC complety from scratch?

Again, uninstalling BOINC will leave everything in the data directory intact, this includes the slots folders and contents, the projects folder and contents, and all files in the data directory.

Removing the project(s) from BOINC will do what you intent, delete all that and more.

But, my question again is: would this step most probably eliminate my problem? Or are there major chances that the problem is located somewhere else?

I cannot answer the question with full 100% certainty. Heck, not even with 50% certainty. So no, the problem could lay elsewhere as well. Even still with an undetected problem with the hard drive, or with something that broke in Windows.

Only through testing can you figure out what may be the cause.

One other option you have is to uninstall BOINC the normal way, then move the data directory to a whole different place, either another place on the same drive, or a different partition, or a different hard drive. Then you reinstall BOINC, in the installer -third screen click Advanced- point to where you put the data directory in its new place, and then continue installing BOINC, before testing if that fixed things.

If that did fix things, there's a problem with the sectors of the hard drive where the directory originally was stored, or with the master file table.
If that didn't fix things, the problem may be in the complete data directory, or Windows still.
ID: 68940 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68942 - Posted: 12 Apr 2016, 11:09:06 UTC

One thing that I completely forgot to mention:

I had moved the BOINC/Slots folder (which carries all these large and frequently changing .vdi-image files for ATLAS and VirtualLHC) to a RAMDisk, in order to avoid early SSD wear; with several ATLAS tasks running simultaneously, some 200GB data was written to the SSD per day. Since I have 32GB RAM, I made use of it by installing a RAMDisc, creating a "junction point" to the BOINC/SLOTS folder.

Could this be the cause for my problem?
Has anyone experienced any problems with running VirtualBox BOINC applications on a RAMdisk?
ID: 68942 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68943 - Posted: 12 Apr 2016, 13:49:30 UTC - in response to Message 68942.  

unfortunately, I could not edit my above posting any more.

Additional thought: could the problem be caused by the fact that within the BOINC folder, the "projects" folder in on the SSD, and the "slots" folder is on the ramdisk?
Any argueable indication that this is not going well?
ID: 68943 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68944 - Posted: 12 Apr 2016, 14:54:51 UTC - in response to Message 68942.  

Why haven't you just moved the whole data directory to the RAMdisk?

As I explained earlier, you can point out in the (Windows) Installer where BOINC should put the data directory. All of it.

So please, uninstall BOINC.
Move the data directory with everything in it (default at C:\Programdata\BOINC\) to the RAMdisk. If it's still outside, then move the Slots directory back into the BOINC data directory.

Now reinstall BOINC and in the third screen in the installer click Advanced.
Choose the second path on the next screen to point to your new data directory on the RAMdisk.

And I hope you make backups to USB, SSD or HDD, for if the power fails and you lose everything in the RAMdisk.
ID: 68944 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68947 - Posted: 12 Apr 2016, 16:31:39 UTC - in response to Message 68944.  

Why haven't you just moved the whole data directory to the RAMdisk?

there was a strong reason for not moving the whole BOINC data to the RAMdisk:

the original intention was to have only that part of BOINC there, where there is a frequent exchange (writing, reading, deleting) of huge amounts of data (in order to avoid early wear of the SSD); and this is the SLOTS folder where ATLAS puts in the .vdi image files (between 1.3GB and 1.8GB each) for the VirtualBox. Each new task writes a new image file. Besides this, each ATLAS task requires up to 2.5 GB system RAM.
The data in the PROJECTS folder, however, is very static, not much exchange at all (at least no big amounts of data are moved in there).

So, in order to make best use of my 32GB RAM with a maximum of ATLAS tasks running simultaneously, it turned out that a RAMdisk size of 13.5GB is about ideal, in view of how much RAM this leaves for the system (18.5GB). In fact, it would balance out even better with 14.5GB / 17.5GB.
Said the above, if I install the complete BOINC data directory to the RAMDisk, I am "wasting" some 3,5GB of RAM, so to say (I hope I was able to explain this correctly so that you understand what I mean).

Do I gather from your posting that you think that the source of my problem might be that part of the BOINC data is on the SSD, and part of it is on the RAMDisk?
Most probably we would only find out for sure by changing my setting to have it the way you were describing it (= all BOINC data on the RAMDisk) - I don't know.

It's just too bad that it was not clear to me beforehand how much writing ATLAS is doing. With up to 200GB per day, my SSD will probably not last longer than 2 years maximum. On the other hand, most of my 32GB RAM would stay unused (which, for me personally, is even a stronger reason than the SSD wear) That's why I decided for the RAMDisk solution.
ID: 68947 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 68960 - Posted: 12 Apr 2016, 22:52:50 UTC - in response to Message 68947.  
Last modified: 12 Apr 2016, 22:54:05 UTC

Am I to understand that you only have an SSD in that computer, no hard drive next to it? Because otherwise you might want to consider that, as HDDs have almost unlimited reads and writes.

The newer SSHDs (Solid State Hybrid Drive) combine 8GB of NAND memory with up to 4TB of hard drive space. The NAND memory is used to cache much used programs so they start up faster.

I bought a Seagate 2TB SSHD for my main computer and will never again look back at mere HDDs. (Great review at http://www.tweaktown.com/reviews/5748/seagate-desktop-2tb-sshd-st2000dx001-review/index.html)

But that's a simple and cost-efficient solution.
The data directory is a complex directory, and yes it's constantly written to. The unofficial advice is always to place it on a hard drive, in its completeness. If you don't have a HDD, then just place the whole directory in the RAMdisk.

Just linking only part of the data directory to elsewhere is a dangerous thing to do. I'd say, don't do it.
ID: 68960 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68963 - Posted: 13 Apr 2016, 5:51:31 UTC - in response to Message 68944.  


And I hope you make backups to USB, SSD or HDD, for if the power fails and you lose everything in the RAMdisk.


This would definitely make sense; however, automated backups by a backup software, in certain time intervals, will unfortunately not work, as BOINC must be closed before it's files can be backed up.

So, the only thing that I can do is to make manual backups by shutting down BOINC first and than start the backup. It's too bad that it doesn't work any other way, but I could not think of anything else.

However, I even doubt the usefulness of such backups unless they are made every hour so. Because what would it help if in case of a power failure or a system crash (or whatever else) I restore a BOINC data folder which is even not older than a day? It would bring me back work units that most probably were finished and uploaded in the meantime.

Or is my thinking wrong and I am missing something? Please give me your comments.
ID: 68963 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68966 - Posted: 13 Apr 2016, 19:30:53 UTC - in response to Message 68964.  
Last modified: 13 Apr 2016, 19:34:18 UTC

Make a backup, restore it later and you have an out-of-sync situation

that's exactly what I guess -

so backups probably wouldn't make any sense.

Anyway, today I re-installed BOINC from scratch, with the complete data-directory on the RAMDisk (in contrast to how it was before: only the SLOTS folder on the RAMDisk) - and now the ATLAS tasks work well.
So, it definitely had NOT been a good idea to split the BOINC data folder to SSD and to RAMDisk.
ID: 68966 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68975 - Posted: 14 Apr 2016, 6:38:34 UTC

I have a new, clean install of BOINC. The complete data directory is on the 14GB RAMDisk now.
And the ATLAS work units, after uploading, are no longer showing the "computation error" with -161 error.

However, what is different to how it was before, is the following:

Presently, just to start with "half speed" and increasing everything later, under BOINC computing preferences, I have set CPU use to 50%. Before, with this setting, 6 ATLAS tasks were running (6-core CPU with HT). Now, only 4 ATLAS tasks are running, altough there is 5,4GB free space on the RAMDisk (1 ATLAS task uses about 1,6GB for the vm_image.vdi in the SLOTS folder).
BOINC settings are such that 97% of disk space can be used, and 95% of RAM (system RAM is 18GB, currently some 5GB are free).
Also, setting for storing work and for storing additional work is at 0,2 days of work ea, (before, it was only at 0,1 days of work), but no ATLAS tasks are downloaded in advance. Only after one of the four gets finished and uploaded, the next one is being downloaded. Before, I always had between 2 and 4 task in "waiting to run" status.

Why are all these things so different to what they were before when I was not working with the RAMDisk?
What am I doing wrong? Which of the settings are incorrect?
ID: 68975 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 68978 - Posted: 14 Apr 2016, 11:57:23 UTC - in response to Message 68975.  

Of course you've been reading the message/event log where most all is being told of what the client is or is not doing, with a why.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 68978 · Report as offensive
Erich56

Send message
Joined: 30 Dec 14
Posts: 95
Austria
Message 68986 - Posted: 15 Apr 2016, 6:28:26 UTC - in response to Message 68978.  

Of course you've been reading the message/event log where most all is being told of what the client is or is not doing, with a why.

after a short while, it's all turned out running okay - what concerns the ATLAS project.

However, after adding the GPUGRID project, another problem came up.
I have described it here: https://boinc.berkeley.edu/dev/forum_thread.php?id=10957
ID: 68986 · Report as offensive

Message boards : Questions and problems : ERR_NOT_FOUND -161

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.