Posts by Wei Hao

1) Message boards : Questions and problems : how to make boinc read app_config.xml (Message 56681)
Posted 14 Oct 2014 by Wei Hao
Post:
Thanks Juha. I tried "boinccmd --read_cc_config" and it works! And Your suggestion to config the server side is promising too. Thanks you very much for sharing your knowledge!
--Wei
2) Message boards : Questions and problems : how to make boinc read app_config.xml (Message 56675)
Posted 13 Oct 2014 by Wei Hao
Post:
Hi Juha, Thanks for the feedback.

I just noticed the --read_cc_config option. I will try that.

I set up a boinc server on AWS cloud which will dispatch my projects and apps. I want to do a simulation, say I want to set up 100 clients on AWS cloud. I have a application which can create work units in "pulse" model. For example, there will be 30 work units sent to the WU queue in less than 5 seconds, and then wait for 4 hours and then 50 work units will be in the queue in a very short time.

In these cases, many jobs will be dispatched to one client and the distribution is not evenly.

Do you have any idea how to make it evenly distributed to all clients? In this way, I can take advantage of a number of clients.

Thanks,
3) Message boards : Questions and problems : BOINC not reading app_config.xml files (Message 56672)
Posted 13 Oct 2014 by Wei Hao
Post:
Thanks for the reply.

Do you know if I can do it use command line tools? I dont want to use the Boinc manager GUI.

Or if I restart the Boinc client, will it read the config file by default? If not, I need to use the Boinc manager to do it again and again?

Thanks,
4) Message boards : Questions and problems : BOINC not reading app_config.xml files (Message 56671)
Posted 13 Oct 2014 by Wei Hao
Post:
Thanks for the reply, Claggy

Do you know if I can do it use command line tools? I dont want to use the Boinc manager GUI.

Or if I restart the Boinc client, will it read the config file by default? If not, I need to use the Boinc manager to do it again and again?

Thanks,
5) Message boards : Questions and problems : how to make boinc read app_config.xml (Message 56670)
Posted 13 Oct 2014 by Wei Hao
Post:
I just want to limit the max number of concurrent jobs for any applications in my project.

I tried the app_config.xml and it works for me but It seems that I need to use the boinc manager GUI to manually make the boinc system read in the app_config.xml file as follows,

<app_config>
<app>
<name>myappname</name>
<max_concurrent>1</max_concurrent>
</app>
</app_config>



Is there any way to do it using command line tools or is it the way that when the boinc client restart, it will read the app_config.xml file automatically?

Thanks!
6) Message boards : Questions and problems : BOINC 7.2.42 and app_config.xml (Message 56667)
Posted 13 Oct 2014 by Wei Hao
Post:
Hi Guys,

I meet similar situation.
My Machine is on AWS cloud. m3.x2large 30M RAM and 8 cores.

I tried reset the project, restart Boinc client. But nothing works!

Anybody can help?
7) Message boards : Questions and problems : BOINC not reading app_config.xml files (Message 56665)
Posted 13 Oct 2014 by Wei Hao
Post:
Hi numbermaniac,

I am trying to do the same thing with my owner application distributed from my own boinc server.

I tried to insert an app_config.xml file into the /var/lib/boinc-client/project/myproject@ip/

but I found nothing changes.

Which log file are you checking?
Do I need to restart the boinc client to make it read the app_config.xml?

Thanks in advance!

Wei
8) Message boards : Server programs : help on how to upload multiple output files? (Message 53563)
Posted 9 Apr 2014 by Wei Hao
Post:
Here is how I checked the upload results:

boincadm@test1:~/projects/mypro/upload$ tree
.
├── 15d
│   └── myapp_2_0_1
└── 37f
└── myapp_2_1_1

I supposed that there should be more than one file in each directory.

I will check the client log and configuration files. Thanks,
9) Message boards : Server programs : help on how to upload multiple output files? (Message 53550)
Posted 8 Apr 2014 by Wei Hao
Post:
Hi colleagues,

I just have an application that has 4 output files. I tried to figure the output template like this:

<output_template>

<file_info>
<name><OUTFILE_0/></name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>5000000</max_nbytes>
<url><UPLOAD_URL/></url>
</file_info>

<file_info>
<name><OUTFILE_1/></name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>5000000</max_nbytes>
<url><UPLOAD_URL/></url>
</file_info>

<file_info>
<name><OUTFILE_2/></name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>5000000</max_nbytes>
<url><UPLOAD_URL/></url>
</file_info>

<file_info>
<name><OUTFILE_3/></name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>5000000</max_nbytes>
<url><UPLOAD_URL/></url>
</file_info>

<result>
<file_ref>
<file_name><OUTFILE_0/></file_name>
<open_name>Results/Output1.asc</open_name>
<copy_file/>
<no_validate>1</no_validate>
</file_ref>
</result>

<result>
<file_ref>
<file_name><OUTFILE_1/></file_name>
<open_name>Results/Output2.csv</open_name>
<copy_file/>
<no_validate>1</no_validate>
</file_ref>
</result>

<result>
<file_ref>
<file_name><OUTFILE_2/></file_name>
<open_name>Results/Output3.csv</open_name>
<copy_file/>
<no_validate>1</no_validate>
</file_ref>
</result>

<result>
<file_ref>
<file_name><OUTFILE_3/></file_name>
<open_name>Results/Output4.asc</open_name>
<copy_file/>
<no_validate>1</no_validate>
</file_ref>
</result>

</output_template>



the application runs as anticipated,

<core_client_version>7.0.27</core_client_version>
<![CDATA[
<stderr_txt>
14:34:37 (26208): wrapper: starting
14:34:37 (26208): wrapper: running crest ()
14:35:39 (26208): called boinc_finish

</stderr_txt>
]]>


But I found in the upload folder on the server that there is only one file successfully uploaded. Do you guys meet this situation before.

Any suggestions or comments would be appreciated.
10) Message boards : Documentation : Error Link on Official Wiki Site? (Message 52735)
Posted 23 Feb 2014 by Wei Hao
Post:
I noticed that.
Thanks!
11) Message boards : Documentation : Error Link on Official Wiki Site? (Message 52700)
Posted 21 Feb 2014 by Wei Hao
Post:
On this page,
https://boinc.berkeley.edu/trac/wiki/BoincFiles

There is a sentence:
File properties are specified in job input and output templates.
The link referred by the above color coded text should be
http://boinc.berkeley.edu/trac/wiki/JobTemplates
Rather than
https://boinc.berkeley.edu/trac/wiki/JobSubmission
12) Message boards : API : Run Java Application with Boinc on Linux Host? (Message 52647)
Posted 18 Feb 2014 by Wei Hao
Post:
Thanks for the reply.

Yes. But do you know how to configure the application so that the downloader know which files and directories to be downloaded to the host machine? Since the application needs several dynamic linked libraries, as listed in,
https://boinc.berkeley.edu/dev/forum_thread.php?id=8929

This is a typical instance for adaptation of current and legacy scientific applications to boinc. Is it possible that you provide an real world example rather than the simply text processing application such as uppercase? I think that could be very helpful.

Thanks,
13) Message boards : API : Can close-coupled workflow type application be deployed on a BOINC server? (Message 52630)
Posted 18 Feb 2014 by Wei Hao
Post:
If all the clients are supposed to be in the same LAN, then it may not be such a big problem. Thanks for the sharing of your ideas!
14) Message boards : API : Run Java Application with Boinc on Linux Host? (Message 52629)
Posted 18 Feb 2014 by Wei Hao
Post:
This is a little disappointing though.

Thanks for the reply anyway.
15) Message boards : API : Run Java Application with Boinc on Linux Host? (Message 52616)
Posted 17 Feb 2014 by Wei Hao
Post:
Hi Guys,

This question is related to my previous question, http://boinc.berkeley.edu/dev/forum_thread.php?id=8929&postid=52611

On the official Wiki Site, I found some information on how to run a java app with BOINC, i.e. a simple wrapper approach without using BOINC api

But on that web page, I found that there is a line saying that "(for windows hosts only, because of jsmooth)". Does that imply that we can not run java application with Boinc on Linux Host?

Thanks,
16) Message boards : API : Help with Deployment of complex application using Boinc Wrapper (Message 52611)
Posted 17 Feb 2014 by Wei Hao
Post:
Hi all.
I successfully get the sample "worker" application(Which extracts the first line of a text file) deployed as a boinc project. Now I am trying some real world applications. Such as transforming a HDF5 file to an image with geo-location information.

The package is something like this,
boincadm@ip-100-122-76-123:/home/ubuntu/SPA/h2g/algorithm$ ls -l
total 56
drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 16 19:39 bin
-rwxrwxr-x 1 ubuntu ubuntu 548 Sep 11 2009 build64bit.sh
-rwxrwxr-x 1 ubuntu ubuntu 205 Sep 29 2009 build_javadoc.sh
drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 16 19:39 classes
drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 16 19:39 colormaps
drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 16 19:39 configFiles
-rwxrwxr-x 1 ubuntu ubuntu 448 Feb 20 2012 configure-ipopp-projection
drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 16 19:39 docs
drwxrwxr-x 4 ubuntu ubuntu 4096 Sep 16 19:39 eosnppstations
drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 16 19:39 lib32
drwxrwxr-x 5 ubuntu ubuntu 4096 Sep 16 19:39 lib64
drwxrwxr-x 4 ubuntu ubuntu 4096 Sep 16 19:39 OLD
drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 16 19:39 src
drwxrwxr-x 4 ubuntu ubuntu 4096 Sep 16 19:39 tileStations

And the command line that invoke the application is something like this,

/home/ubuntu/SPA/h2g/algorithm/bin/h2g.sh /home/ubuntu/SPA/h2g/wrapper/h2g/../../algorithm standard lst dataset /home/ubuntu/SPA/h2g/testdata/inpu
t/LST.08085185938.hdf geoloc /home/ubuntu/SPA/h2g/testdata/input/MYD03.08085185938.hdf mask /home/
ubuntu/SPA/h2g/testdata/input/MYD014.08085185938.hdf otype geotiff.u8cm output ./LST.tif projection DEFAULT resolution -999.0
centerlat -999.0 centerlon -999.0 height_lat -999.0 width_lon -999.0 height_km -999.0 width_km -999.0


The h2g.sh script is something like this,
$ cat /home/ubuntu/SPA/h2g/algorithm/bin/h2g.sh
#!/bin/bash
case $0 in
/*) H2G_HOME="`dirname $0`/.." ;;
*) PWD=`pwd`; H2G_HOME="`dirname $PWD/$0`/.." ;;
esac
LIB=$H2G_HOME/lib64
#LIB=$H2G_HOME/lib64
java \
-Xmx2g \
-Djava.library.path=$LIB/linux:$LIB/proj \
-Djava.awt.headless=true \
-classpath \
$LIB/jproj.jar:\
$LIB/h2g.jar:\
$LIB/jhdf5.jar:\
$LIB/jhdf5obj.jar:\
$LIB/jhdf4obj.jar:\
$LIB/jhdf.jar:\
$LIB/jhdfobj.jar:\
$LIB/jai_imageio.jar \
gov.nasa.gsfc.drl.h2g.Create $@

And there are some dependent libraries,
:/home/ubuntu/SPA/h2g/algorithm/lib64$ tree
.
├── ext
│   ├── fitsobj.jar
│   └── nc2obj.jar
├── fits.jar
├── h2g.jar
├── jai_imageio.jar
├── jhdf4obj.jar
├── jhdf5.jar
├── jhdf5obj.jar
├── jhdf.jar
├── jhdfobj.jar
├── jhdfview.jar
├── jproj.jar
├── junit.jar
├── linux
│   ├── libhdf4.settings
│   ├── libhdf5.settings
│   ├── libjhdf5.so
│   └── libjhdf.so
├── netcdf.jar
└── proj
├── COPYING
├── libproj.a
├── libproj.so -> libproj.so.0.5.5
├── libproj.so.0 -> libproj.so.0.5.5
└── libproj.so.0.5.5

So the successful running of the application depends on a bunch of jar files and some dynamic libraries.

I did not find detailed instructions on how to deploy/wrap this kind of application.
And I have several questions here,
-Is there any xml file to specify the location of the java libraries and other dynamic linked libraries?
- How to make the client know where to download the whole directories? Should I make a zip file for the client to download and then how the client will know to unzip the binary directories.
- How to configure the application read in multiple input files?

Thanks,
17) Message boards : API : Can close-coupled workflow type application be deployed on a BOINC server? (Message 52089)
Posted 22 Jan 2014 by Wei Hao
Post:
Thanks Christian. By 'as large as', I mean a typical output file size is ~3GB. As you said, it is not good for average volunteer computing environment.
18) Message boards : API : Can close-coupled workflow type application be deployed on a BOINC server? (Message 52086)
Posted 22 Jan 2014 by Wei Hao
Post:
Hi Christian, I am pretty new with BOINC. Thank you for your feedback. In our case, we have large data sets for each section of work flow. For instance, a single output file of Section A could be as large as 3GB. And the processing time could be more than an hour. We can not split each section into smaller pieces since we don't have the source code. Do you have more suggestions such that we can deploy the project more efficiently?
19) Message boards : API : Can close-coupled workflow type application be deployed on a BOINC server? (Message 52085)
Posted 22 Jan 2014 by Wei Hao
Post:
Cool. Thanks!
20) Message boards : API : Can close-coupled workflow type application be deployed on a BOINC server? (Message 52060)
Posted 21 Jan 2014 by Wei Hao
Post:
we have an earth science work flow which can be divided into around ten sections. The ten sections are interdependent such that (for instance,)Section C's input should be Section A and B's output. And each section takes ~10 to ~90 minutes to process on regular workstations. Would this kind of project be efficient or even possible to deploy on a BOINC Server? As my understanding, BOINC will be very useful for computing tasks that are loosely coupled, i.e., the computing units should be quite independent on each other and the granule of the computing unit should be very light.


Next 20

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.