Hello,
For my project the data files used for a WU are downloaded already gzipped.

I use libz to unzip them.

I notice when the job is done the unzipped files are not removed from the client.

I can remove them my self via code when a WU is done but was wondering if there was any existing mechanism to do this?

The tricky bit here for me is when a data file is shared by multiple work units, I'll have to figure our the wu_name and use that as part of the unzipped file name so I don't remove a file thats being used by another WU.

Cheers
Seth

Hi,
this depends on how you unzip the files. If you unzip to the project directory you have to clean up yourself. If you unzip into the slot directory BOINC will clean up for you.

If your application can handle uncompressed input files you can leave compressing/decompressing to BOINC. See <gzip/> in Input templates. (File compression says this is not supported for input files but it's probably outdated info.)

Yes Jhua but I think this doesn't work if you stage files externally like I am in an S3://bucket. In anycase I've adapted my client to unzip using the WU name for uniqueness.

Yes Jhua but I think this doesn't work if you stage files externally like I am in an S3://bucket. In anycase I've adapted my client to unzip using the WU name for uniqueness.

It should work too. Since you give your own URL in the input template either way you can just et the URL to the correct one but remove the ".gz" at the end. Then the Client (>7.x) will add the ".gz" because of the </gzip>, download and unpack the file into the slot directory. The file is kept on the computer for as long as there are tasks that need it and you don't need special handling in your application.