I wanted to send a quick note to everyone and let you know
about an easy way to compress the outgoing files from your servers.
We started using this on World Community Grid about 3 weeks ago
and we have had no reports of problems due to it
(and we have seen a significant drop in our outbound bandwidth consumption).
Apache 2.0 includes a module called mod_deflate.
You can read about it here:
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html
This module allows you to specify that certain files will be
compressed dynamically when it is being sent to clients that specify
that they can handle it.
The BOINC client 5.4 and higher includes the ability to
decompress compressed files as they are downloaded.
If a BOINC client 5.2 or earlier requests work,
then the server will simply not compress the file so that
the client can handle the file.
We were expecting to only compress a few key files due to
the expected load on the server.
However, it turns out that the load on the server
is actually quite small so we are compressing most of the files
downloaded from our servers.
The average file is about 60% of the original file size.
Adding the compression on the fly only added about 5%
to the system CPU utilization (obviously it will vary
based on the power of your servers).
You need to read the Apache 2.0 documentation about this
module to make sure you understand it.
However, our httpd.conf file for these changes includes the following:
# Enable module
LoadModule deflate_module modules/mod_deflate.so
# Log file compression
DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
CustomLog logs/deflate_log deflate
# Use low settings for compression to make sure impact on server is low
DeflateMemLevel 2
DeflateCompressionLevel 2
Alias /boinc/download /path/to/files/download
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gz|gif|jpg|jpeg|png)$ no-gzip dont-vary
This configuration tells Apache to compress all files served from
the download direction except for files that end with gz,gif,jpg,
jpeg and png.
An alternate way to specify the files is the following:
Alias /boinc/download /path/to/files/download
AddOutputFilter DEFLATE .faa .mask
This configuration tells apache to compress only the file types
.faa and .mask served from the download director.