wiki:GPUApp

Version 21 (modified by TuanLe, 14 years ago) (diff)

--

Building BOINC applications with Cuda and OpenCL

We have built sample BOINC-Cuda and BOINC-OpenCL applications as templates for making your own app. The source files are in the BOINC source tree? in samples/nvcuda, samples/nvopencl, and samples/atiopencl.

Windows

The 32-bit Windows compile under Visual Studio

NVIDIA Cuda

The VS project file, example_app_nvcuda.vcproj, is in "boinc/win_build/". Note that NVIDIA Cuda SDK 3.0 or older supports device emulation mode. If your machine doesn't have a CUDA-enabled GPU, then you should build the sample application in the device emulation mode. Do so by setting "Build -> Configuration Manager -> Configuration" to either Emudebug or Emurelease.

The executable file can be found at "boinc/win_build/Build/Win32/build_mode/" (build_mode could be release, debug, emurelease or emudebug).

NVIDIA OpenCL

The project file, example_app_nvopencl.vcproj, is in "boinc/win_build/". OpenCL doesn't have emulation mode, but it could run on CPU alone by specifying CL_DEVICE_TYPE_CPU or CL_DEVICE_TYPE_ALL when creating an OpenCL context.

The executable file can be found at "boinc/win_build/Build/Win32/build_mode/" (build_mode could be release or debug).

ATI Stream OpenCL

The project file, example_app_atiopencl.vcproj, is in "boinc/win_build/". It is identical to NVIDIA OpenCL.

Linux

NVIDIA Cuda

Before running the Makefile, you will need to install gcc 4.3 and g++ 4.3. This is because the NVIDIA Cuda SDK 3.0 has not yet worked with gcc 4.0 and g++ 4.0. There should be no issue compiling cuda files with gcc 4.3 and g++ 4.3 on newer NVIDIA Cuda SDK versions. For a successful compilation, please follow these steps:

1) Install gcc-4.3 and g++-4.3:

$ sudo aptitude install gcc-4.3 g++-4.3

2) Go to SDK source directory:

$ cd ~/NVIDIA_GPU_Computing_SDK/C

3) Create a directory and create symlinks to gcc-4.3/g++-4.3

$ mkdir mygcc
$ cd mygcc
$ ln -s $(which g++-4.3) g++
$ ln -s $(which gcc-4.3) gcc

The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location "$(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/". On my Linux machine, for example, it would be "$(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/". Since the absolute path is used in Makefile, you will probably need to edit file "common.mk" before compiling the sample application. Do so by looking for "/home/tuanle/NVIDIA_GPU_Computing_SDK" in file "common.mk" and replace this path with an appropriate path on your machine.

You will also need to build the necessary Cuda SDK libraries by running makefile in "~/NVIDIA_GPU_Computing_SDK/C/". The resulting libraries such as "libcutil_i386" are created in "~/NVIDIA_GPU_Computing_SDK/C/lib/". Notice that if these libraries are not built, running nvcuda-sample-app makefile could result in the following error:

/usr/bin/ld: cannot find -lcutil_i386

If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine. To build the sample application in emulation mode, simply type:

make emu=1

The executable file is created in "/boinc/samples/nvcuda/linux/build_mode/" (build_mode could be release or emurelease). Also, note that you will need to define some environment variables before running the executable file.

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH

This will resolve the following error:

./example_app_nvcuda: error while loading shared libraries: libcudart.so.3: cannot open shared object file: No such file or directory

NVIDIA OpenCL

It appears that NVIDIA OpenCL SDK is missing shared library file libOpenCL.so which is needed at compile time. This library file comes with the kernel driver package which can be installed only when your machine has an NVIDIA Cuda-enabled GPU. Thus, any attempt to compile OpenCL sample codes on a non-cuda-enabled GPU machine might result in the following error:

/usr/bin/ld: cannot find -lOpenCL

The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location "$(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/". On my Linux machine, for example, it would be "$(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/". Since the absolute path is used in Makefile, you will probably need to edit file "common_opencl.mk" before compiling the sample application. Do so by looking for "/home/tuanle/NVIDIA_GPU_Computing_SDK" in file "common_opencl.mk" and replace this path with an appropriate path on your machine.

You will also need to build the necessary Cuda SDK libraries by running makefile in "~/NVIDIA_GPU_Computing_SDK/OpenCL/". The resulting libraries such as "liboclUtil_i386" are created in "~/NVIDIA_GPU_Computing_SDK/OpenCL/common/lib/". Notice that if these libraries are not built, running nvopencl-sample-app makefile could result in the following error:

/usr/bin/ld: cannot find -loclUtil_i386

The executable file is created in "/boinc/samples/nvopencl/linux/release/".

ATI Stream OpenCL

The Makefile for Linux is made with the assumption that the ATI Stream OpenCL SDK version 2.1 for 32-bit Linux and BOINC source code are installed in the same parent directory, i.e. "ati-stream-sdk-v.2.1-lnx32" directory and "boinc" directory are both installed at "$(ROOT)/home/" directory. If you are using the SDK v2.1 for 64-bit Linux, or if BOINC source code and the OpenCL SDK are not installed in the same parent directory, then Makefile needs to be edited prior to any compilation. Do so by looking for

CXXFLAGS = -g \
     ...
     -I ../../../ati-stream-sdk-v2.1-lnx32/include \
     ...

atiopencl: atiopencl.o libstdc++.a
     ...
     -lOpenCL -L../../../ati-stream-sdk-v2.1-lnx32/lib/x86 \
     ...

in file "Makefile" and replace these paths with appropriate paths on your machine.

Most problems related to running the ATI Stream OpenCL executable files can be found in ATI_Stream_SDK_Installation_Notes.pdf section 2.2. Here are the two most common run-time errors:

1) ./atiopencl: error while loading shared libraries: libOpenCL.so: cannot open shared object file: No such file or directory -> Solution: Set the environment variable ATISTREAMSDKROOT

$ export ATISTREAMSDKROOT=<location where Stream SDK is extracted>

Set the library path LD_LIBRARY_PATH

  • For 32-bit systems:
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86:$LD_LIBRARY_PATH
    
  • For 64-bit systems:
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86_64:$LD_LIBRARY_PATH
    
  • For the 32-bit SDK to run on 64-bit systems:
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86:$ATISTREAMSDKROOT/lib/x86_64:$LD_LIBRARY_PATH
    

2) ./atiopencl: error CL_PLATFORM_NOT_FOUND ->

Solution: Register the OpenCL ICD by following these steps:

  • Download icd-registration.tgz at http://developer.amd.com/Downloads/icd-registration.tgz
  • In the command shell, type "cd /"
  • Type "ls", you will see a directory named "etc". The icd-registration.tgz needs to be unzipped to that directory.
  • Now copy icd-registration.tgz to "cd /" by typing "sudo cp -r /home/tuanle/Downloads/icd-registration.tgz ."
  • You will need to replace the specified path above with the appropriate path on your machine, and don't forget "sudo". Otherwise you will get a "Permission denied".
  • Unzip by typing "sudo tar xzf icd-registration.tgz"
  • Now you should be able to find an "OpenCL" directory in "etc". Check etc/OpenCL/vendors/, make sure there are "atiocl32.icd" and "atiocl64.icd" there.
  • Done!

Mac OS X

The Mac versions are built using gcc/make (not xcode).

NVIDIA Cuda

Unlike Windows and Linux, the 'nvcc' cuda compiler for Mac has some trouble compiling .cu files that contain both BOINC and Cuda code. Any attempt to compile such .cu files might result in errors like these:

/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(55): error: identifier "__builtin_ia32_emms" is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(68): error: identifier "__builtin_ia32_vec_init_v2si" is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(111): error: identifier "__builtin_ia32_vec_ext_v2si" is undefined
...

The solution that we came up with is to compile BOINC and Cuda separately, meaning that compile .c files that have BOINC codes with gcc and compile .u files that have Cuda codes such as kernel definitions with nvcc (This can simply be done by putting .cu and .c file names at corresponding "CUFILES := " and "CCFILES := " entries in Makefile_mac). To handle the results computed by the kernels, you will need to write external functions. The function definitions that make kernel calls will be put in .cu file while the function headers with the "extern" prefix are put in .c file and can be called in main function or elsewhere.

If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine.

The makefile for Mac is made with the assumption that the NVIDIA Cuda SDK is installed at the default location: "$ROOT/Developer/GPU Computing/" on Mac. If it is installed at a different location on your machine, then you will need to edit file "common_mac.mk". Do so by looking for "ROOTDIR ?= /Developer/GPU\ Computing" in file "common_mac.mk" and replace this path with an appropriate path on your machine.

To build the sample application in emulation mode, simply type:

make -f Makefile_mac emu=1

The executable file is created in "/boinc/samples/nvcuda/darwin/build_mode/" (build_mode could be release or emurelease). Also, note that you will need to define some environment variables before running the executable file.

export PATH=/usr/local/cuda/bin:$PATH
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH

NVIDIA OpenCL

The NVIDIA OpenCL SDK for MacOS doesn't contain the OpenCL library file. This file often comes with the kernel driver package for MacOS which can be installed only when your machine has an NVIDIA Cuda-enabled GPU. The other way to obtain this file is to have Mac OS X 10.6 installed on your machine.

The makefile for Mac is made with the assumption that the NVIDIA Cuda SDK is installed at the default location: "$ROOT/Developer/GPU Computing/" on Mac. If it is installed at a different location on your machine, then you will need to edit file "common_opencl_mac.mk". Do so by looking for "ROOTDIR ?= /Developer/GPU\ Computing" in file "common_opencl_mac.mk" and replace this path with an appropriate path on your machine.

To build the sample application, simply type:

make -f Makefile_mac

The executable file is then created in "/boinc/samples/nvopencl/darwin/release/".

ATI Stream OpenCL

ATI Stream does not have a separate OpenCL SDK for MacOS users to download. Thus, the only way to build OpenCL applications on your Mac is to have Mac OS X 10.6 version (which supports OpenCL) installed on your machine.

To build the sample application, simply type:

make -f Makefile_mac

The executable file is then created in "/boinc/samples/atiopencl/".