[Linux] Trick to use AMDGPU-Pro OpenCL capabilities without installing the whole propertary driver stack

Message boards : GPUs : [Linux] Trick to use AMDGPU-Pro OpenCL capabilities without installing the whole propertary driver stack
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75367 - Posted: 18 Jan 2017, 16:01:49 UTC
Last modified: 18 Jan 2017, 16:02:16 UTC

Linux only:
There is a trick to use OpenCL capabilities of lastest AMD GPUs supported by AMDGPU FOSS driver even not installing the AMDGPU-Pro closed driver:
Download the AMDGPU-Pro drivers, unpack libdrm, libAMDOpenCL, libkms and put them under
/opt/amdgpu-pro

Don't forget the file amdocl64.icd has to be copied in the path
/etc/OpenCL/vendors/

It seems hard, but once you have unpacked those libraries you will understand everything.

Ok once you have done, simply start the application with the following command
$ LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64 application_name

For example I run darktable with command
$ LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64 darktable


I tried with BOINC too
$ LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64 /usr/bin/boinc_client


and it detects the GPU correctly
18-Jan-2017 16:32:48 [---] OpenCL: AMD/ATI GPU 0 (ignored by config): AMD POLARIS10 (DRM 3.8.0 / 4.9.3-200.fc25.x86_64, LLVM 3.8.0) (driver version 13.0.3, device version OpenCL 1.1 Mesa 13.0.3, 8159MB, 8159MB available, 3709 GFLOPS peak)
18-Jan-2017 16:32:48 [---] OpenCL: AMD/ATI GPU 1: Ellesmere (driver version 2236.5, device version OpenCL 1.2 AMD-APP (2236.5), 7868MB, 7868MB available, 622 GFLOPS peak)
18-Jan-2017 16:32:48 [---] OpenCL CPU: pthread-AMD Phenom(tm) II X4 965 Processor (OpenCL driver vendor: The pocl project, driver version 0.13, device version OpenCL 2.0 pocl)
18-Jan-2017 16:32:48 [---] OpenCL CPU: AMD Phenom(tm) II X4 965 Processor (OpenCL driver vendor: Advanced Micro Devices, Inc., driver version 2236.5 (sse2), device version OpenCL 1.2 AMD-APP (2236.5))


Since on Fedora we run BOINC as a systemd service, I edited
/usr/lib/systemd/system/boinc-client.service

[Unit]
Description=Berkeley Open Infrastructure Network Computing Client
Documentation=man:boinc(1)
After=network-online.target

[Service]
Type=forking
Nice=10
User=boinc
WorkingDirectory=/var/lib/boinc
ExecStart=/usr/bin/boinc_client --daemon --start_delay 1
ExecStop=/usr/bin/boinccmd --quit
ExecReload=/usr/bin/boinccmd --read_cc_config
ExecStopPost=/bin/rm -f /var/lib/boinc/lockfile
IOSchedulingClass=idle
Environment=LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64

[Install]
WantedBy=multi-user.target

The problem is that by doing that I obtain error message
No usable GPU found
. I already gave to BOINC all SELinux permissions, and I have also done some tries with file permissions on the mentioned OpenCL libraries.
What do you suggest me to do?
ID: 75367 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 75379 - Posted: 18 Jan 2017, 21:33:11 UTC - in response to Message 75367.  

I'm not quite sure if I understand what you are doing here. Looking at the log you seem to be using Mesa OpenCL for Polaris card and AMDGPU for Ellesmere. I would have thought you'd want them the other way around?

Well anyway. With LD_LIBRARY_PATH in unit file did GPU detection break or continue to not work? Does it appear in /proc/pid-of-boinc/environ ? If it doesn't try adding something like FOO=bar . LD_LIBRARY_PATH is a bit special, it can be used to create security holes.
ID: 75379 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75390 - Posted: 19 Jan 2017, 11:10:58 UTC - in response to Message 75379.  
Last modified: 19 Jan 2017, 11:11:47 UTC

Looking at the log you seem to be using Mesa OpenCL for Polaris card and AMDGPU for Ellesmere. I would have thought you'd want them the other way around?

I am not very expert about OpenCL libraries, indeed this is the first time I hear about Ellesmere

With LD_LIBRARY_PATH in unit file did GPU detection break or continue to not work?

I get these messages
gio 19 gen 2017 12:06:41 CET |  | cc_config.xml not found - using defaults
gio 19 gen 2017 12:06:41 CET |  | Starting BOINC client version 7.6.22 for x86_64-pc-linux-gnu
gio 19 gen 2017 12:06:41 CET |  | log flags: file_xfer, sched_ops, task
gio 19 gen 2017 12:06:41 CET |  | Libraries: libcurl/7.51.0 NSS/3.27 zlib/1.2.8 libidn2/0.11 libpsl/0.14.0 (+libidn2/0.10) libssh2/1.8.0 nghttp2/1.13.0
gio 19 gen 2017 12:06:41 CET |  | Running as a daemon
gio 19 gen 2017 12:06:41 CET |  | Data directory: /var/lib/boinc
gio 19 gen 2017 12:06:41 CET |  | OpenCL CPU: pthread-AMD Phenom(tm) II X4 965 Processor (OpenCL driver vendor: The pocl project, driver version 0.13, device version OpenCL 2.0 pocl)
gio 19 gen 2017 12:06:41 CET |  | No usable GPUs found
gio 19 gen 2017 12:06:41 CET |  | Processor: 4 AuthenticAMD AMD Phenom(tm) II X4 965 Processor [Family 16 Model 4 Stepping 3]
gio 19 gen 2017 12:06:41 CET |  | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid eagerfpu pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate vmmcall npt lbrv svm_lock nrip_save
gio 19 gen 2017 12:06:41 CET |  | OS: Linux: 4.9.3-200.fc25.x86_64
gio 19 gen 2017 12:06:41 CET |  | Memory: 15.67 GB physical, 1.95 GB virtual
gio 19 gen 2017 12:06:41 CET |  | Disk: 456.47 GB total, 28.55 GB free
gio 19 gen 2017 12:06:41 CET |  | Local time is UTC +1 hours
[cut]
gio 19 gen 2017 12:06:41 CET |  | Reading preferences override file
gio 19 gen 2017 12:06:41 CET |  | Preferences:
gio 19 gen 2017 12:06:41 CET |  | max memory usage when active: 8022.21MB
gio 19 gen 2017 12:06:41 CET |  | max memory usage when idle: 14439.97MB
gio 19 gen 2017 12:06:41 CET |  | max disk usage: 28.57GB
gio 19 gen 2017 12:06:41 CET |  | don't use GPU while active
gio 19 gen 2017 12:06:41 CET |  | (to change preferences, visit a project web site or select Preferences in the Manager)
gio 19 gen 2017 12:06:41 CET |  | gui_rpc_auth.cfg is empty - no GUI RPC password protection
gio 19 gen 2017 12:06:41 CET |  | Suspending computation - initial delay



Does it appear in /proc/pid-of-boinc/environ ?

yes
# cat /proc/4333/environ 
LANG=it_IT.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binHOME=/var/lib/boincLOGNAME=boincUSER=boincSHELL=/sbin/nologinJOURNAL_STREAM=8:178191LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64


LD_LIBRARY_PATH is a bit special, it can be used to create security holes.

Oh I did not know about that, is it so dangerous? What should I use instead?
ID: 75390 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 75399 - Posted: 19 Jan 2017, 21:30:25 UTC - in response to Message 75390.  

Looking at the log you seem to be using Mesa OpenCL for Polaris card and AMDGPU for Ellesmere. I would have thought you'd want them the other way around?

I am not very expert about OpenCL libraries, indeed this is the first time I hear about Ellesmere


I can't remember the code names. I misread the memory size as 768 MB and assumed it was an older card. Oops.

With LD_LIBRARY_PATH in unit file did GPU detection break or continue to not work?

I get these messages


Ok. I meant, does GPU detection work at all without LD_LIBRARY_PATH, did things change worse or is there no change?

I don't know if it still is but adding user 'boinc' to group 'video' used to be necessary.

LD_LIBRARY_PATH is a bit special, it can be used to create security holes.

Oh I did not know about that, is it so dangerous? What should I use instead?


It's nothing you need to worry but some system programs remove it as safety precaution. That was not the case with systemd.
ID: 75399 · Report as offensive
Profile Agentb
Help desk expert
Avatar

Send message
Joined: 30 May 15
Posts: 265
United Kingdom
Message 75415 - Posted: 20 Jan 2017, 22:48:55 UTC

Ellesmere is the codename for the RX-470 an RX-480, Polaris10 is a psudonym for Ellesmere - see https://en.wikipedia.org/wiki/AMD_Radeon_400_series

So this host has only one graphics card or two?
ID: 75415 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75438 - Posted: 22 Jan 2017, 17:20:32 UTC - in response to Message 75415.  

So this host has only one graphics card or two?

One
ID: 75438 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75440 - Posted: 22 Jan 2017, 17:26:50 UTC - in response to Message 75399.  

Ok. I meant, does GPU detection work at all without LD_LIBRARY_PATH, did things change worse or is there no change?

There is no change using
Environment=/opt/amdgpu-pro/lib64

in systemd unit file

I don't know if it still is but adding user 'boinc' to group 'video' used to be necessary.

My user is not in video group, and running boinc_client manually I managed to let it detect the GPU. So there should be no reason to add boinc user to video group
ID: 75440 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 75458 - Posted: 23 Jan 2017, 21:15:06 UTC - in response to Message 75440.  

Does it work if you let boinc access X:

xhost +si:localuser:boinc


If it works you'll need to run the command and restart the client every time you restart X or put the commands in your display manager's startup script.
ID: 75458 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75459 - Posted: 23 Jan 2017, 22:20:11 UTC - in response to Message 75458.  

Does it work if you let boinc access X:

xhost +si:localuser:boinc


If it works you'll need to run the command and restart the client every time you restart X or put the commands in your display manager's startup script.

It does not work: I have tried first with
Environment=LD_LIBRARY_PATH=/opt/amdgpu-pro/lib64

then with
Environment=/opt/amdgpu-pro/lib64


How can I revert
xhost +si:localuser:boinc

?
ID: 75459 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 75537 - Posted: 28 Jan 2017, 21:48:07 UTC - in response to Message 75459.  

How can I revert
xhost +si:localuser:boinc


With this:

xhost -si:localuser:boinc


(+ adds, - removes)

How about running the client from within X but as user boinc:

sudo -u boinc /usr/bin/boinc --dir /var/lib/boinc


Or if that fails:

sudo -E -u boinc /usr/bin/boinc --dir /var/lib/boinc


Check the paths before running the commands. The latter command keeps your environment while running the command as boinc.
ID: 75537 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 75939 - Posted: 15 Feb 2017, 8:51:14 UTC - in response to Message 75537.  

It already runs as boinc user
ID: 75939 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 76014 - Posted: 18 Feb 2017, 20:10:14 UTC - in response to Message 75939.  

The idea I had was that since the client detect GPU when running under your user account and in X session but not when running as service then what happens if those are combined.

Or the other way around, run the client outside of X from virtual console but under your user account. You'll probably need to logout of X and maybe even shutdown X for this test to really work.
ID: 76014 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 76015 - Posted: 18 Feb 2017, 21:05:18 UTC - in response to Message 76014.  

I think that BOINC should be able to run OpenCL working unit even if the host is not running an X org server
ID: 76015 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 682
Finland
Message 76016 - Posted: 18 Feb 2017, 22:15:07 UTC - in response to Message 76015.  

I agree that it would be nice to have it work like that. But whether X is a requirement or not is something we don't know until you test it. If it turns out to be so then I'm afraid there is nothing BOINC can do about it - it is a driver limitation.
ID: 76016 · Report as offensive
jlavi

Send message
Joined: 18 Mar 17
Posts: 3
Finland
Message 76524 - Posted: 18 Mar 2017, 22:03:58 UTC - in response to Message 75367.  

Germano, could you please show the output from 'tree /opt/amdgpu-pro'?

Tried to follow the recipe and darktable crashes before it even gets started. Obviously something is wrong in my setup. I am using amdgpu-pro-16.60-379184 with RX 480.
ID: 76524 · Report as offensive
jlavi

Send message
Joined: 18 Mar 17
Posts: 3
Finland
Message 76526 - Posted: 18 Mar 2017, 22:57:32 UTC - in response to Message 76524.  

Got it working, I guess. At least it doesn't crash and clinfo is reporting sensible data. I copied steps from install script here: https://aur.archlinux.org/packages/opencl-amd/

And my tree loks like:
/opt/amdgpu-pro/
└── lib64
├── libamdocl12cl64.so
├── libamdocl64.so
├── libdrm_amdgpu.so.1
└── libdrm_amdgpu.so.1.0.0
ID: 76526 · Report as offensive
Profile Malcolm Beeson

Send message
Joined: 30 Aug 13
Posts: 13
France
Message 76553 - Posted: 20 Mar 2017, 15:56:21 UTC

Hi

Will this work for a Radeon HD 5770 GPU as well? I'm running Debian 8 and lspci gives me :
"01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]"
So the card is there but BOINC returns the usual "No usable GPU........."

Very sad of Béziers!
ID: 76553 · Report as offensive
jlavi

Send message
Joined: 18 Mar 17
Posts: 3
Finland
Message 76566 - Posted: 21 Mar 2017, 14:06:17 UTC - in response to Message 76526.  
Last modified: 21 Mar 2017, 14:14:23 UTC

├── libdrm_amdgpu.so.1

libdrm_amdgpu.so.1 should be a symbolic link pointing to libdrm_amdgpu.so.1.0.0 instead of regular file. Well, a beauty and style issue, both ways work. When copying with cp alone without d/a option, the link was copied as a file. The missing link was the reason for segfaults.

I have RX480. I've tested clinfo both distribution version (clinfo -l is nice) and AMD's version, cl-demo (from OpenCLHowto), MemtestCL, Darktable ("darktable -d opencl" is very informing), Boinc command line version and Hashcat (hashcat -I is good) and they all work with OpenCL. Agisoft Photoscan also works after editing its startup script so that LD_LIBRARY_PATH is not written over.

With MemtestCL I got random block errors with 16.40 but no errors with 16.60. These errors have been seen in the past and it was then a synchronization bug in memtestCL which was fixed in 2012 by adding a barrier. Possibly the 16.40 had buggy barrier implementation.
ID: 76566 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 76567 - Posted: 21 Mar 2017, 14:20:57 UTC - in response to Message 76524.  

Germano, could you please show the output from 'tree /opt/amdgpu-pro'?

Tried to follow the recipe and darktable crashes before it even gets started. Obviously something is wrong in my setup. I am using amdgpu-pro-16.60-379184 with RX 480.

$ ls -latr /opt/amdgpu-pro/lib64
totale 109712
lrwxrwxrwx. 1 root root       15 30 nov 04.29 libkms.so.1 -> libkms.so.1.0.0
lrwxrwxrwx. 1 root root       15 30 nov 04.29 libkms.so -> libkms.so.1.0.0
lrwxrwxrwx. 1 root root       15 30 nov 04.29 libdrm.so.2 -> libdrm.so.2.4.0
lrwxrwxrwx. 1 root root       15 30 nov 04.29 libdrm.so -> libdrm.so.2.4.0
lrwxrwxrwx. 1 root root       22 30 nov 04.29 libdrm_radeon.so.1 -> libdrm_radeon.so.1.0.1
lrwxrwxrwx. 1 root root       22 30 nov 04.29 libdrm_radeon.so -> libdrm_radeon.so.1.0.1
lrwxrwxrwx. 1 root root       22 30 nov 04.29 libdrm_amdgpu.so.1 -> libdrm_amdgpu.so.1.0.0
lrwxrwxrwx. 1 root root       22 30 nov 04.29 libdrm_amdgpu.so -> libdrm_amdgpu.so.1.0.0
-rwxrwxrwx. 1 root root    15528 30 nov 04.30 libkms.so.1.0.0
-rwxrwxrwx. 1 root root    16010 30 nov 04.30 libkms.a
-rwxrwxrwx. 1 root root    58288 30 nov 04.30 libdrm.so.2.4.0
-rwxrwxrwx. 1 root root    48816 30 nov 04.30 libdrm_radeon.so.1.0.1
-rwxrwxrwx. 1 root root    59272 30 nov 04.30 libdrm_radeon.a
-rwxrwxrwx. 1 root root    53056 30 nov 04.30 libdrm_amdgpu.so.1.0.0
-rwxrwxrwx. 1 root root    71860 30 nov 04.30 libdrm_amdgpu.a
-rwxrwxrwx. 1 root root    80564 30 nov 04.30 libdrm.a
-rwxrwxrwx. 1 root root    27336 30 nov 05.30 libOpenCL.so.1
lrwxrwxrwx. 1 root root       14 30 nov 05.30 libOpenCL.so -> libOpenCL.so.1
-rwxrwxrwx. 1 root root 69343768 30 nov 05.30 libamdocl64.so
-rwxrwxrwx. 1 root root 42534400 30 nov 05.30 libamdocl12cl64.so
drwxrwxrwx. 6 root root     4096 17 gen 14.53 ..
drwxrwxrwx. 2 root root     4096 17 gen 14.53 pkgconfig
drwxrwxrwx. 3 root root     4096 17 gen 14.53 .

ID: 76567 · Report as offensive
Germano

Send message
Joined: 21 May 16
Posts: 26
Italy
Message 78481 - Posted: 2 Jun 2017, 8:41:51 UTC

Any idea?
ID: 78481 · Report as offensive
1 · 2 · Next

Message boards : GPUs : [Linux] Trick to use AMDGPU-Pro OpenCL capabilities without installing the whole propertary driver stack

Copyright © 2017 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.