Ubuntu 10.04 Build Setup
This guide covers how to take a bone stock Ubuntu 10.04 system from out of the box to a full Cryptohaze dev environment for both CUDA and OpenCL. It assumes you have both an nVidia and an ATI GPU installed in the system, with the display attached to the ATI card (because otherwise it's more difficult - CUDA works better headless than ATI does, as ATI requires X11 to communicate with the card). We'll also install the Intel OpenCL runtime. If you have an AMD CPU, this step is not needed, because the AMD OpenCL SDK is for both their CPUs and GPUs.
If you think you'll just use the AMD OpenCL SDK with Intel, think again. Performance sucks. :)
Please go through all these steps carefully! I've tested them and they work. If not, please either fix them or ping me with details!
Install Ubuntu 10.04 Desktop 64-bit. Why? Because 32-bit sucks, that's why. You can make all this work on server, but I'm assuming this is a local development system, which you will be sitting in front of. I would highly suggest installing with only the ATI card - it's easier to make nVidia work after everything is set up.
After you've installed, update the system fully. This will bring you around to the newest kernel version, updated everything, etc. Why? Because you'll be installing kernel modules, and it's annoying to have to reinstall them after the system is up and running.
sudo apt-get update sudo apt-get dist-upgrade
Install the base packages you'll need - these are a good start on any Linux install.
sudo apt-get install build-essential screen vim ia32-libs openjdk-6-jre openjdk-6-jdk
You'll need to now download a bunch of things.
- ATI Driver for Linux (Currently Catalyst 12.1)
- ATI APP SDK for Linux
- nVidia developer drivers for Linux
- nVidia CUDA Toolkit for Ubuntu 10.04
- nVidia GPU SDK for Linux
- Intel OpenCL for Linux
- Boost from the website (1.47 or higher, currently 1.48).
- Netbeans (currently 7.1, and I like the "all" download)
Once these are all downloaded, proceed.
Install the ATI driver
https://help.ubuntu.com/community/BinaryDriverHowto/ATI is a good guide to this.
We'll be building a .deb package to install, as this should, in theory, improve the ability to actually uninstall it later on.
First, install libqtgui4:
sudo apt-get install libqtgui4
Now, change to your downloads directory, and run the installer to build the .deb packages:
sudo sh amd-driver-installer-12-1-x86.x86_64.run --buildpkg Ubuntu/lucid
This will probably pop up an Ubuntu package installer window - let it run, close when finished. It's just installing stuff it needs.
This will likely take a while to run. When it's done, you should have 4 packages created in Downloads - fglrx*.deb.
Now, install them.
sudo apt-get install dkms sudo dpkg -i *.deb
Check for errors, and if there are none, continue.
The next step is to set up the adapters:
sudo aticonfig --initial -f --adapter=all
This will overwrite your current /etc/X11/xorg.conf with the required data for the new driver.
Now, reboot. If all goes well, you'll start up with the new ATI driver. Since this is ATI and Linux we're talking about, good luck - if it fails, you're mostly on your own. Google for solutions until you get something working.
If you've started with a clean, shiny 10.04 install, and followed these steps exactly, there's a non-zero chance it will work out of the box. It did for me!
Once you've restarted, verify that things are working with:
You should get a description of your card version and it should actually run successfully.
You can also verify the build date of the fglrx module in the kernel:
dmesg | grep fglrx
Look for the line containing the module date, and verify that it's fairly recent.
If this is all done, you've successfully installed the ATI drivers under Linux! Congratulations! The hard part is done.
Now is a great time to physically install your nVidia card with no displays attached.
Install nVidia Drivers
Now you get to install the nVidia drivers.
NOTE: For whatever reason, GTX295s don't seem to play nice with ATI cards, at least in my system.
Step 1 is to blacklist any of the other drivers that may load.
Edit /etc/modprobe.d/blacklist.conf and add the following:
When the system comes back up, switch to the text console (ctrl-alt-F1) and log in. You need to kill off X to get the nVidia drivers installed:
sudo service gdm stop
Now, install the drivers:
sudo sh Downloads/NVIDIA_290*
Accept the license. The pre-install script failing is normal on Ubuntu.
Say "No" to 32-bit OpenGL libraries.
Say "No" to updating the X configuration file. This is important!
Now, because nVidia and ATI both like some of the same files (I think), you should probably reinstall the fglrx packages you created earlier, or the system may not bring up X again.
cd ~/Downloads sudo dpkg -i fglrx*.deb
If you do get a black screen with X11, reboot in recovery mode, and reinstall the fglrx packages you created.
Reboot, and you should be at X11 again, with both drivers installed.
However, the nVidia driver won't auto-create the device nodes unless X is started, or a tool is started as root. That's annoying.
So, borrowing from here: http://foldingforum.org/viewtopic.php?f=54&t=6793
We just create the nvidia nodes!
Create a new file: /etc/rc2.d/S99nvidia
Paste the following in it:
#!/bin/bash PATH=$PATH:/usr/local/cuda/bin modprobe nvidia if [ "$?" -eq 0 ]; then # Count the number of NVIDIA controllers found. N3D=`/usr/bin/lspci | grep -i NVIDIA | grep "3D controller" | wc -l` NVGA=`/usr/bin/lspci | grep -i NVIDIA | grep "VGA compatible controller" | wc -l` N=`expr $N3D + $NVGA - 1` for i in `seq 0 $N`; do mknod -m 666 /dev/nvidia$i c 195 $i; done mknod -m 666 /dev/nvidiactl c 195 255 else exit 1 fi
Save, exit, and make it executable:
sudo chmod 755 /etc/rc2.d/S99nvidia
Now, reboot again, and when you come back up, do the following:
ls -l /dev/ | grep nv
You should see nvidia0 & nvidiactl listed.
If you have a CUDA app (say, the Cryptohaze tools), you should be able to enumerate the CUDA capable devices as a non-root user:
cd Cryptohaze-Linux/ ./Cryptohaze-Multiforcer -q
And get output like, say:
CUDA Device Information: Device 0: "GeForce GTX 260" Integrated: 0 Can map host mem: 1 Number of cores: 216 Clock rate: 1.24 GHz Performance Number: 268272 Note: Performance number is clock in mhz * core count, for comparing devices.
You've got both drivers installed, both GPUs installed, and the system (hopefully) still boots. The rough part is over!
Install the nVidia CUDA toolchain & SDK
Install the CUDA toolkit with the stock options.
cd Downloads sudo sh ./cudatoolkit_4*
The default install path is preferred.
Now you need to add some stuff so your shell can find the tools and libraries.
Edit /etc/profile as root (sudo vim /etc/profile)
Add the following line at the end:
Now, edit /etc/ld.so.conf as root (sudo vim /etc/ld.so.conf) and add the following:
Now, update the cache:
Log out and back in, and things should work.
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2011 NVIDIA Corporation Built on Thu_Jan_12_14:41:45_PST_2012 Cuda compilation tools, release 4.1, V0.2.1221
If this is true, you can move to the next step!
To install the SDK into your home directory, run:
cd ~/Downloads sh gpucomputing*
Note that you don't need or want to run this as root - it installs into your home directory under your user.
The defaults are fine, and it should install quickly.
Install the needed libraries for the CUDA SDK build:
sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev
Fix an OpenGL library issue that will block some builds:
sudo rm /usr/lib/libGL.so sudo ln -s /usr/lib/libGL.so.1 /usr/lib/libGL.so
Now, make the SDK.
cd ~/NVIDIA_GPU_Computing_SDK make
If this works, you're done installing the SDK! Awesome!
Installing the AMD SDK
Actually, to be honest, I'm not sure why you need to do this. The drivers handle the OpenCL compiler, and unless you want to do something specifically with their SDK, there's just no need to install it. The nVidia stuff handles all the headers you need.
But, if you want to install it, I'll provide documentation at some point.
Now, it's time to install Boost. We'll go with Boost 1.48 for now.
Download it from boost.org. I personally use the .tar.bz2, but they're all the same.
Let's install it!
sudo apt-get install python-dev libbz2-dev cd ~/Downloads tar -xvjf boost_1_48_0.tar.bz2 cd boost_1_48_0 ./bootstrap.sh ./b2 -j [number of CPUs]
Time for coffee. This will take a while.
sudo ./b2 install --prefix=/usr/local/
And now, you have Boost 1.48 installed! YAY!
cd Downloads sudo sh netbeans-7.1-ml-linux.sh
Defaults are fine. I like to install the kitchen sink initially so I don't have to download it later.
You'll need to install the Google Protbuf stack to build the tools now.
Get it here: http://code.google.com/p/protobuf/downloads/list
tar -cvjf protobuf-2.4.1.tar.bz2 cd protobuf-2.4.1 ./configure make sudo make install
Should be good to go with this!
Download the Cryptohaze repo and set up to build it
Finally - Cryptohaze!
Install the needed prereqs and libraries.
cd sudo apt-get install subversion libargtable2-dev libargtable2-0 libncurses5 libncurses5-dev libcurl4-openssl-dev svn co https://cryptohaze.svn.sourceforge.net/svnroot/cryptohaze/Cryptohaze-Combined ~/Cryptohaze-Dev
Now, in THEORY, you can just run make!
cd ~/Cryptohaze-Dev/ make
And, if you've followed the steps correctly, and I haven't messed up, it should build cleanly and run! ZOMG!
Setting up Netbeans for sanity
So, you want to make Netbeans actually useful instead of a Java text editor.
Step 1 is to create a new project with the Cryptohaze files.
File->New Project, C/C++, C/C++ Project with Existing Sources.
Select the Cryptohaze directory and you should be good to go - it will do a make clean & then a make.
Now, you can view the files, edit, etc. But there's a problem - tons and tons of red, squiggly, "I have no idea what this is!" messages.
That's because it doesn't know where to look for all the include files. But we can fix this!
Go to Tools->Options, C/C++, Code Assistance, C++ Compiler.
Add all of the Cryptohaze-Dev/inc directories except the "windows" directory (as that will mess up Unix includes).
Now, add "/usr/local/include" - this will get the boost stuff.
Add "/usr/local/cuda/include" - this will most of the CUDA related files.
And, add "~/NVIDIA_GPU_Computing_SDK/C/common/inc/" - this gets the last of the CUDA files.
The final step is to tell it that a .cu file is really a C/C++ file, and should be interpreted as such.
Go to Tools->Options, C++, Other. Add "cu" as a C++ file extension.
Now everything should be working smoothly, with full object completion, syntax checking, etc. Enjoy!