Ubuntu 10.04 Build Setup

This guide covers how to take a bone stock Ubuntu 10.04 system from out of the box to a full Cryptohaze dev environment for both CUDA and OpenCL. It assumes you have both an nVidia and an ATI GPU installed in the system, with the display attached to the ATI card (because otherwise it's more difficult - CUDA works better headless than ATI does, as ATI requires X11 to communicate with the card). We'll also install the Intel OpenCL runtime. If you have an AMD CPU, this step is not needed, because the AMD OpenCL SDK is for both their CPUs and GPUs.

If you think you'll just use the AMD OpenCL SDK with Intel, think again. Performance sucks. :)

Please go through all these steps carefully! I've tested them and they work. If not, please either fix them or ping me with details!

Base Setup

Install Ubuntu 10.04 Desktop 64-bit. Why? Because 32-bit sucks, that's why. You can make all this work on server, but I'm assuming this is a local development system, which you will be sitting in front of. I would highly suggest installing with only the ATI card - it's easier to make nVidia work after everything is set up.

After you've installed, update the system fully. This will bring you around to the newest kernel version, updated everything, etc. Why? Because you'll be installing kernel modules, and it's annoying to have to reinstall them after the system is up and running.

sudo apt-get update
sudo apt-get dist-upgrade

Install the base packages you'll need - these are a good start on any Linux install.

sudo apt-get install build-essential screen vim ia32-libs openjdk-6-jre openjdk-6-jdk

Download Software

You'll need to now download a bunch of things.

• ATI Driver for Linux (Currently Catalyst 12.1)
• ATI APP SDK for Linux
• nVidia developer drivers for Linux
• nVidia CUDA Toolkit for Ubuntu 10.04
• nVidia GPU SDK for Linux
• Intel OpenCL for Linux
• Boost from the website (1.47 or higher, currently 1.48).
• Netbeans (currently 7.1, and I like the "all" download)

Once these are all downloaded, proceed.

Install the ATI driver

https://help.ubuntu.com/community/BinaryDriverHowto/ATI is a good guide to this.

We'll be building a .deb package to install, as this should, in theory, improve the ability to actually uninstall it later on.

First, install libqtgui4:

sudo apt-get install libqtgui4

Now, change to your downloads directory, and run the installer to build the .deb packages:

sudo sh amd-driver-installer-12-1-x86.x86_64.run --buildpkg Ubuntu/lucid

This will probably pop up an Ubuntu package installer window - let it run, close when finished. It's just installing stuff it needs.

This will likely take a while to run. When it's done, you should have 4 packages created in Downloads - fglrx*.deb.

Now, install them.

sudo apt-get install dkms
sudo dpkg -i *.deb

Check for errors, and if there are none, continue.

The next step is to set up the adapters:

sudo aticonfig --initial -f --adapter=all

This will overwrite your current /etc/X11/xorg.conf with the required data for the new driver.

Now, reboot. If all goes well, you'll start up with the new ATI driver. Since this is ATI and Linux we're talking about, good luck - if it fails, you're mostly on your own. Google for solutions until you get something working.

If you've started with a clean, shiny 10.04 install, and followed these steps exactly, there's a non-zero chance it will work out of the box. It did for me!

Once you've restarted, verify that things are working with:

fglrxinfo

You should get a description of your card version and it should actually run successfully.

You can also verify the build date of the fglrx module in the kernel:

dmesg | grep fglrx

Look for the line containing the module date, and verify that it's fairly recent.

If this is all done, you've successfully installed the ATI drivers under Linux! Congratulations! The hard part is done.

Now is a great time to physically install your nVidia card with no displays attached.

Install nVidia Drivers

Now you get to install the nVidia drivers.

NOTE: For whatever reason, GTX295s don't seem to play nice with ATI cards, at least in my system.

Step 1 is to blacklist any of the other drivers that may load.

Edit /etc/modprobe.d/blacklist.conf and add the following:

blacklist nouveau

Now reboot.

When the system comes back up, switch to the text console (ctrl-alt-F1) and log in. You need to kill off X to get the nVidia drivers installed:

sudo service gdm stop

Now, install the drivers:

sudo sh Downloads/NVIDIA_290*

Accept the license. The pre-install script failing is normal on Ubuntu.

Say "No" to 32-bit OpenGL libraries.

Say "No" to updating the X configuration file. This is important!

Now, because nVidia and ATI both like some of the same files (I think), you should probably reinstall the fglrx packages you created earlier, or the system may not bring up X again.

cd ~/Downloads
sudo dpkg -i fglrx*.deb

If you do get a black screen with X11, reboot in recovery mode, and reinstall the fglrx packages you created.

Reboot, and you should be at X11 again, with both drivers installed.

However, the nVidia driver won't auto-create the device nodes unless X is started, or a tool is started as root. That's annoying.

So, borrowing from here: http://foldingforum.org/viewtopic.php?f=54&t=6793

We just create the nvidia nodes!

Create a new file: /etc/rc2.d/S99nvidia

Paste the following in it:

#!/bin/bash

PATH=$PATH:/usr/local/cuda/bin

modprobe nvidia

if [ "$?" -eq 0 ]; then

# Count the number of NVIDIA controllers found.
N3D=`/usr/bin/lspci | grep -i NVIDIA | grep "3D controller" | wc -l`
NVGA=`/usr/bin/lspci | grep -i NVIDIA | grep "VGA compatible controller" | wc -l`

N=`expr $N3D + $NVGA - 1`
for i in `seq 0 $N`; do
mknod -m 666 /dev/nvidia$i c 195 $i;
done

mknod -m 666 /dev/nvidiactl c 195 255

else
exit 1
fi

Save, exit, and make it executable:

sudo chmod 755 /etc/rc2.d/S99nvidia

Now, reboot again, and when you come back up, do the following:

ls -l /dev/ | grep nv

You should see nvidia0 & nvidiactl listed.

If you have a CUDA app (say, the Cryptohaze tools), you should be able to enumerate the CUDA capable devices as a non-root user:

cd Cryptohaze-Linux/
./Cryptohaze-Multiforcer -q

And get output like, say:

CUDA Device Information:
Device 0: "GeForce GTX 260"
  Integrated:                                    0
  Can map host mem:                              1
  Number of cores:                               216
  Clock rate:                                    1.24 GHz
  Performance Number:                            268272
  Note: Performance number is clock in mhz * core count, for comparing devices.

Awesome!

You've got both drivers installed, both GPUs installed, and the system (hopefully) still boots. The rough part is over!

Install the nVidia CUDA toolchain & SDK

Install the CUDA toolkit with the stock options.

cd Downloads
sudo sh ./cudatoolkit_4*

The default install path is preferred.

Now you need to add some stuff so your shell can find the tools and libraries.

Edit /etc/profile as root (sudo vim /etc/profile)

Add the following line at the end:

export PATH=/usr/local/cuda/bin:$PATH

Now, edit /etc/ld.so.conf as root (sudo vim /etc/ld.so.conf) and add the following:

/usr/local/cuda/lib64
/usr/local/cuda/lib

Now, update the cache:

sudo ldconfig

Log out and back in, and things should work.

nvcc --version

Should output:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2011 NVIDIA Corporation
Built on Thu_Jan_12_14:41:45_PST_2012
Cuda compilation tools, release 4.1, V0.2.1221

If this is true, you can move to the next step!

To install the SDK into your home directory, run:

cd ~/Downloads
sh gpucomputing*

Note that you don't need or want to run this as root - it installs into your home directory under your user.

The defaults are fine, and it should install quickly.

Install the needed libraries for the CUDA SDK build:

sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev

Fix an OpenGL library issue that will block some builds:

sudo rm /usr/lib/libGL.so
sudo ln -s /usr/lib/libGL.so.1 /usr/lib/libGL.so

Now, make the SDK.

cd ~/NVIDIA_GPU_Computing_SDK
make

If this works, you're done installing the SDK! Awesome!

Installing the AMD SDK

Actually, to be honest, I'm not sure why you need to do this. The drivers handle the OpenCL compiler, and unless you want to do something specifically with their SDK, there's just no need to install it. The nVidia stuff handles all the headers you need.

But, if you want to install it, I'll provide documentation at some point.

Install Boost

Now, it's time to install Boost. We'll go with Boost 1.48 for now.

Download it from boost.org. I personally use the .tar.bz2, but they're all the same.

Let's install it!

sudo apt-get install python-dev libbz2-dev
cd ~/Downloads
tar -xvjf boost_1_48_0.tar.bz2
cd boost_1_48_0
./bootstrap.sh
./b2 -j [number of CPUs]

Time for coffee. This will take a while.

Install now!

sudo ./b2 install --prefix=/usr/local/

And now, you have Boost 1.48 installed! YAY!

Netbeans

cd Downloads
sudo sh netbeans-7.1-ml-linux.sh

Defaults are fine. I like to install the kitchen sink initially so I don't have to download it later.

Google Protobuf

You'll need to install the Google Protbuf stack to build the tools now.

Get it here: http://code.google.com/p/protobuf/downloads/list

tar -cvjf protobuf-2.4.1.tar.bz2
cd protobuf-2.4.1
./configure
make
sudo make install

Should be good to go with this!

Download the Cryptohaze repo and set up to build it

Finally - Cryptohaze!

Install the needed prereqs and libraries.

cd
sudo apt-get install subversion libargtable2-dev libargtable2-0 libncurses5 libncurses5-dev libcurl4-openssl-dev
svn co https://cryptohaze.svn.sourceforge.net/svnroot/cryptohaze/Cryptohaze-Combined ~/Cryptohaze-Dev

Now, in THEORY, you can just run make!

cd ~/Cryptohaze-Dev/
make

And, if you've followed the steps correctly, and I haven't messed up, it should build cleanly and run! ZOMG!

Setting up Netbeans for sanity

So, you want to make Netbeans actually useful instead of a Java text editor.

Awesome.

Step 1 is to create a new project with the Cryptohaze files.

File->New Project, C/C++, C/C++ Project with Existing Sources.

Select the Cryptohaze directory and you should be good to go - it will do a make clean & then a make.

Now, you can view the files, edit, etc. But there's a problem - tons and tons of red, squiggly, "I have no idea what this is!" messages.

That's because it doesn't know where to look for all the include files. But we can fix this!

Go to Tools->Options, C/C++, Code Assistance, C++ Compiler.

Add all of the Cryptohaze-Dev/inc directories except the "windows" directory (as that will mess up Unix includes).

Now, add "/usr/local/include" - this will get the boost stuff.

Add "/usr/local/cuda/include" - this will most of the CUDA related files.

And, add "~/NVIDIA_GPU_Computing_SDK/C/common/inc/" - this gets the last of the CUDA files.

The final step is to tell it that a .cu file is really a C/C++ file, and should be interpreted as such.

Go to Tools->Options, C++, Other. Add "cu" as a C++ file extension.

Now everything should be working smoothly, with full object completion, syntax checking, etc. Enjoy!