[Kernel, courtesy IowaFarmer.com CornCam]

CS 235 Advanced Operating Systems, Fall 2010

Tools

You'll use two sets of tools in this class: an x86 emulator, QEMU, for running your kernel; and a compiler toolchain, including assembler, linker, C compiler, and debugger, for compiling and testing your kernel. Here's the information you'll need to download and install your own copies. This class assumes familiarity with Unix commands throughout.

Compiler Toolchain

A "compiler toolchain" is the set of programs, including C and C++ compilers, assemblers, and linkers, that turn code into executable binaries. You'll need a compiler toolchain that generates code for 32-bit Intel architectures ("x86" architectures) in the ELF binary format. We recommend you configure one of your machines (such as a laptop) with a working toolchain, but if you cannot do this, the School of Engineering's SEASnet and a private class compute server will be made available shortly.

Test Your Compiler Toolchain

Modern Linux and BSD UNIX distributions already provide a toolchain suitable for CS 235. To test your distribution, try the following command:

% objdump -i

If the second line is elf32-i386, you're all set, and don't need to compile your own toolchain.

Using a Virtual Machine

Otherwise, the easiest way to get a compatible toolchain is to install a modern Linux distribution on your computer. With platform virtualization, Linux can cohabitate with your normal computing environment. Installing a Linux virtual machine is a two step process. First, you download the virtualization platform.

VirtualBox (free for Mac, Linux, Windows) — Download page
VMware Player (free for Linux and Windows, registration required)
VMware Fusion ($80 for Mac, $40 with academic discount)

VirtualBox is a little slower and less flexible, but free!

Once the virtualization platform is installed, download a boot disk image for the Linux distribution of your choice.

Ubuntu Desktop is what we use.

This will download a file named something like ubuntu-10.04.1-desktop-i386.iso. Start up your virtualization platform and create a new (32-bit) virtual machine. Use the downloaded Ubuntu image as a boot disk; the procedure differs among VMs but is pretty simple. Type objdump -i, as above, to verify that your toolchain is now set up. You will do your work inside the VM.

Building Your Own Toolchain

This will take longer to set up, but give slightly better performance than a virtual machine, and lets you work in your own familiar environment. (Mac/Unix Only)

We assume that you are installing the toolchain into /usr/local. You will need a fair amount of disk space to compile the tools (around 1GiB). If you don't have that much space, delete each directory after its make install step.

Download the following packages:

Unpack and build the packages. The green bold text shows you how to install into /usr/local, which is what we recommend. To install into a different directory, $PFX, click herenote the differences in lighter type (hide). If you have problems, see below.

export PATH=$PFX/bin:$PATH
export LD_LIBRARY_PATH=$PFX/lib:$LD_LIBRARY_PATH      # On Mac OS X, use DYLD_LIBRARY_PATH

tar xjf gmp-5.0.1.tar.bz2
cd gmp-5.0.1
./configure --prefix=/usr/local--prefix=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..

tar xjf mpfr-3.0.0.tar.bz2
cd mpfr-3.0.0
./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..

tar xzf mpc-0.8.2.tar.gz
cd mpc-0.8.2
./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..


tar xjf binutils-2.20.1.tar.bz2
cd binutils-2.20.1
./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --disable-werror
make
make install             # This step may require privilege (sudo make install)
cd ..

i386-jos-elf-objdump -i
# Should produce output like:
# BFD header file version (GNU Binutils) 2.20.1.20100303
# elf32-i386
#  (header little endian, data little endian)
#   i386...


tar xjf gcc-core-4.5.1.tar.bz2
tar xjf gcc-g++-4.5.1.tar.bz2
cd gcc-4.5.1
mkdir build              # GCC will not compile correctly unless you build in a separate directory
cd build
../configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX --with-mpc=$PFX \
    --target=i386-jos-elf --disable-werror \
    --disable-libssp --disable-libmudflap --with-newlib \
    --without-headers --enable-languages=c,c++
make all-gcc
make install-gcc         # This step may require privilege (sudo make install-gcc)
make all-target-libgcc
make install-target-libgcc     # This step may require privilege (sudo make install-target-libgcc)
cd ../..

i386-jos-elf-gcc -v
# Should produce output like:
# Using built-in specs.
# COLLECT_GCC=i386-jos-elf-gcc
# COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/i386-jos-elf/4.5.1/lto-wrapper
# Target: i386-jos-elf


tar xjf gdb-7.2.tar.bz2
cd gdb-7.2
./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --program-prefix=i386-jos-elf- \
    --disable-werror
make all
make install             # This step may require privilege (sudo make install)
cd ..

Troubleshooting

Q. I can't run make install because I don't have root permission on this machine.

A. Our instructions assume you are installing into the /usr/local directory. However, this may not be allowed in your environment. If you can only install code into your home directory, that's OK. In the instructions above, replace --prefix=/usr/local with --prefix=$HOME (and click here to update the instructions further). You will also need to change your PATH and LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac OS X) environment variables, to inform your shell where to find the tools. For example:

export PATH=$HOME/bin:$PATH
export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH

Enter these lines in your ~/.bashrc file so you don't need to type them every time you log in.

Q. My build fails with an inscrutable message about "library not found".: A. You need to set your LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac OS X) environment variable to include the libraries you built; see above. The environment variable must include the PREFIX/lib directory (for instance, /usr/local/lib).

QEMU Emulator

We provide a patched version of the QEMU emulator with improved support for debugging. (Credit to Kohler and Austin Clements and others at MIT 6.828.) We recommend you compile QEMU from this source; if you use the version that came installed on your Linux distribution, you will not be able to complete all the debugging exercises.

First, download the source:

http://www.cs.ucla.edu/~kohler/class/10f-aos/qemu-0.12.5-aos.tar.bz2

On a Linux machine, you will need to install the SDL graphics library to build QEMU. On Ubuntu or Debian:

sudo aptitude install libsdl1.2-dev

Then unpack, compile, and install QEMU as follows.

tar xjf qemu-0.12.5-aos.tar.bz2
cd qemu-0.12.5-aos
On Mac OS X:
   ./configure --prefix=/usr/local --disable-sdl --enable-cocoa \
      --disable-docs --target-list="i386-softmmu x86_64-softmmu"
On Linux:
   ./configure --prefix=/usr/local --target-list="i386-softmmu x86_64-softmmu"
make
make install             # This step may require privilege (sudo make install)

The QEMU monitor is accessible by pressing Ctrl-Alt-2 inside the QEMU window. Return to normal display with Ctrl-Alt-1. QEMU tends to take control of your mouse. If you can't find a mouse pointer, check the QEMU title bar for text like "Press Ctrl-Alt to exit grab" and, if so, press Ctrl-Alt to regain control.

Source Code Control

You will do your work using a source code control system. This will help you maintain two parallel branches—one with your changes, and one with the labs as we release them. We will support two source code control systems: CVS and Git. Git is by far the better system, and strongly preferred by your instructor for his own work, but this is the first time we're trying it in CS 235, so it will be more experimental.

You don't need to compile CVS or Git yourself. The packaged versions that come with your OS will be good enough. You just need to install them.

Ubuntu/Debian Linux: Run sudo aptitude install cvs git-core
Mac OS X: Install MacPorts, then run sudo port install cvs git-core
Windows: Versions are available; contact the instructor if you need help

Using Cygwin on a Windows Machine

If you have a Windows machine, but cannot run a Linux virtual machine or install a dual-boot configuration, the easiest way to work on labs from home will be to use a class compute server or SEASnet coupled with VNC remote display. Let us know as soon as possible if you want this configuration to work.

Nevertheless, it should be possible to execute many of the above steps and get a functional build environment running on Windows using Cygwin. Here's how to install Cygwin on your Windows machine.

Install Cygwin/X. Follow this procedure. Make sure you select the "xorg-x11-base" package and the "openssh" package. Also select the "flex" and "bison" packages from the development header. This sets up a Unix-like environment on your Windows machine, including an X server.
Start the Cygwin X server by running startxwin.bat. "Run /usr/X11R6/bin/startxwin.bat by double-clicking it in Windows Explorer." http://x.cygwin.com/docs/ug/using.html

You should also be able to compile the class tools and the labs on your Windows machine, once Cygwin has been installed. Let us know if you have success with this.

Using the Class Compute Server

A compute server is available for your use. Request a login from your instructor; your mail should include your preferred username and an SSH public key (use PuTTY on Windows [PuTTY authentication notes]).

Using VNC

VNC is a compressed remote desktop viewing protocol that QEMU understands. Using VNC will be faster; QEMU is very slow over a remote X connection. You will need a VNC viewer on your local computer. Good, free viewers are available for Linux, Mac OS X, and Windows.

When the instructor creates your account, you'll receive a VNC port allocation (something like 5500). Remember this number; we call it VNCPORT below.

Download a VNC viewer. On Linux, we have tested TightVNC (Ubuntu/Debian package xtightvncviewer). On Mac OS X, try Chicken of the VNC (an installer disk image is available). On Windows, TightVNC also works.
Start your VNC viewer in “listening” mode, on port 5500 (display 0). On Linux with TightVNC, try
```
vncviewer -listen 0 -encodings "copyrect zlib raw"
```
(You can leave off the -encodings argument, but display will be slower.) On Mac OS X with Chicken, try "Connection > Listen for Server...", and enter 5500.
Log on to the class server, forwarding the server's VNCPORT to your local port 5500. This will let the server QEMU display into your local VNC window. Sample command:
```
ssh -RVNCPORT:localhost:5500 seek.cs.ucla.edu
```
When running QEMU on the server using make, supply the argument VNC=VNCPORT. For example:
```
make run VNC=VNCPORT
```

Using X windows

This may be simpler if you have trouble finding a VNC viewer. Simply start up an X server and run

ssh -Y seek.cs.ucla.edu

Do your work as usual. The -Y option will forward QEMU's window onto your local display. Try make run-nox if you don't need the QEMU window.

Back to CS 235 Advanced Operating Systems