Setup Instructions

To develop the Pintos projects, you’ll need two essential sets of tools:

  • 80x86 cross-compiler toolchain for 32-bit architecture including a C compiler, assembler, linker, and debugger.
  • x86 emulator, QEMU or Bochs

This page contains instructions to help you with the setup of the core development environment needed for Pintos on your own machines. They are intended for Unix and Mac OS machines. If you are running Windows, we recommend you to run a virtual machine with Linux or you will have to setup Cygwin first. This guide, and the course in general, assumes you are familiar with Unix commands.

Getting Started

To get started, you need to clone the Pintos repository (StarFork) we provided.

$ git clone https://github.com/jhu-cs318/pintos.git

1. Development Environment

Option A: JHU CS Lab Machine

The CS department’s lab machines already have these tools available under /usr/local/data/cs318/x86_64. You just need to modify your PATH setting to include it.

  • See what shell you are using with echo $SHELL.
  • For Bash, that is to put the following at the end of your ~/.bash_profile:
    $ export PATH=/usr/local/data/cs318/x86_64/bin:$PATH
    
  • For tcsh (the default login shell in ugrad lab machines in the CS department), the syntax is different: add set path = (/usr/local/data/cs318/x86_64/bin $path) to the end of your ~/.tcshrc. Log out and re-login to let it take effect.

If you are using macOS to connect to the CS Lab machine, you should install XQuartz, so that you can get GUI windows to show up locally in your macOS for the exercise 0.1 in Lab 0.

Choosing option A, you do NOT need to follow steps 2-4 below.

Option B: Your Own Machine

Besides the lab machines, you may want to work on the projects on your own laptop/desktop to be more productive.

  • If you computer runs Linux, it is the perfect development environment for Pintos. In terms of suitable Linux distributions, Ubuntu 18.04 is what we use. But you may also try other distributions like Ubuntu 20.04 or Fedora.
  • If your computer runs macOS, it should work too. You have two choices: (i) follow step 2 below to build the toolchain, so you can later compile Pintos natively on macOS; (ii) install a Linux virtual machine (VM). The first choice is recommended because native compilation is faster, but feel free to go with the second choice.
  • If you are using a Windows machine, however, we recommend you to install a Linux virtual machine.
For virtual-machine-based development, you can choose our pre-built VirtualBox VM running Ubuntu 18.04 with the necessary toolchain installed. The VM image is available here. The image is large (2.5 GB), so the downloading can take a while. The md5sum for the VM image is 69c89938d4b768bdcca4362fd39f06e4. The initial login password is jhucs318.
If you would like to build/use your own virtual machine, Vagrant is a popular tool to make VM-based development environment convenient. We have provided a Vagrant configuration that you can use to set up your Pintos dev box.
Unless you are using our pre-built VM, you have to follow steps 2-4 below to install the compiler toolchain, the emulator, and the Pintos utility tools.

2. Compiler Toolchain

2.1: Test Your Compiler Toolchain

The compiler toolchain is a collection of tools that turns source code into binary executables for a target architecture. Pintos is written in C and x86 assembly, and runs on 32-bit x86 machines. So we will need the appropriate C compiler (gcc), assembler (as), linker (ld) and debugger (gdb).

If you are using a Linux machine, it is likely equipped with the compiler toolchain already. But it should support 32-bit x86 architecture. A quick test of the support is to run objdump -i | grep elf32-i386 in the terminal. If it returns matching lines, your system’s default tool chain supports the target, then you can skip Section 2.2. Otherwise, you will need to build the toolchain from source.

If you are using MacOS, you have to build the toolchain from source because MacOS's object file format is not ELF that we need, and the objdump -i test won't work.
If you are using Ubuntu 18.04, you might pass the objdump -i test. However, you will likely encounter an issue later presumably due to a gcc 7 toolchain bug (see discussion). It is recommended that you build the toolchain from source according to Section 2.2.
If you are using Ubuntu 16.04, the stock GCC 5.4 should work. If you don't have GCC installed yet, install it with sudo apt-get install build-essential gdb gcc-multilib. The objdump -i test should pass.

2.2: Build Toolchain from Source

When you are building the toolchain from source, to distinguish the new toolchain from your system’s default one, you should add a i386-elf- prefix to the build target, e.g., i386-elf-gcc, i386-elf-as.

2.2.1 Prerequisite

  • Standard build tools including make, gcc, etc. To build GDB, you will need the ncurses and texinfo libraries.
  • For Ubuntu, you can install these packages with
    $ sudo apt-get install build-essential automake git libncurses5-dev texinfo
    
  • For macOS, first you should have the command-line tools in Xcode installed:
    $ xcode-select --install
    

    You can then install ncurses and texinfo with brew:

    $ brew install ncurses texinfo
    

2.2.2 The Easy Way

Note
We've provided a script (pintos/src/misc/toolchain-build.sh) that automates the building instructions. So you can just run the script and modify your PATH setting after the build finishes. The script has been tested on recent version of Ubuntu, Mac OS and Fedora.

Replace /path/to/setup below with a real path to store the toolchain source and build, e.g., /home/ryan/318/toolchain; and replace /path/to/pintos with the real path where you cloned the pintos repo, e.g., ~/318/pintos.

$ SWD=/path/to/setup
$ mkdir -p $SWD
$ cd /path/to/pintos
$ src/misc/toolchain-build.sh $SWD

If the above commands succeeded, add the toolchain path to your PATH environment variable settings in the .bashrc (or .zshrc if you are using zsh) file in your home directory.

$ export PATH=$SWD/x86_64/bin:$PATH

Don’t forget to replace the $SWD above with the real path, e.g., export PATH=/home/ryan/318/toolchain/x86_64/bin:$PATH.

If you are curious to build the toolchain manually, below are the detailed instructions.

  1. Directory and environment variables:
    • Create a setup directory (e.g., ~/318/toolchain) and subdirectories that look like this:
      /path/to/setup
      ├── build
      ├── x86_64
      └── src
      
    • Set the environment variables (remember to replace /path/to/setup with the full path to the actual setup directory you’ve created, e.g., SWD=/home/ryan/318/toolchain).
      $ SWD=/path/to/setup
      $ PREFIX=$SWD/x86_64
      $ export PATH=$PREFIX/bin:$PATH
      $ export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH
      

      For Mac users, the last command is export DYLD_LIBRARY_PATH=$PREFIX/lib:$DYLD_LIBRARY_PATH instead.

  2. GNU binutils:
    • Download:
      $ cd $SWD/src 
      $ wget https://ftp.gnu.org/gnu/binutils/binutils-2.27.tar.gz && tar xzf binutils-2.27.tar.gz
      
    • Build:
      $ mkdir -p $SWD/build/binutils && cd $SWD/build/binutils
      $ ../../src/binutils-2.27/configure --prefix=$PREFIX --target=i386-elf \
        --disable-multilib --disable-nls --disable-werror
      $ make -j8
      $ make install
      
  3. GCC:
    • Download:
      $ cd $SWD/src
      $ wget https://ftp.gnu.org/gnu/gcc/gcc-6.2.0/gcc-6.2.0.tar.bz2 && tar xjf gcc-6.2.0.tar.bz2
      $ cd $SWD/src/gcc-6.2.0 && contrib/download_prerequisites
      
    • Build:
      $ mkdir -p $SWD/build/gcc && cd $SWD/build/gcc
      $ ../../src/gcc-6.2.0/configure --prefix=$PREFIX --target=i386-elf \
      --disable-multilib --disable-nls --disable-werror --disable-libssp \
      --disable-libmudflap --with-newlib --without-headers --enable-languages=c,c++
      $ make -j8 all-gcc 
      $ make install-gcc
      $ make all-target-libgcc
      $ make install-target-libgcc
      
  4. GDB:
    • Download:
      $ cd $SWD/src
      $ wget https://ftp.gnu.org/gnu/gdb/gdb-7.9.1.tar.xz  && tar xJf gdb-7.9.1.tar.xz
      
    • Build:
      $ mkdir -p $SWD/build/gdb && cd $SWD/build/gdb
      $ ../../src/gdb-7.9.1/configure --prefix=$PREFIX --target=i386-elf --disable-werror
      $ make -j8
      $ make install
      
If you are using macOS and the above compilation failed. It might be caused by changes in the latest gcc or libraries on macOS. You can post the errors in the discussion forum. We will investigate. In the meantime, you can try to directly use our pre-built toolchain binaries by following instructions in this [README](https://github.com/jhu-cs318/cross-compiler-toolchain).

2.2.3 Verify Installation

Now check if the toolchain has been installed to the proper place and the version is 6.2.0:

$ which i386-elf-gcc
$ i386-elf-gcc --version
Tips

If you see the command not found error message, check if you have added export PATH=$SWD/x86_64/bin:$PATH to your .bashrc or .zshrc (depending on your shell echo $SHELL). If so, you may need to restart your terminal for the new PATH to take effect.

After successfully building the toolchain, you may delete the source and build directories in $SWD/{src,build} to save space.

3. Emulator

Besides the cross-compiler toolchain, we also need an x86 emulator to run Pintos OS. We will use two popular emulators QEMU and Bochs.

3.1 QEMU

  • QEMU is modern and fast. To install it:
    • For Ubuntu: sudo apt-get install qemu
    • For MacOS: brew install qemu
    • Other: build it from source

3.2 Bochs

  • Bochs is slower than QEMU but provides full emulation (i.e., higher accuracy).
  • For Lab 1, we will use Bochs as the default emulator and for Lab 2-4, we will use QEMU as the default emulator. Nevertheless, nothing will prevent you from using one or another for all the labs.
  • There are some bugs in Bochs that should be fixed when using it with Pintos. Thus, we need to install Bochs from source, and apply the patches that we have provided under pintos/src/misc/bochs*.patch. We will build two versions of Bochs: one, simply named bochs, with the GDB stub enabled, and the other, named bochs-dbg, with the built-in debugger enabled.
  • Version 2.6.2 (note: not 2.2.6) has been tested to work with Pintos. Newer version of Bochs has not been tested.
Note
We have provided a build script pintos/src/misc/bochs-2.6.2-build.sh that will download, patch and build two versions of the Bochs for you. But you need to make sure X11 and its library is installed.
  • For Mac OS, you should install XQuartz 2.7.11 (here). Note: bochs build does not work with XQuartz version 2.8.x. Later XQuartz might prompt you to upgrade to 2.8.x, please do not upgrade it!
  • For Ubuntu, sudo apt-get install libx11-dev libxrandr-dev

($SWD should be set previously, e.g., /home/ryan/318/toolchain)

$ src/misc/bochs-2.6.2-build.sh $SWD/x86_64
  • After build succeeds, make sure the bochs or bochs-db are in PATH. You can verify the install with bochs -h. The output should contain 2.6.2.
To reiterate, if you are using macOS, you should install XQuartz 2.7.11 and do not upgrade it to higher version!. Otherwise, the bochs build will fail.

4. Pintos Utility Tools

The Pintos source distribution comes with a few handy scripts that you will be using frequently. They are located within src/utils/. The most important one is the pintos Perl script, which you will be using to start and run tests in pintos. You need to make sure it can be found in your PATH environment variable. In addition, the src/misc/gdb-macros is provided with a number of GDB macros that you will find useful when you are debugging Pintos. The pintos-gdb is a wrapper around the i386-elf-gdb that reads this macro file at start. It assumes the macro file resides in ../misc.

The commands to do the above setup for the Pintos utilities are: (make sure SWD is set previously to the correct directory path)

$ dest=$SWD/x86_64
$ cd pintos/src/utils && make
$ cp backtrace pintos Pintos.pm pintos-gdb pintos-set-cmdline pintos-mkdisk setitimer-helper squish-pty squish-unix $dest/bin
$ mkdir $dest/misc
$ cp ../misc/gdb-macros $dest/misc

5. Others

  • Required: Perl. Version 5.8.0 or later.
  • Recommended:
    • cgdb (strongly recommended)
      • For Ubuntu, sudo apt-get install cgdb; For macOS, brew install cgdb.
      • Once you installed cgdb, the pintos-gdb script would use cgdb to invoke GDB.
    • ctags
    • cscope
    • NERDTree
    • YouCompleteMe.
  • Optional:
    • GUI IDEs like Eclipse CDT or clion. The instructor has not tried them. Vim or Emacs plus the standard Unix development tools would suffice for the course. But if you can’t live without GUI IDEs. You may explore the setup yourself (potential reference) and let us know if they are helpful!

Happy hacking :)


Ryan Huang | Last updated 2022-12-06 14:59:13 -0500.