CSE 522S: Studio 1

Welcome To Linux

"It's a dangerous business, Frodo, going out of your door. You step into the road, and if you don't keep your feet, there is no knowing where you might be swept off to."

—Bilbo, The Fellowship of the Ring, Book 1, Chapter 3

Welcome to Advanced Operating Systems! We're going to start by compiling Linux from source. Even if you've done this before, this studio will get you the source code and resources that this class depends on, and will give you an experience with the Raspberry Pi platform.

In this studio, you will:

  1. Set up your Raspberry Pi 2
  2. Download the Linux source code and apply a source code patch
  3. Configure the Linux kernel
  4. Compile and install your own custom kernel

Please complete the required exercises below, as well as any optional enrichment exercises that you wish to complete.

As you work through these exercises, please record your answers, and when finished email your results to dferry@email.wustl.edu with the phrase Welcome To Linux Studio in the subject line.

Make sure that the name of each person who worked on these exercises is listed in the first answer, and make sure you number each of your responses so it is easy to match your responses with each exercise.

Required Exercises

  1. As the answer to the first exercise, list the names of the people who worked together on this studio.

  2. To start, you need to set up your Raspberry Pi 2 so that you're able to access a working version of Linux. If you have the reccommended starter kit from Canakit, you can follow these directions. NOTE: This course was developed using the default Raspbian distribution. You're free to use other distributions, but you're on your own if something doesn't work properly. You will need a graphical interface later in the course.

    Once you have successfully booted into Linux, you can use the command uname to get a variety of information about the currently running system, such as the current kernel version or the date on which the currently running kernel was compiled. As the answer to this exercise, copy and paste the output of the command "uname -a".

  3. Now it's time to download the Linux kernel source code and patch it. For a general-purpose project, you would go to kernel.org and download the Linux source from there. However, since we're using the Raspberry Pi we'll be starting with a port designed for our platform. The Raspberry Pi project maintains it's own distribution on GitHub, at https://github.com/raspberrypi. We'll be modifying the kernel building guide that you can find here.

    Make a new directory called linux_source to organize your source code, move into that directory, and then issue the following commands, both of which will take several minutes:

    wget https://github.com/raspberrypi/linux/archive/bc1669c846b629cface0aaa367afb2b9c6226faf.tar.gz

    tar -xzf bc1669c846b629cface0aaa367afb2b9c6226faf.tar.gz

    This has the effect of downloading a specific version of the Raspberry Pi 2 linux distrbution, which is the version that this course was developed with. Newer versions exist, which we won't use for class but you can pursue on your own time if you'd like. Once the files finish unzipping you'll have a new directory, which I would suggest renaming something simpler with the mv command.

    Move into your new directory, and issue the command make kernelversion, which will tell you the linux kernel version you just checked out. As the answer to this exercise, copy the output of the command make kernelversion

  4. Next, you will apply a Linux source code patch. Source code patches are the way that cutting-edge kernel software is shared among kernel hackers. Long before features find their way into major distributions they exist as source code patches, and many contributions that aren't accepted into the Linux mainline exist only as source code patches. In this case, we're going to apply the PREEMPT_RT patch, which allows the kernel to be fully preemptible.

    First, go to kernel.org/pub/linux, which is the public Linux source code repository, and click through "kernel", "projects", and then "rt". You will need to use the answer to the previous exercise in order to find the correct source code patch that matches the version of the Linux kernel that you're using. For example, if you have source code version "4.0.7", then you will move to the directory "4.0" and download "patch-4.0.7-rt11.patch.gz". (Warning: do NOT use the file that starts with "patches", the patch file contains the complete, current version of the patch, while the patches file contains data necessary to update from the last issued version of the rt patch.)

    Download the appropriate patch file ending in gz, and move it into linux_source. If you're working from the command line, you can download a file directly into your current directory with the command wget. To apply the patch, move into the root source code directory (the one with arch, drivers, init, etc.) and issue the following command: (you will need to change version numbers)

    zcat ../patch-4.0.7-rt11.patch.gz | patch -p1

    To explain, the command zcat works like the Linux command cat but for compressed files. If you just issue the command zcat ../patch-4.0.7-rt11.patch.gz by itself you'll see that the source code patch is actually stored in diff format. The vertical bar character pipes the output of zcat into another program called patch, which just applies each individual diff to the appropriate files in the source code directory.

    If no errors are reported, then proceed to the next step. Leave this answer blank.

  5. Now we'll take a second to verify that our system has all of the required prerequisites for building the kernel. Issue the command:

    sudo apt-get install bc ncurses-dev

    Leave this answer blank, and proceed to the next step.

  6. Now it's time to configure the Linux kernel. There are thousands of possible configuration options- many quite interesting and many quite dull. Rather than setting all these options manually, we'll we'll create the default Raspberry Pi 2 configuration and then tweak it. Issue the commands:

    KERNEL=kernel7 (this is used by some build scripts)

    make bcm2709_defconfig

    To reiterate, if you were building a general purpose kernel, you wouldn't use the previous two commands. They're setting a default configuration for Raspberry Pi . Next, we want to set a custom configuration option. Issue the command:

    make menuconfig

    After a moment, you'll get a kernel configuration menu. As you can see, there are a _lot_ of options. For now, we're just going to do two things. First, we will enable the RT_PREEMPT patch we applied previously. Go into "Kernel Features", and then select "Preemption Model". If the patch was applied correctly, you'll have the option "Fully Preemptible Kernel (RT)", which you should select.

    Next you'll add your own unique identifier to the kernel you build. Navigate to "Local version" under "General setup". The local version string you specify here will be appended to the output of the uname command. If you applied the default Raspberry Pi configuration correctly, this should be set to "-v7". Go ahead and add your own unique identifier, though recall that uname already gives you the kernel version number, the day and time the kernel was compiled, as well as other info.Warning: do not include a space in the local version string- this will break the build script when you run sudo make modules_install.

    As the answer to this exercise, find a neat sounding option and use the "H" key to bring up a short description. Provide the option's name, a short summary, and the option's symbol. Once you're done, exit the configurator and be sure to answer "Yes" when asked to save your changes.

  7. Finally, we're able to start compiling the kernel. This will take about an hour and a half, so you probably don't want to do this now (and if another class is coming in soon, you can't). My reccomendation is to let the kernel build for a few minutes in-class to make sure there aren't any obvious compilation errors, then cancel the process with CTRL-C and take it home. To begin, issue the command:

    make -j4 zImage modules dtbs

    That command takes about 90 minutes. Once it's done, issue the following:

    KERNEL=kernel7 (if you've logged out since issuing this command before)

    sudo make modules_install

    sudo cp arch/arm/boot/dts/*.dtb /boot/

    sudo cp arch/arm/boot/dts/overlays/*.dtb* /boot/overlays/

    sudo cp arch/arm/boot/dts/overlays/README /boot/overlays/

    sudo scripts/mkknlimg arch/arm/boot/zImage /boot/$KERNEL.img

    At this point, your new kernel is installed. When you reboot, you'll be running your very own, custom kernel.

    Go ahead and reboot now. If everything went OK, the new system should look and feel the same as before. You can verify that your new kernel is running with the command uname -a. In particular, the build date should be today, the version string should include the local version string you specifed during configuration, and your linux source code version with the characters "rt" should appear, which indicates that the PREEMPT_RT patch has been applied.

    As the answer to this exercise, copy the output of uname -a.

    Congratulations! You've just compiled an operating system from source!

Optional Enrichment Exercises

  1. The instructions above are lengthier and convoluted because we're building a kernel specifically for the Raspberry Pi 2. If you'd like the experience of bulding a general purpose kernel, or if you're curious about building on a different machine, you can do so easily. First, go to kernel.org and download any recent release. Unzip the release (and if you'd like, apply the PREEMPT_RT patch). Then, building a general kernel is just a few commands:

    make menuconfig

    make -jN where N is the number of cores you want to use to compile

    sudo make modules_install install

    That's all there is to it! The install targets will automatically and correctly install the new kernel on the common Linux distributions. In the base source code directory there's a file called README which contains compilation and installation instructions for a wide range of scenarios.

    For a sufficently powerful machine, building a new kernel is pretty painless. Our lab's 16 core, 3.2Ghz Intel machines take about 15 minutes to compile and install.