Engineering.wustl.edu

CSE 522S: Advanced Operating Systems Class Page

Spring 2016

Instructor David Ferry, Jolley Hall 425, Homepage
Course Web Site http://cse.wustl.edu/~ferryd/courses/cse522/
Course meeting times Tuesday & Thursday 2:30 PM - 4:00 PM, Lopata Hall 201 / Urbauer 115
Midterm exam
2:30 on March 10th, Lopata Hall 201
Final exam
May 11, 2016 3:30 - 5:30, regular classroom
Office hours by appointment due to ongoing construction

Contents
  1. Course Description
  2. Prerequisites
  3. Discussions/Studios
  4. Labs
  5. Course Project
  6. Class FAQ
  7. Textbooks and Other Resources
  8. Grading
  9. Academic Honesty
  10. Accessibility

Course Description

What do you do when you discover that your operating system kernel doesn't do what you need it to do? If you work in special-purpose domains, such as real-time systems or high performance computing, you may find yourself wanting to optimize bottlenecks or provide more deterministic timing guarantees. Or, if you work in traditional OS domains such as scheduling or networking, you may want to include new functionality in the kernel. Even if you never plan on writing a line of kernel code outside of school, the kernel lies at the heart of everything we do as computer scientists, so it's useful to have a deep understanding of how userspace programs interact with kernel mechanisms.

The purpose of this course is for each student to:

The class taught in the Spring 2016 semester should be considered a trial run of the course. Hiccups are to be expected as the class finds new and exciting ways to break course exercises and the instructor tries to fine-tune the content and scope of each class period.


Prerequisites


Discussions and Studios

This course is conducted in a studio format, both to emphasize the hands-on nature of kernel hacking and to engage students in practical learning exercises. The focus of lecture components is to facilitate in-class discussion of the kernel design process, while the focus of studio exercises is to gain experience kernel hacking, and to familiarize students with the tools and techniques of kernel development. Studios will be completed as small group exercises, and are submitted for credit. All studios assigned before the midterm are due the Friday prior to the midterm, and all studios assigned after the midterm are due the Friday of the last day of classes.

Most class periods are accompanied by suggested readings. The Linux kernel is amazing in that many of the discussions and disagreements of the original developers have been saved verbatim in repositories such as the the Linux Kernel Mailing List (LKML) or documented by firsthand witnesses at sites such as lwn.net. It is the instructor's goal that the course textbook can be used as a technical reference for kernel mechanisms, and that supplemental readings can be used to understand the particular design choices made in the course of kernel development.

A tentative course schedule is below. Note that this schedule may change over the course of the semester. When changes occur, students will be notified in class and given enough advance notice so that readings and other preparation may be accommodated.

Date Topic Readings Studio
1 Tues, Jan 19th Course Introduction LKD chapters 1 & 2
Code Pointers
Setting up the Raspberry Pi 2 and building the Linux kernel
2 Thurs, Jan 21st How and when does the kernel run? LKD chapter 5
Userspace / Kernelspace API Split
Code Pointers
Creating a syscall
3 Tues, Jan 26th No assigned studio
4 Thurs, Jan 28th Time sources and timing LKD chapter 11
High Resolution Timers Subsystem
Description of the Timer Wheel System
Introduction of hrtimer Patch (Formerly ktimers)
Code Pointers
Userspace benchmarking
5 Tues, Feb 2nd Kernel tracing Ftrace design document
Code Pointers
Tracing with ftrace and Kernelshark
6 Thurs, Feb 4th Structure and infrastructure of the Linux kernel LKD pp.85-96
LKD Ch. 6 as reference
LKD pp.338-348
Writing a kernel module
7 Tues, Feb 9th Interrupts and interrupt handlers LKD Ch. 7
LKD Ch. 8
Lab 1 Assigned
Due Feb. 19th at 11:59PM
8 Thurs, Feb 11th Processes LKD Ch. 3
LKD Ch. 15
Code Pointers
Process Family Tree
9 Tues, Feb 16th Traditional process scheduling LKD Ch. 4 Completely Fair Scheduling
10 Thurs, Feb 18th Real-time process scheduling Deadline Scheduling vs POSIX Real-time Scheduling
Software Interrupts and Real-time
Real-time Scheduling
Lab 1 due tomorrow, 11:59PM
11 Tues, Feb 23rd Kernel synchronization LKD Ch. 9
LKD Ch. 10
What is RCU, fundamentally?
Kernel Reference on RCU
Synchronizing Threads in the Kernel
12 Thurs, Feb 25th Read-copy-update (RCU) synchronization man 7 futex
man 2 futex
Build Your Own Locks
13 Tues, Mar 1st Linux details: The boot process No assigned studio
14 Thurs, Mar 3rd Linux details: Program execution and the dynamic linker No assigned studio

All assigned studios due tomorrow, Mar 4th

15 Tues, Mar 8th Midterm review No assigned studio
16 Thurs, Mar 10th Midterm Exam
17 Tues, Mar 15th Spring Break - No class
18 Thurs, Mar 17th Spring Break - No class
19 Tues, Mar 22nd Inter-process communication: signals Linux Signals
20 Thurs, Mar 24th In-class project time
21 Tues, Mar 29th Inter-process communication: pipes Linux Pipes and FIFOs
22 Thurs, Mar 31st In-class project time
23 Tues, Apr 5th Inter-process communication: sockets Linux Sockets
24 Thurs, Apr 7th In-class project time
25 Tues, Apr 12th Inter-process communication: shared memory Linux Shared Memory
26 Thurs, Apr 14th In-class project time
27 Tues, Apr 19th Linux details: Hardware device drivers In-class project time
28 Thurs, Apr 21st Final Exam Review No assigned studio
29 Tues, Apr 26th Project Presentations No assigned Studio
30 Thurs, Apr 28th Project Presentations No assigned studio

All assigned studios due tomorrow, Apr 29th

31 Wed, May 11th Final Exam

Labs

There will be two lab assignments for this course. The purpose of these labs is to apply course concepts and to evaluate competing design alternatives for kernel mechanisms. As such, each lab will require a written report detailing your findings in addition to a code submission.

Each lab will be completed in a team of two or three students, and teams may be different for each lab. Students from different teams may discuss the lab assignments only during course meeting times. Students from the same team are of course encouraged to discuss and work on lab assignments at any time.

Labs submitted on time (as determined by email time stamp) will be given full credit. Labs submitted up to 24 hours late will be given a ten percent penalty. Labs submitted between 24 and 48 hours late will be given a twenty percent penalty. Labs submitted after 48 hours late will not be given credit, except in the case of extenuating circumstances pre-approved by the instructor.

The following labs have been so far assigned:


Course Project

The final project for this course is for a team of two or three students to propose and complete their own significant modification to the Linux kernel. This might be an entirely new capability or it might be an alteration to an existing functionality. The students will evaluate the effectiveness of their modification through the creation of user program test cases that successfully demonstrate the new behavior. Students are encouraged to look to their own interests and/or research projects for inspiration.

In either case, students will justify their proposal in a written document, in order to ensure that the scope of the project is achieveable. The proposal document will:

  1. Propose a modification
  2. Clearly explain the purpose of the modification
  3. Identify kernel files that will require modification
  4. Identify kernel data structures that will require modification
  5. Identify kernel concepts and control paths that will require modification
  6. Propose a set of user program test cases that demonstrate the modification
  7. Include a five-week schedule for the project

The project will be evaluated via a 20 minute presentation at the end of the semester, as well as a code submission to the instructor. This presentation should explain the modification to the class; describe the final alterations to kernel files, data structures, concepts that the students enacted; present timing graphs or other data to verify the behavior of the modified system; and include a short demo (if possible).


Class FAQ

Past students of this course have contributed tips, tricks, and solutions to problems that they have encountered. Let your instructor know if you have something you'd like to add here!

Class FAQ


Textbook and Class Resources

Students should consider the in-class studios and assigned readings to be a guided tour of the Linux kernel. Like any good tour group, we want to see the sights and learn some neat highlights about who built it and what they were trying to accomplish. But, the studios and assigned readings will not make you an expert on the Linux kernel. When it comes to learning the details of a code base as deep and complex as this, there is no substitute for reading source code. Lectures and studios will come with pointers to relevant source code files, and it is expected that students will spend time absorbing the content there as well. Therefore, assigned readings are kept moderately short with the expectation that you will spend some independent time in the source code.

To be clear, you are not expected to understand or memorize every line of the Linux source. Exams will not have questions derived from the Linux source (though basic kernel concepts and pseudo-code are fair game). However, the kernel is a huge set of deeply interdependent source code files. The only way you will ever understand it is by looking through it many times.

There is one required course textbook: Linux Kernel Development by Robert Love, 2010. This is an excellent, compact, and inexpensive text that gives the reader a basic understanding of kernel design, written with the depth of a kernel veteran.

There are a variety of decent references. None of these will give you the level of competency that comes from looking at code itself, but they are very useful for starting points and clarifying problems. However, the Linux kernel is rapidly updated, so while the general information in these books is correct, details such as source code, names of files, and where files exist is likely to can and does change.

Students will require a Raspberry Pi 2 in order to complete daily studios and lab assignments. The course has been designed with the intention that students bring these devices to class, plug them in, and do some hacking there. Monitors, keyboards, and mice will all be provided for this purpose- you will need to provide the Raspberry Pi 2, a power cord, and an HDMI cable.

If you so desire, you may set up your Raspberry Pi 2 in your home or office and configure it for remote access. However, the instructor cannot support you in this, and we are modifying the kernel so you'll be lost in the dark if you reboot and can't ssh back into your machine. Also, be forewarned that some exercises will require an X11 window, so simple ssh access will not be enough to do everything remotely.


Grading

There are four activities for which you will receive credit in this course: studios, labs, exams, and a course project. Studios are daily guided assignments primarily designed to familiarize students with course concepts, development tools, and the kernel source code (i.e. knowledge and comprehension tasks). Two lab assignments will ask students to apply general course concepts and analyze kernel design alternatives. The course project requires that students propose and complete a novel kernel modification. A midterm and final exam will evaluate your technical understanding of course concepts.

Your grade will be determined as follows:

Activity Grade Percentage
Studios 25%
Lab 1 10%
Lab 2 15%
Midterm 10%
Final 15%
Project 25%

Academic Honesty

All work assigned in this course, other than exams, is expected to be completed collaboratively. Student teams may change from assignment to assignment, but the sharing of written work or significant portions of code between teams is strictly prohibited.

Please familiarize yourself with the Washington University policy on academic integrity. Any cheating will result in a score of zero for the assignment. The course staff will make final determinations on what constitutes cheating. If in doubt, please ask first.


Accessibility

Students with disabilities or suspected disabilities are strongly encouraged to both bring any additional considerations to the attention of the instructor and make full use of the University's Disability Resource Center (http://disability.wustl.edu).