Take a look at our ideas page:

http://wiki.qemu.org/Google_Summer_of_Code_2010

We have all kinds of projects there: a library, application protocol, hardware emulation, kernel, and a lot more.

Please, note that QEMU is not only an emulator. Some people I know still think of QEMU as being the old emulator they used to use years ago. But today QEMU is also a virtualizer, it supports Xen and KVM, being the upstream project of KVM user-space.

My try:

(define mylengh
  (lambda (ilist)
    (letrec ((mylengh-rec
               (lambda (rlist nr)
                 (if (eqv? rlist '()) nr
                   (mylengh-rec (cdr rlist) (+ nr 1))))))
             (mylengh-rec ilist 0))))

Solution:

(define (mylength a)
  (if (null? a) 0
      (+ 1 (mylength (cdr a)))))

Well… It is a start. :)

So, in this specific case gcc has searched its VMA tree 1570 in five seconds while (probably) compiling some C file. Is that a big number? I am not sure but X, for example, is calling find_vma() more than 2600 times in a second if I switch fast between work spaces in Window Maker. I did not expect this.

Git is the winner so far, more than 8000 searches in one second while resolving deltas in a clone operation.

I also wanted to know what are the most called system calls that may search (or change) the VMAs’ tree, and here goes some more numbers:

This snapshot has been taken while compiling the Linux kernel and while the numbers are of no surprise, there are two interesting functions in that table, brk() and vfork(). I thought that their usage would be very rare these days: users of brk() are now using mmap() and I was wondering whether it makes sense to use vfork() in a modern kernel like Linux, with very fast process creation time and copy on write support.

But a quick look at the code shows that sys_vfork() passes the CLONE_VM flag to do_fork(), this in turn will avoid the creation of a new address space for the child. In other words, they will share the same mm_struct and that makes vfork() faster than regular fork() (threads also share the same mm_struct).

Very cool stuff, I am being good at choosing projects.

PS. I already bugged people on the git lists with this, but since I’m
totally shameless and can’t help but hope that some random kernel person
also does tcl/tk or just wants to help improve my kids-time-tracker, I can
just point to

http://torvalds-family.blogspot.com/2008/10/tracking-time-kids-spend-online.html

and hope that somebody would like to make that GUI better and/or feel like
they really want to do a “management interface” too ;)

I got a bit confused because the keywords that blinked in my brain were “git”, “tcl/tk”, “kernel”, “kids” and “family”.

Then I discovered that Linus has a blog now and the coolest thing about it is that it is not just another tech- or kernel- blog, he is going to talk about his family too.

That is cool because I am always looking for ways to improve my son’s education and there are not lots of kernel hackers’ blogs about that out there. :)

This post will briefly explain what copy-on-write fork() is, then we will discuss JOS’s very particular copy-on-write fork() implementation which is mostly done in user-space.

Hopefully this post will give you a good understanding of the system calls presented in the first part of this series.

To be honest, I am not sure whether copy-on-write fork() is the best example for beginners, but it is very helpful to demonstrate how exokernel implementations move functionality normally implemented in kernel-space to user-space.

I recommend you to read the first post of this series if you have not done so yet.

Copy-on-write fork()

The fork() system call creates a new process by duplicating the calling process. That means that the new process created by fork(), known as the child, is a complete copy of the process which called fork(), known as the parent (actually, not everything is copied but we can ignore the details for now).

One of the parent’s resources which has to be duplicated is its memory address space, which include all the pages allocated for the parent. There are two ways to do this.

The simpler way is easy to understand and implement, it is something like the following:

for each mapped page in the parent do
	1. allocate a new page
	2. insert it into the address space of the child, at the same virtual address
           of the parent's page
	3. copy the parent's page contents into the child's allocated page

Again, that is not the full story, I have omitted some details but the algorithm does show the problems of this solution which is basically an unnecessary overhead.

For example, if the child process uses only a small subset of its address space, all the work done on the unused pages would have been completely wasted.

Think about the shell now, its main duty in life is to fork() and exec() new programs. Exec() destroys the address space of the process calling it and builds a new one. So, if you are using the dumb algorithm showed above, for each command you type into the shell prompt, fork() will duplicate the address of the shell just to allow exec() to destroy it.

Of course this algorithm is awful and today’s operating systems use something better called copy-on-write.

Copy-on-write fork() works like this: instead of duplicating the address space of the parent into the child, the kernel marks all the parent’s pages as read-only and inserts them into child address space. By doing that both processes will share the parent’s pages. When any of the two writes into one of the shared pages, the MMU hardware issues a page fault. The page fault handler executes and learns that a process is trying to write into a copy-on-write page. At this moment the page fault handler does the following:

1. Allocates a new page
2. Makes the new page read-write
3. Copies the contents of the copy-on-write page into the allocated one
4. Inserts the new page into the address space of the process doing the write, at the
   same virtual address of the copy-on-write page

After the fourth step, the page fault handler exits and the processor continues to execute in the instruction which caused the page fault. In this way the change is transparent for the running process.

Copy-on-write fork() is a little bit more complex to implement and it depends on the hardware to work, but it does make fork() low overhead as it does not have to duplicate the entire address space of the calling process.

Let’s consider the shell case again. In a system with copy-on-write support, the fork()/exec() sequence is much faster because no page is duplicated between the fork() and exec() calls.

Note that copy-on-write is potentially useful to any process which forks new processes, because pages are duplicated only when needed.

JOS copy-on-write fork()

In JOS fork() is not a system call. It is a library function which uses various system calls to perform its task.

The code of fork() can be found here, I recommend you to keep it open while reading this section. Also note that for now on, when I say fork(), I will be referring to JOS’s fork().

Besides the memory management system calls presented in the first part of this series, fork() also uses the following system calls which deals with environment handling:

envid_t sys_exofork(void);
int sys_env_set_status(envid_t envid, int status);
int sys_env_set_pgfault_upcall(envid_t envid, void *upcall);

I will describe now how fork() works and will introduce the functions above when necessary. It is important to note that my goal here is just to give you a general overview.

In JOS, page faults caused by environments are handled in user-space. When a page fault happens, the kernel checks whether the environment which issued the page fault has registered a page fault handler. If it did, the kernel returns to user-space and executes the registered handler. If the environment did not register a page fault handler the kernel will kill it when a page fault happens.

The first thing fork() does on line 122 is to setup the calling process page fault handler. Both the parent and the child have to setup a page fault handler.

The next step on line 124 is to create a new environment by calling sys_exofork(). That function allocates a new environment from the poll maintained by the kernel, initializes it and copies the current context of the parent (that is, the parent’s registers’ contents) into it. It is important to realize that sys_exofork() does only the very first part of the forking process.

After the new environment has been created, fork() executes its main code. It goes all over the parent’s page tables and each writable page is marked COW and read-only before being inserted into the address space of the child. Note that we have to remap the page into the parent as well, since the page permissions have been changed.

When the page setup is done, the parent does the final setup. Firstly it configures the child’s page fault handler and then the parent marks the child runnable with the call to sys_env_set_status(). That is, in exokernels it is up to the application to say when a newly created environment is runnable, not up to the kernel.

As long as the child is marked runnable, it can execute at anytime. If there are just a few environments running, it will probably run within the next few clock interruptions.

When the child is put to run by the kernel, it starts executing as if it was returning from sys_exofork(). The code in fork() is prepared to deal with this situation and on line 127 it checks whether it is the child which is executing and if it is, fork() finishes the setup and returns 0.

The process is now forked, and when any of the two tries to write to a page a page fault is issued by the hardware.

When the page fault happens the kernel will return to user-space and branch out to the page fault handler registered by fork(), which is pgfault() function on line 15. This function duplicates the page exactly as was explained in the first section of this post.

In my last post I mentioned I was writing about JOS’ virtual memory implementation. At first, I was going to write only one post but I have written so much that I had to split the post into two parts.

The first (this one), will talk about the exokernel design and will have a first look at JOS’ memory management interface with user-space.

The second post will explain how the system calls presented in the first post work. But, instead of a boring explanation, I will show how the system calls fit together to implement a real JOS’ feature: fork() with copy-on-write support. This feature is impressive because it is done mostly in user-space.

For both posts some background in memory management and operating systems is needed. For those who have learned it but forgot the details, the wikipedia article on virtual memory seems to be sufficient. If you have never studied this subject before, I recommend you to read the memory management chapter from some operating systems book (I have learned from Tanenbaum’s).

Something important to bear in mind is that I am just a beginner on this subject. Everything I know I have learned while working on JOS, so it is likely that you will find mistakes in this text. If you do, please, let me know by sending me an email.

Quick introduction to exokernel design

The important idea behind the exokernel design is that its interface with user-space is as simple as possible. In fact, what an exokernel does is to securely export all hardware resources to user-space and give applications the freedom to manage the hardware resources for themselves.

Most of what a Unix user would recognize as being an operating system task is implemented as an application library in an exokernel.

For example, in Unix kernels the basic system call to create a new process is fork(). The application calls fork() and a new process is created. Everything is done in kernel-space and, roughly speaking, the application has no control over the process.

In an exokernel design, though, fork() is implemented in user-space. Here the application has full control over the process and could even define its own fork() with different semantics. Actually, fork() is not even special because it is just an ordinary library function.

Additionally, most kernel services that are usually implemented in kernel-space (such as File-systems and Networking) can be implemented by user-space applications in an exokernel design.

The main advantages of this design are:

1. The kernel does not limit the freedom of the application to design its own abstractions for physical resources
2. In some cases, due to the high costs of heavy abstractions in kernel-space, you can have performance benefits from implementing abstractions in user-space
3. Exokernels are small

Of course, there are disadvantages, one of them being that exokernel applications may not be portable among different exokernel implementations.

For more information regarding the exokernel design, I recommend to read the original exokernel’s paper.

A First look at JOS’ memory management interface

The following system calls are available to user-space applications:

int sys_page_alloc(envid_t envid, void *va, int perm);
int sys_page_map(envid_t srcenvid, void *srcva, envid_t dstenvid, void *dstva, int perm);
int sys_page_unmap(envid_t envid, void *va);
int sys_env_set_pgfault_upcall(envid_t envid, void *func);

Before going into the details, it is important to understand what an environment is. In exokernels, an environment is the same entity as a process in Unix. Environments also have identification numbers, referred to as ‘envid’.

I will now describe in detail what each of the system calls above do. You may have the feeling that things are not very clear. Do not worry, things will get clearer soon:

sys_page_alloc(): allocates a page of physical memory from the free list maintained by the kernel and inserts that page into the address space of the environment at address ‘va’. The page permissions is set to perm.

sys_page_map(): maps the page of memory at ’srcva’ of environment ’srcenvid’ into address ‘dstva’ of environment ‘destva’. The page permissions are set to ‘perm’.

sys_page_unmap(): unmap the page of memory at ‘va’ in the address space of ‘envid’. If this is the lasting mapping of the page, it’s inserted back to the free list.

sys_env_set_pgfault_upcall(): set the environment ‘envid’ page fault handler. When a page fault happens, the kernel branches to ‘func’.

Basically, to allocate memory in JOS, an application has to call sys_page_alloc(). It means that it has to know where the physical page should be mapped in its address space.

Of course, for widely used features (like memory allocation), the application could (and should) implement a memory allocation function, say malloc(). Such a function would define a specific area in the address space of the application for memory allocations, and manage it.

Indeed, JOS has malloc() implemented as a library function and it works as just explained. But it is important to note that it is more like a convention, because the application is still free to implement its own memory allocation function with whatever features it needs.

The next post will explain how the system calls presented above work in a real life JOS feature: fork() with copy-on-write support.

Although I did not write JOS from scratch and had some help from others, I feel that I have finally archived my goal, because I have written most of JOS’ important parts.

But let me explain what JOS is about.

Last year, MIT’s computer science department has made their Operating System Engineering course available through MIT OpenCourseWare (OCW).

The course is just amazing. The goal is to write a small operating system called JOS based on exokernel design, and all the laboratory material is available online.

Besides that, they provide test-cases for your code. The tests make it easy to know if you have done something wrong. Also, at the end of each class you are asked to run a script which will tell you whether you have passed to the next class.

This is the operating system course I have dreamed about since I started studying kernels, and I am very glad because I finished the course last December.

The final version of the system features the following:

- i386 support
- Virtual memory
- Preemptive multitasking
- Simple IPC mechanism
- Kernel debuger and monitor
- Simple FS (supports file creation, writting and removal)
- Basic shell
- Minimalist bootloader

There are two more important features I have implemented but they are not fully functional: 4M pages and SYSENTER/SYSEXIT support.

If I remember correctly I did all the exercises and most of the “mandatory” challenges.

The source code is available online in a GIT repository, you can clone it as follows:

$ git clone git://repo.or.cz/mit-so-course.git

You can also browse the source tree by clicking here.

Now, if you are an MIT student, I have two things to tell you. First, before looking at my code be honest and check with your TA whether it is ok to study solutions from others.

The second thing is that your computer science course is just fantastic, I would not miss a single class if I were you!

By the way, I am about to finish an article about JOS’ virtual memory implementation and will post it soon.

PS: I have another post about JOS in Portuguese, which I have written while I was still working on JOS, you can read it here.

It’s a GIT binding for Python. So, you’ll have to know a little bit about them to understand the demonstration.

For example, say that you want to get some information about commit
4fbef206daead133085fe33905f5e842d38fb8da from Linus’ GIT tree, you could do:

>>> import pygit
>>>
>>> linux = '/home/lcapitulino/src/kernels/upstream/linux-2.6'
>>> repo = pygit.open(linux)
>>>
>>> id = '4fbef206daead133085fe33905f5e842d38fb8da'
>>> com = repo.lookup_commit(id)
>>>
>>> print com.id()
4fbef206daead133085fe33905f5e842d38fb8da
>>>
>>> print com.message()
nfsd: fix nfsd_vfs_read() splice actor setup

>>>

You can also, compare two commits.

>>> com1 = repo.lookup_commit(repo.head_commit())
>>>
>>> id = '4fbef206daead133085fe33905f5e842d38fb8da'
>>> com2 = repo.lookup_commit(id)
>>>
>>> if com1 > com2:
... print 'com1 is more recent'
...
com1 is more recent
>>>

[ For a complete demonstration click here ].

One of the good things about the binding is that it doesn’t fork()/exec() GIT processes. Instead, it uses a library which makes it fast and easy to deal with errors.

The goal of the project was to write a high-level library for GIT. The reason for this is that GIT’s own library is very low-level. It means that to understand how to use it you have to understand GIT’s internals, but even then the API isn’t stable and it’s hard to link against (since it was written to only link against GIT’s programs).

Currently, the standard way to create software which accesses GIT repositories is to fork()/exec() GIT programs. The main disadvantages of this approach are:

1. It can be a slow operation
2. It may not be easy to handle process execution
3. It’s not easy to deal with errors
4. There’s no flexibility

A well written library can solve all those problems. Besides that, in order to demonstrate the API usage and usefulness, I would write a binding for any high-level language. I chose Python.

The project was just wonderful. I’ve had a great time working on it. I’ve learned a lot about GIT, Python bindings and a little bit more about C and general program design. My mentor is just a great person. He was always patient and taught me everything I know about GIT and its internals.

However, I’m not very proud of the final result. Firstly, I was very excited about the project as a whole and my expectations were quite high. I thought it would be merged upstream or at least would attract the attention of lots of interested people.

But that wasn’t exactly what happened. The main problem is that at the end of the project I had only a Proof-Of-Concept of the thing. Only a small amount of functions to extract information from repositories were written (plus the Python binding).

We did some successful experiments though, which showed us we were on the right track. For example, we made an initial port of one of the GIT graphical browsers, QGit. We’ve removed a big portion of its code (which was dedicated to handling GIT processes) and although some changes were needed, the library fell into place nicely.

But I still feel a bit frustrated for not getting the project up to a “production” level. Some people are still interested in it, I receive about two emails per month asking questions about the project… Which makes me feel a bit worse.

After thinking about it for some time, I concluded that the main problems were:

1. I thought the four hours per day I had would be enough
2. I didn’t expect to spend time as long as I did studying GIT and trying different designs

These are the reasons why I have decided not to participate this year (I still have only four hours per day). Maybe next year I’ll try again.

But it was a great project. I will never forget it.

There’re examples of: mmap(), longjump(), fifos, signals, sockets and a lot more. On the data structures side: lists, queues , hash tables (here and here) and others.

But it’s important to say that most of these examples were taken from books (mainly from this and this one), although I might have modified some of them.

There’re also some interesting Linux kernel code examples. For example, when I was working on the USB-Serial layer port to the Serial Core API, I’ve written two drivers for a fake serial device. One driver uses the Serial layer directly (that’s what the USB-Serial layer does) and the other one uses the Serial Core API. Cool stuff, and was fun coding it.

The problem is that this:

http://repo.or.cz/w/lcapit-junk-code.git

Is not a valid git repository to be cloned, it’s just my repository’s gitweb page. Yeah, I know, the ‘.git’ at the end makes you think it’s a git repository…

But it can get even more confusing! Besides the gitweb’s URL, each project hosted by repo.or.cz also has three additional URLs, take a look:

git://repo.or.cz/lcapit-junk-code.git

http://repo.or.cz/r/lcapit-junk-code.git

git+ssh://repo.or.cz/srv/git/lcapit-junk-code.git

The first is the one you should use to clone my repository:

$ git clone git://repo.or.cz/lcapit-junk-code.git

So that git-clone can clone by using git’s protocol.

The HTTP one can also be used to clone the repository, but git-clone will clone through (guess what) HTTP and that’s slow. You don’t want to use it, trust me. You should only use it if you’re behind a firewall or something like that.

The last one is used by the repository’s owner to push into the repository, just ignore it.