Luiz on Nothing

Since I really understood for the first time what an operating system kernel was, I had this utopian desire to write a simple (but functional) kernel.

Although I did not write JOS from scratch and had some help from others, I feel that I have finally archived my goal, because I have written most of JOS’ important parts.

But let me explain what JOS is about.

Last year, MIT’s computer science department has made their Operating System Engineering course available through MIT OpenCourseWare (OCW).

The course is just amazing. The goal is to write a small operating system called JOS based on exokernel design, and all the laboratory material is available online.

Besides that, they provide test-cases for your code. The tests make it easy to know if you have done something wrong. Also, at the end of each class you are asked to run a script which will tell you whether you have passed to the next class.

This is the operating system course I have dreamed about since I started studying kernels, and I am very glad because I finished the course last December.

The final version of the system features the following:

- i386 support
- Virtual memory
- Preemptive multitasking
- Simple IPC mechanism
- Kernel debuger and monitor
- Simple FS (supports file creation, writting and removal)
- Basic shell
- Minimalist bootloader

There are two more important features I have implemented but they are not fully functional: 4M pages and SYSENTER/SYSEXIT support.

If I remember correctly I did all the exercises and most of the “mandatory” challenges.

The source code is available online in a GIT repository, you can clone it as follows:

$ git clone git://repo.or.cz/mit-so-course.git

You can also browse the source tree by clicking here.

Now, if you are an MIT student, I have two things to tell you. First, before looking at my code be honest and check with your TA whether it is ok to study solutions from others.

The second thing is that your computer science course is just fantastic, I would not miss a single class if I were you!

By the way, I am about to finish an article about JOS’ virtual memory implementation and will post it soon.

PS: I have another post about JOS in Portuguese, which I have written while I was still working on JOS, you can read it here.

“O Unix foi reescrito em C porque o Fortran era lento e não rodava em todas as máquinas”

Last year my application was selected by Google for their Summer of Code program, and before I tell you how wonderful the program is, I’ll show you one of the things I’ve done.

It’s a GIT binding for Python. So, you’ll have to know a little bit about them to understand the demonstration.

For example, say that you want to get some information about commit
4fbef206daead133085fe33905f5e842d38fb8da from Linus’ GIT tree, you could do:

>>> import pygit
>>>
>>> linux = '/home/lcapitulino/src/kernels/upstream/linux-2.6'
>>> repo = pygit.open(linux)
>>>
>>> id = '4fbef206daead133085fe33905f5e842d38fb8da'
>>> com = repo.lookup_commit(id)
>>>
>>> print com.id()
4fbef206daead133085fe33905f5e842d38fb8da
>>>
>>> print com.message()
nfsd: fix nfsd_vfs_read() splice actor setup

>>>

You can also, compare two commits.

>>> com1 = repo.lookup_commit(repo.head_commit())
>>>
>>> id = '4fbef206daead133085fe33905f5e842d38fb8da'
>>> com2 = repo.lookup_commit(id)
>>>
>>> if com1 > com2:
... print 'com1 is more recent'
...
com1 is more recent
>>>

[ For a complete demonstration click here ].

One of the good things about the binding is that it doesn’t fork()/exec() GIT processes. Instead, it uses a library which makes it fast and easy to deal with errors.

The goal of the project was to write a high-level library for GIT. The reason for this is that GIT’s own library is very low-level. It means that to understand how to use it you have to understand GIT’s internals, but even then the API isn’t stable and it’s hard to link against (since it was written to only link against GIT’s programs).

Currently, the standard way to create software which accesses GIT repositories is to fork()/exec() GIT programs. The main disadvantages of this approach are:

1. It can be a slow operation
2. It may not be easy to handle process execution
3. It’s not easy to deal with errors
4. There’s no flexibility

A well written library can solve all those problems. Besides that, in order to demonstrate the API usage and usefulness, I would write a binding for any high-level language. I chose Python.

The project was just wonderful. I’ve had a great time working on it. I’ve learned a lot about GIT, Python bindings and a little bit more about C and general program design. My mentor is just a great person. He was always patient and taught me everything I know about GIT and its internals.

However, I’m not very proud of the final result. Firstly, I was very excited about the project as a whole and my expectations were quite high. I thought it would be merged upstream or at least would attract the attention of lots of interested people.

But that wasn’t exactly what happened. The main problem is that at the end of the project I had only a Proof-Of-Concept of the thing. Only a small amount of functions to extract information from repositories were written (plus the Python binding).

We did some successful experiments though, which showed us we were on the right track. For example, we made an initial port of one of the GIT graphical browsers, QGit. We’ve removed a big portion of its code (which was dedicated to handling GIT processes) and although some changes were needed, the library fell into place nicely.

But I still feel a bit frustrated for not getting the project up to a “production” level. Some people are still interested in it, I receive about two emails per month asking questions about the project… Which makes me feel a bit worse.

After thinking about it for some time, I concluded that the main problems were:

1. I thought the four hours per day I had would be enough
2. I didn’t expect to spend time as long as I did studying GIT and trying different designs

These are the reasons why I have decided not to participate this year (I still have only four hours per day). Maybe next year I’ll try again.

But it was a great project. I will never forget it.

1. Depois de levantar os requerimentos, escreva vários documentos com dezenas de páginas cada um. Bons exemplos são: diagramas, glossários, requerimentos, relatórios, design, etc…
2. Baseando-se nesses documentos, faça um design detalhado da arquitetura do software
3. Obrigue os programadores a seguirem a arquitetura e todos os outros documentos quando forem implementar o software, não permita modificações nos documentos (muito menos na arquitetura)
4. Não incentive unit testing, não use controle de versão nem bug tracking
5. Tente extrair o máximo de produtividade dos programadores: evite distrações como ficar conversando com colegas de trabalho ou sites de internet não relacionados (filtre tudo o que puder, inclusive email externo e IRC)
6. Testes e integração serão feitos somente na fase final do projeto, quando todos os componentes estiverem praticamente prontos
7. Não permita interferências do cliente, ele não sabe o que quer

Parabéns ao cinema brasileiro, especialmente a Padilha e a todos que fizeram Tropa de Elite: o filme ganhou o prêmio de melhor filme em um dos mais importantes festivais do cinema internacional, o festival de Berlim.

Durante a semana houve dúvida se Tropa de Elite ganharia algum prêmio, pois como a Folha reportou, muitos críticos classificaram o filme como “fascista”.

Infelizmente, desde a sua estréia no Rio de janeiro, Tropa de Elite vem sendo mal interpretado. Pior, muitos no Brasil se identificaram com o tom do personagem Capitão Nascimento e frases como “Chamem o Capitão Nascimento” ou “Chamem o BOPE” se tornaram um tanto comuns.

Padilha, no entanto, explicou na sabatina (somente para assinantes) promovida pela Folha que o principal argumento do filme é retratar aquele tipo de policial. Mostrar como pensa, seus argumentos, como ele pode “surgir”, etc. Isso sem interferir com prós ou contras.

Excelente trabalho! Que venham mais prêmios!