Fork me on GitHub

What

What this is about

Inspired by the osdev-forums and the likes of TempleOS, as a challenge and to learn more about modern computer architectures, I’ll give writing a tiny custom operating system from scratch a try. I’ll be documenting the process in this multi-part series.

The objective here is not write a full operating system with task scheduling, multi-processing and advanced hardware support or even caching and page faults, but something simple and minimalistic. Getting it to boot, print text on screen, get input from the keyboard and read files from disk is all I’ll be trying to achieve here.

A Computer's boot process

In a computer the central processing unit (CPU) runs commands, the machine instructions. These commands are run, one after the other, in sequence. During boot of computer a series of events occurs. First the CPU loads instructions from a hard-coded address in the ROM, then from the Flash-Chip containing the BIOS, then the Bootloader, which is stored on the Hard Disk’s boot partition, the Kernel and finally the actual Operating System.

graph TD; A(Power Button was pressed) --> B(CPU jumps to predefined memory location of BIOS in ROM) B --> C(CPU loads BIOS and runs it) C --> D(BIOS performs POST, the Power-On Self Tests) D --> E(it then loads the Hard Disk Partition Table) E --> F(and runs the Bootloader from Hard Disk Partition) F --> G(which contains instructions to load the Operating System Kenel) G --> H(that then runs the Operating System);

For a Custom Operating System we step in at the bootloader stage. We write our own Bootloader and place it on the primary boot disk. This bootloader will have to be compiled with a custom cross-compile-chain, without much of the modern dependencies, for it to be able to be compatible to run barebones on our x86 CPU. We can then add the functionality we need to our bootloader to extend it far enough for our own Custom OS.

When

Scope of this multi-part series

Writing an even moderately useful custom Operating System is a large task. After setting up the bootloader we will be greated with text upon boot and the possibility to print custom messages from our code. From there we will need to achieve keyboard input, and perhaps mouse input. Then we can load a hard disk driver and read the directory tree from the hard disk. We can add a simple text editor to read and write files. At that point things begin to become somewhat useful - we can now read and write files and do simple note-taking. From there, if we can add a compiler directly to our kernel, we can build and run code from the hard disk. By enabling VESA-Video-Output we gain the ability to draw on the screen. At that stage the rest is a huge programming exercise of adding useful software, perhaps a window manager and for instance network drivers.

Some experience can be taken from working with microcontrollers that can be programmed to receive PS/2-Keyboard-Input or load entire TCP/IP-Network-Stacks and can of course drive displays. A modern PC is in essense no more than a very complex micro-controller with lots of hardware peripherals.

Features of modern Operating Systems compared with a Custom OS

Modern Operating Systems provide vast amounts of features which can’t all be supported in a Custom OS developed by a single person or a group of hobbyists. Basic funcitonality can be achieved, but it’s important to manage expectations.

Feature Modern OS Custom OS
Keyboard full USB HID Device support Use BIOS PS/2 emulation
Graphics Graphics driver, mostly proprietary, with multi-monitor support, high resolutions and framerates with 3D accelleration Graphics via Bios VESA modes, utilizes only 64k VRAM, 320x200 at 256 colors or 640x480 at 16 colors
Paging Pagefile on hard disk to expand Memory and automatic caching Probably no paging support
Disk IO fast vendor specific device drivers for NVMes, SSDs, CD-Rom Drives, USB-Sticks, Floppy's and more only rudimentary Floppy Disk IO
Network driver for multiple Network cards at full speeds none or perhaps basic support for common features in most Intel NIC's
Software Installers, Package Managers, thousands of applications. A few applications to provide basic functionality such as text editing, either hardcoded into the Kernel or run at runtime by C-Script.

Milestones

As such the milestones are as follows:

  1. Create a Custom-Cross-Compile Chain - We can then produce machine code that can be run directly on the CPU
  2. Compile our own Bootloader - We can place it on the boot disk and print simple text on screen.
  3. Read Keyboard-Input by using low-level inb/outb to read/write to the x86-compatible 8259A Programmable Interrupt Controller (PIC) - We can now build a simple dialogue system that prints a text menu and accepts user interaction.
  4. Add a file system driver for FAT32 or similar - this allows us to issue a "dirlist" command that will list the files on the hard disk partition.
  5. Write a simple text editor with, for instance, "textcreate [file]", "textappend [file] [line of text]", "textview [file]" - we can now edit basic text
  6. Add a Compiler, such as TinyCC, to the Kernel - with it we can now run C-Code from source code text files on the CPU like scripts (Just-In-Time Compilation)
  7. Enable VESA-Video Output - now we can draw shapes on the screen
  8. Write first useful applications, e.g. a graphical Window Manager - this allows us to actually use the operating system for basic tasks

The process will, of course, require a lot of work even for a tiny custom OS. I’m doing this purely for fun and educational purposes. The goal is not to compete with existing operating systems that take hundreds of thousands of hours in development time.

Why

Why are we doing this?

When most people think of an Operating System they think of Windows, MacOS, Linux and perhaps OpenBSD. But there is an entire community of developers [1], some hobbyists, some from research, that are writing their own custom OS as a learning experience, for fun or to experiment with new concepts. In fact one custom Operating System that was designed for research is Minix. Minix evolved when AT&T restricted licensing for Unix and universities began developing Minix as an alternative for their lectures. As a side note Linus Torvalds, the initiator of Linux, wrote his first versions of Linux on Minix - and so Linux started as the Custom OS of a single developer.

The developer of one such CustomOS is the late Terry A. Davis. He spent over a decade developing TempleOS and became somewhat of an internet sensation due to his project and his oftentimes awkward social media posts. His work, though perhaps not useful to most, is fascinating. It completly rethinks modern operating systems and shows that a project of this scale is possible even by a single person.

Reading and understanding the ideas and concepts behind TempleOS has lead me to internet communities of hobbyists specialized in Custom Operating Systems and, being a devolper myself, has inspired me to give developing a small Custom OS a try myself.

Background

Other Custom-OSes

Looking at Terry Davis’ Custom Operating System, which was the main inspiration for me to try writing my own OS, you can find the binary blob of the Kernel he wrote [2].

When booting TempleOS on real hardware this Kernel is eventually loaded to RAM and executed by a bootloader. This is best seen by looking at the syslinux, a booloader for Linux, configuration files of TinkerOS, a TempleOS fork, that was made to make TempleOS more accessible [3]. The syslinux configuration of TinkerOS loads the memdisk Kernel, that triggers loading the TempleOS ISO-CD-Image, which in turn contains the TempleOS Bootloader, that then boots TempleOS as intended by its creator.

The TempleOS Bootloader contains a C-Compiler that supports Just-In-Time execution - that is running C-Code like a script directly, without creating binary executables first. The modified C-Compiler, dubbed HolyC, is then able to run the TempleOS Kernel via KMain.HC [4] which then loads Adam, the TempleOS Window-Manager.

How

Setting up a cross-compile chain for i686-elf

As a first step to get our CustomOS to even boot, we will need to set up a cross-compile chain to build our own bootloader.

Now to build this code they recommend setting up a GCC Cross-Compiler for i686-elf binaries which is what I’ve also done.

In order to build the GCC-Cross-Compiler I’ve built a small bash-script based on various sources and adapted to requirements of my machine, an Ubuntu 20.04.2 LTS.

gccbuild.sh

#!/bin/bash
BASEDIR=$(cd `dirname $0` && pwd)
export PREFIX="${BASEDIR}/out/path/"
export TARGET=i686-elf
export PATH="$PREFIX/bin:$PATH"

echo "[ ] Install dependencies"
sudo apt -y install libgmp3-dev libmpfr-dev libisl-dev  libmpc-dev texinfo -y
sudo apt -y install nasm qemu-system-i386 xorriso

echo "[ ] Create directories"
rm -rf ${BASEDIR}/out/
mkdir -p ${PREFIX}
mkdir -p ${BASEDIR}/out/src/
cd ${BASEDIR}/out/src/

echo "[ ] Download code archives"
cd ${BASEDIR}
wget --no-clobber ftp://ftp.gnu.org/gnu/binutils/binutils-2.26.tar.gz
wget --no-clobber ftp://ftp.gnu.org/gnu/gcc/gcc-6.1.0/gcc-6.1.0.tar.gz

echo "[ ] Build binutils"
mkdir ${BASEDIR}/out/src/build-binutils/
cd ${BASEDIR}/out/src/
tar -xzf ${BASEDIR}/binutils-2.26.tar.gz
cd ${BASEDIR}/out/src/build-binutils/
../binutils-2.26/configure --target=$TARGET --prefix="$PREFIX" --disable-nls --disable-werror
make
make install

echo "[ ] Build gcc"
mkdir ${BASEDIR}/out/src/build-gcc/
cd ${BASEDIR}/out/src/
tar -xzf ${BASEDIR}/gcc-6.1.0.tar.gz
cd ${BASEDIR}/out/src/build-gcc/

# -- patch isl include header
sed -i 's/#include <isl\/ast_build.h>/#include <isl\/ast_build.h>\n#include <isl\/id.h>\n#include <isl\/space.h>\n/g' ../gcc-6.1.0/gcc/graphite.h

../gcc-6.1.0/configure CXXFLAGS="-fpermissive" --target=$TARGET --prefix="$PREFIX" --disable-nls --enable-languages=c,c++ --without-headers

make all-gcc
make all-target-libgcc

make install-gcc
make install-target-libgcc

After running this script the compiler should be created and reside under

out/path/bin/i686-elf-g++

Progress

Result

So far we’ve built a custom cross-compiler. In the next post we’ll be using that cross-compiler to compile our first kernel that can print text on a computer screen. We will then be booting it in QEmu to test it.