Bochs, PC Bootstrap, and GCC Calling Conventions

Introduction

This overview of the Bochs PC emulator, PC bootstrap procedures, and GCC calling conventions was originally written by the course staff for MIT's 6.828.

Part 1: PC Bootstrap

The purpose of the first exercise is to introduce you to x86 assembly language and the PC bootstrap process, and to get you started with the Bochs debugger.

Getting Started with x86 assembly

If you are not already familiar with x86 assembly language, you will quickly become familiar with it during this course! The PC Assembly Language Book is an excellent place to start; for a good hyper-linked reference to x86 assembly, go here.

Warning: Unfortunately the examples in the book are written for the NASM assembler, whereas we will be using the GNU assembler. NASM uses the so-called Intel syntax while GNU uses the AT&T syntax. While semantically equivalent, an assembly file will differ quite a lot, at least superficially, depending on which syntax is used. Luckily the conversion between the two is pretty simple, and is covered in Brennan's Guide to Inline Assembly.

Read the section "The Syntax" in Brennan's Guide to familiarize yourself with the most important features of GNU assembler syntax.

Simulating the x86

Instead of developing the operating system on a real, physical personal computer (PC), one often uses a simulator, which emulates a complete PC faithfully (i.e., the code you write for the simulator, boots on a real PC too). Using a simulator simplifies debugging; you can, for example, set break points inside of the simulated x86, which is difficult to do with the silicon-version of an x86 unless you have an expensive ICE.

We recommend using the Bochs PC Emulator. This emulator has been around for quite a while, and is slow and quirky but has a great many useful features. Another freely available PC emulator is QEMU, which is much faster than Bochs, but has less mature debugging facilities. You are welcome to give QEMU a try (or any of the commercially available PC virtual machine programs), but we'll generally assume you are either running on the bare metal or using Bochs.

Now you're ready to run Bochs. The configuration file for Bochs is named .bochsrc; a sample configuration file may be found here. To install the configuration file, copy it to a file named .bochsrc in your home directory. The sample configuration includes a command to make Bochs use the file disk.img as the contents of the simulated PC's "virtual hard disk" once Bochs starts running. You may need to edit the configuration file to match the local installation of Bochs or to point to the disk image if you have stored it somewhere other than the current directory; if you're having trouble, try looking through the Bochs documentation or come to office hours. To create a new 10M disk image, use the following command:

pucs% dd if=/dev/zero of=bochs.img bs=1024 count=10240
To put your boot loader and kernel stored in the file image into the Bochs hard disk image, use the following:
pucs% dd if=image of=bochs.img conv=notrunc
Once you start, Bochs, you'll see a sequence of output something like this:
pucs% bochs
========================================================================
                       Bochs x86 Emulator 2.1.1
                           February 08, 2004
========================================================================
00000000000i[     ] reading configuration from .bochsrc
------------------------------
Bochs Configuration: Main Menu
------------------------------
 
This is the Bochs Configuration Interface, where you can describe the
machine that you want to simulate.  Bochs has already searched for a
configuration file (typically called bochsrc.txt) and loaded it if it
could be found.  When you are satisfied with the configuration, go
ahead and start the simulation.
 
You can also start bochs with the -q option to skip these menus.
 
1. Restore factory default configuration
2. Read options from...
3. Edit options
4. Save options to...
5. Begin simulation
6. Quit now
 
Please choose one: [5]
Bochs has read the file .bochsrc describing the virtual x86 PC it will emulate for our kernel, and it is stopping to give you an opportunity to change the settings if desired before beginning the actual simulation. Since the configuration settings are already correct, just press Enter to start the simulation. (As Bochs points out, you can bypass this step in the future by typing 'bochs -q' instead of just 'bochs'.)
Next at t=0
(0) [0x000ffff0] f000:fff0 (unk. ctxt): jmp f000:e05b             ; ea5be000f0
<bochs:1>
Bochs has now started the simulated machine, and is ready to execute the first instruction. An X window should have popped up to show the "virtual display" of the simulated PC. The window is blank because the simulated PC hasn't actually booted yet - it's frozen in the state a real PC would be in just after being turned on and coming out of hardware reset but before executing any instructions.

The text that Bochs printed on your normal shell window, and the <bochs:1> prompt, is part of the Bochs debugging interface, which you can use to control and examine the state of the simulated PC. The main reference for this debugging interface that you should become familiar with is the section Using Bochs internal debugger in the Bochs User Manual. You can always get a reminder of the names of the most common commands by typing help:

<bochs:1> help
help - show list of debugger commands
help 'command'- show short command description
-*- Debugger control -*-
    help, q|quit|exit, set, instrument, show, trace-on, trace-off,
    record, playback, load-symbols, slist
-*- Execution control -*-
    c|cont, s|step|stepi, p|n|next, modebp
-*- Breakpoint management -*-
    v|vbreak, lb|lbreak, pb|pbreak|b|break, sb, sba, blist,
    bpe, bpd, d|del|delete
-*- CPU and memory contents -*-
    x, xp, u|disas|disassemble, r|reg|registers, setpmem, crc, info, dump_cpu,
    set_cpu, ptime, print-stack, watch, unwatch, ?|calc

For now, just type c to "continue" (i.e., start) execution from the current point. Some text should now appear in the Bochs window. If you have a disk full of zeros, the X window will disappear after briefly displaying output from the BIOS and Bochs will exit with an error saying that the disk does not contain a bootable image. It is your job to fix this!

Exercise 2. Scan through the Bochs internal debugger section of the Bochs user manual to get a feel for these commands and their syntax. Play with the commands a little: do some stepping and tracing through the code, examining CPU registers and memory and disassembling instructions at different points, without worrying too much yet about what the code is actually doing. While your kernel is waiting for user input (or at any other time the simulation is running), you can always hit CTRL-C in the shell window from which you ran Bochs in order to halt the simulation and break back into the Bochs debugger. Be sure you understand the distinction between which software you're interacting with when you type commands in the emulated PC versus in the Bochs debugger.

The PC's Physical Address Space

We will now dive into a bit more detail about how a PC starts up. A PC's physical address space is hard-wired to have the following general layout:

+------------------+  <- 0xFFFFFFFF (4GB)
|      32-bit      |
|  memory mapped   |
|     devices      |
|                  |
/\/\/\/\/\/\/\/\/\/\

/\/\/\/\/\/\/\/\/\/\
|                  |
|      Unused      |
|                  |
+------------------+  <- depends on amount of RAM
|                  |
|                  |
| Extended Memory  |
|                  |
|                  |
+------------------+  <- 0x00100000 (1MB)
|     BIOS ROM     |
+------------------+  <- 0x000F0000 (960KB)
|  16-bit devices, |
|  expansion ROMs  |
+------------------+  <- 0x000C0000 (768KB)
|   VGA Display    |
+------------------+  <- 0x000A0000 (640KB)
|                  |
|    Low Memory    |
|                  |
+------------------+  <- 0x00000000
The first PCs, which were based on the 16-bit Intel 8088 processor, were only capable of addressing 1MB of physical memory. The physical address space of an early PC would therefore start at 0x00000000 but end at 0x000FFFFF instead of 0xFFFFFFFF. The 640KB area marked "Low Memory" was the only random-access memory (RAM) that an early PC could use; in fact the very earliest PCs only could be configured with 16KB, 32KB, or 64KB of RAM!

The 384KB area from 0x000A0000 through 0x000FFFFF was reserved by the hardware for special uses such as video display buffers and firmware held in nonvolatile memory. The most important part of this reserved area is the Basic Input/Output System (BIOS), which occupies the 64KB region from 0x000F0000 through 0x000FFFFF. In early PCs the BIOS was held in true read-only memory (ROM), but current PCs store the BIOS in updateable flash memory. The BIOS is responsible for performing basic system initialization such as activating the video card and checking the amount of memory installed. After performing this initialization, the BIOS loads the operating system from some appropriate location such as floppy disk, hard disk, CD-ROM, or the network, and passes control of the machine to the operating system.

When Intel finally "broke the one megabyte barrier" with the 80286 and 80386 processors, which supported 16MB and 4GB physical address spaces respectively, the PC architects nevertheless preserved the original layout for the low 1MB of physical address space in order to ensure backward compatibility with existing software. Modern PCs therefore have a "hole" in physical memory from 0x000A0000 and 0x00100000, dividing RAM into "low" or "conventional memory" (the first 640KB) and "extended memory" (everything else). In addition, some space at the the very top of the PC's 32-bit physical address space, above all physical RAM, is now commonly reserved by the BIOS for use by 32-bit PCI devices.

With recent x86 processors it is now possible in fact for PCs to have more than 4GB of physical RAM, which means that RAM can extend further above 0XFFFFFFFF. In this case the BIOS must therefore arrange to leave a second hole in the system's RAM at the top of the 32-bit addressable region, in order to leave room for these 32-bit devices to be mapped.

The ROM BIOS

Close and restart Bochs, so that you once again see the first instruction to be executed:
Next at t=0
(0) [0x000ffff0] f000:fff0 (unk. ctxt): jmp f000:e05b             ; ea5be000f0
<bochs:1>

From this output you can conclude a few things:

Why does the Bochs start like this? This is how Intel designed the 8088 processor, which IBM used in their original PC. Because the BIOS in a PC is "hard-wired" to the physical address range 0x000f0000-0x000fffff, this design ensures that the BIOS always gets control of the machine first after power-up or any system restart - which is crucial because on power-up there is no other software anywhere in the machine's RAM that the processor could execute. The Bochs simulator comes with its own BIOS, which it maps at this location in the processor's simulated physical address space. On processor reset, the (simulated) processor sets CS to 0xf000 and the IP to 0xfff0, and consequently, execution begins at that (CS:IP) segment address. But how did the segmented address 0xf000:fff0 turn into the physical 0x000ffff0 we mentioned above?

To answer that we need to know a bit about real mode addressing. In real mode (the mode that PC starts off in), address translation works according to the formula: physical address = 16 * segment + offset. So, when the PC sets CS to 0xf000 and IP to 0xfff0, the physical address referenced is:

   16 * 0xf000 + 0xfff0   # in hex multiplication by 16 is
   = 0xf0000 + 0xfff0     # easy--just append a 0.
   = 0xffff0 

We can see that the PC starts executing 16 bytes from the end of the BIOS code. Therefore we shouldn't be surprised that the first thing that the BIOS does is jmp backwards to an earlier location in the BIOS; after all how much could it accomplish in just 16 bytes?

Exercise 3. Use the Bochs debugger to trace into the ROM BIOS for a few more instructions, and try to guess what it might be doing. You might find the common I/O address assignments table in Phil Storrs PC Hardware book handy, as well as other materials on the 318 reference materials page. No need to figure out all the details - just the general idea of what the BIOS is doing first.

When the BIOS runs, it sets up an interrupt descriptor table and initializes various devices such as the VGA display. This is where the "Plex86/Bochs VGABios" messages you see in the Bochs window come from. How can you find out exactly where in the BIOS this is happening? It happens that while in text mode, the VGA display is mapped in memory at address 0xb8000, and you can use a data watchpoint, or a breakpoint that fires when a particular memory location is read or written (instead of executed), to find out when and where the BIOS is writing these messages to the display.

To set a data watchpoint:

<bochs:1> watch write 0xb8000
<bochs:2> watch stop
<bochs:3> c
The first line sets the watchpoint. The second line instructs bochs to stop the simulation whenever a watchpoint fires.

Exercise 4. Set a watchpoint in the video display memory as instructed above. How many times is the watchpoint hit between when Bochs starts running and when the BIOS transfers control to the boot loader? Is it the "main" ROM BIOS in the 0x000f0000-0x000fffff region that is performing this task, or something else? Think about why the PC is designed this way.

After initializing the PCI bus and all the important devices the BIOS knows about, it searches for a bootable device such as a floppy, hard drive, or CD-ROM. Eventually, when it finds a bootable disk, the BIOS reads the boot loader from the disk and transfers control to it.

Part 2: The Boot Loader

Floppy and hard disks for PCs are by historical convention divided up into 512 byte regions called sectors. A sector is the disk's minimum transfer granularity: each read or write operation must be one or more sectors in size and aligned on a sector boundary. If the disk is bootable, the first sector is called the boot sector, since this is where the boot loader code resides. When the BIOS finds a bootable floppy or hard disk, it loads the 512-byte boot sector into memory at physical addresses 0x7c00 through 0x7dff, and then uses a jmp instruction to set the CS:IP to 0000:7c00, passing control to the boot loader. Like the BIOS load address, these addresses are fairly arbitrary - but they are fixed and standardized for PCs.

The ability to boot from a CD-ROM came much later during the evolution of the PC, and as a result the PC architects took the opportunity to rethink the boot process slightly. As a result, the way a modern BIOS boots from a CD-ROM is a bit more complicated (and more powerful). CD-ROMs use a sector size of 2048 bytes instead of 512, and the BIOS can load a much larger boot image from the disk into memory (not just one sector) before transferring control to it. For more information, see the "El Torito" Bootable CD-ROM Format Specification.

For 318, however, we will use the conventional hard drive boot mechanism, which means that our boot loader must fit into a measly 512 bytes. The boot loader must perform two main functions:

  1. First, the boot loader switches the processor from real mode to 32-bit protected mode, because it is only in this mode that software can access all the memory above 1MB in the processor's physical address space. Protected mode is described briefly in sections 1.2.7 and 1.2.8 of PC Assembly Language, and in great detail in the Intel architecture manuals. At this point you only have to understand that translation of segmented addresses (segment:offset pairs) into physical addresses happens differently in protected mode, and that after the transition offsets are 32 bits instead of just 16.
  2. Second, the boot loader reads the kernel from the hard disk either by directly accessing the IDE disk device registers via the x86's special I/O instructions or by using special BIOS routines. You will not need to learn much about programming specific devices in this class: writing device drivers is in practice a very important part of OS development, but from a conceptual or architectural viewpoint it is also one of the least interesting.

You can set breakpoints in Bochs with the b command. You have to give the base explicitly, so say something like b 0x7c00 for hexadecimal. A full command overview is at http://bochs.sourceforge.net/doc/docbook/user/x2461.html. From the debugger, you can continue execution using the c and s commands: c causes Bochs to continue execution indefinitely; and s allows you to step through the instructions, executing exactly n instructions (a default of 1 if the parameter n is not specified) before suspending execution again. trace-on and trace-off can be used to set tracing before using the other commands.

To examine instructions in memory (besides the immediate next one to be executed, which Bochs prints automatically), you use the u command. This command has the syntax u/n addr, where n is the number of consecutive instructions to disassemble and addr is the memory address at which to start disassembling.