osdev journal: The virtual memory unit test experiment

In streams 97-100 or so, I start a basic implementation of code that manages the page tables of an address space. I figured this is fairly generic data structure code in the end, so it might be useful to write unit tests for it.

In the end the experiment sort of succeeded, but also I’ve abandoned the strong unit testing focus for the forseeable future.

The theory was mostly correct that this code is fairly generic data structure code, with some exceptions.

The code needs to allocate physical memory frames (in production), for page tables. It uses a frame allocator in the kernel to do so. That’s not available in the userspace unit tests, so that needs to be mocked out by a mock physical memory allocator.

Also, there’s address conversion. The VM code needs to convert between virtual and physical addresses because page tables store physical frame numbers. We need to convert it back to virtual in order to work with the data in the kernel, e.g. when walking the page tables.

These were a bit annoying to mock, but overall the approach worked and I could write unit tests for the map function.

But the reason I abandoned the approach in general for the kernel is because there are so many other things that will eventually need to be mocked. TLB flushing after page tables are modified. Other kinds of hardware accesses. And that’s not even to mention timing and concurrency which will play a major role.

A major difference between a kernel and normal application is the extent to which the code interfaces with “external services”. It’s commonplace to mock out external services, but in app code, there is usually finite set of them. In the kernel, there are so many touchpoints with hardware and external services that the mock layer would grow huge.

This actively gets in the way of development and causes friction. It’s not to say it’s not worth it, but there are reasons it’s hard to unit test kernels and why higher level system and integration tests are preferred.

Any thoughts?