Featured post

Linux Internals: How /proc/self/mem writes to unwritable memory

2 Replies

Introduction

An obscure quirk of the /proc/*/mem pseudofile is its “punch through” semantics. Writes performed through this file will succeed even if the destination virtual memory is marked unwritable. In fact, this behavior is intentional and actively used by projects such as the Julia JIT compiler and rr debugger.

This behavior raises some questions: Is privileged code subject to virtual memory permissions? In general, to what degree can the hardware inhibit kernel memory access?

By exploring these questions1, this article will shed light on the nuanced relationship between an operating system and the hardware it runs on. We’ll examine the constraints the CPU can impose on the kernel, and how the kernel can bypass these constraints.

Continue reading →
Featured post

Double fetches, scheduling algorithms, and onion rings

1 Reply

Most people thought I was crazy for doing this, but I spent the last few months of my gap year working as a short order cook at a family-owned fast-food restaurant. (More on this here.) I’m a programmer by trade, so I enjoyed thinking about the restaurant’s systems from a programmer’s point of view. Here’s some thoughts about two such systems.

Continue reading →
Featured post

What they don’t tell you about demand paging in school

2 Replies

This post details my adventures with the Linux virtual memory subsystem, and my discovery of a creative way to taunt the OOM (out of memory) killer by accumulating memory in the kernel, rather than in userspace.

Keep reading and you’ll learn:

  • Internal details of the Linux kernel’s demand paging implementation
  • How to exploit virtual memory to implement highly efficient sparse data structures
  • What page tables are and how to calculate the memory overhead incurred by them
  • A cute way to get killed by the OOM killer while appearing to consume very little memory (great for parties)

Note: Victor Michel wrote a great follow up to this post here.

Continue reading →
Featured post

How setjmp and longjmp work (2016)

2 Replies

Pretty recently I learned about setjmp() and longjmp(). They’re a neat pair of libc functions which allow you to save your program’s current execution context and resume it at an arbitrary point in the future (with some caveats1). If you’re wondering why this is particularly useful, to quote the manpage, one of their main use cases is “…for dealing with errors and interrupts encountered in a low-level subroutine of a program.” These functions can be used for more sophisticated error handling than simple error code return values.

I was curious how these functions worked, so I decided to take a look at musl libc’s implementation for x86. First, I’ll explain their interfaces and show an example usage program. Next, since this post isn’t aimed at the assembly wizard, I’ll cover some basics of x86 and Linux calling convention to provide some required background knowledge. Lastly, I’ll walk through the source, line by line.

Continue reading →

3 weeks of GTD

Leave a reply

I read GTD (Getting Things Done) a few weeks ago and have been applying it since then.

I can say since implementing it: my stress is lower, I feel much more in control & at peace, and I’m happier overall. So I’d say it works — or at least there’s really something to it.

Concretely:

The process of taking a fuzzy/vague/unpleasant idea of a project and progressively making it concrete by 1. Identifying the specific outcome, and 2. The specific next action has been critical for me. I wasn’t aware of this and would often get overwhelmed by these fuzzy ideas and let them linger (which made them worse). I often find that after doing the 2 steps, the idea is much less overwhelming, and often much easier than I thought (or even can be done trivially).

I was using Omnifocus wrong — I now work out of my Forecast view which I believe is the intended way to use OF and actually works well. OF 4 is great, in particular for how much better the iOS app is (specifically implementing Focus mode).

I find that I make more forward progress on things in general. Adding an action to a list gives me extra “credit” for having done it, because now I can check it off. But the big thing is I’m now aware of things that I can take action on (and how small and simple they often are). Often I get stuck on actions that involve other people (Ask person X this, post on Slack asking for help with X), and having an action be made concrete and on a list can help me power through it and simply send the text or make the call. I’m more aware that I have no answer to the question “So what’s you’re excuse for not taking the action to move X forward?”, and just do the thing.

A physical inbox has been useful. I’ve discovered that having my physical environment be clean is very important for me, and a physical inbox facilitates this by creating a designated, controlled place for clutter.

I’m amazed at how much of GTD and being productive is about writing things down in such a way that will trick your future brain into not being overwhelmed, and actually doing them.

The first part is writing specific, exciting, and inspiring project names (i.e. “Host best friend for a great weekend” instead of “Best friend visit”).

But the second and more important part is realizing how not to name actions. For example, I’ve learned that I shouldn’t write tasks like “Decide A”, or “Problem solve B situation”. My brain will immediately get overwhelmed and resist the urge to even consider this. Breaking those down into even smaller steps is a must. Like “List pros/cons of A decision” (Not too hard – just listing bullets, not “making a decision”), or “Brain dump B situation on whiteboard”).

There’s an art to writing the next action.

Even with GTD it’s still possible to overload yourself and put more into your system than you have capacity for. I’m still struggling with how to manage when I have tasks I constantly postpone week after week.

What I learned in my 20s

Leave a reply

I had the privilege of speaking to my friend Andre’s high school class this week about my career and path to it. I didn’t have time for all the advice I’d give, so I’m putting it here:

It’s ok to not be able to answer “So where do you see yourself in 5 years?”.

That’s a hard question, and it’s ok to not immediately know the answers to hard questions.

In my experience, most of my life was in a state of not really knowing this, with one major exception: When I realized in 2017-2018 that I really wanted to work for Ableton in Germany. Then it became startingly clear where I wanted to be, and approximately what I needed to do.

My advice would be to simply start taking actions while being observant of yourself, and your strengths, interests, and natural inclinations. At what things do you naturally work harder than other people? What things seem like play to you, but work to others? Those are hints at areas you can excel and become world class.

Eventually after enough action (and reflection), you might have an insight about something you deeply want to make happen. And then suddenly it becomes clear.

“As you start to walk on the way, the way appears.” – Rumi

It might seem like life is a race, from start to finish, where checkpoints are things like: university, job, marriage, children. When you “graduate high school” (i.e. become an adult), the gun goes off. Everyone starts running and the first one to make it through, wins.

In my experience, the “race” is actually a custom trail for every single person. When you “graduate high school” (i.e. become an adult), the gun goes off and everyone starts running in different directions. Another person’s progress towards their endpoint has little to no relevance on your progress towards yours.

The only competition is to know yourself as fully as possible, and act with maximum authenticity towards that truth.

A simple strategy towards achieving success and fulfillment is looking for:

  1. A “vertical”: An industry which you have particular interest (e.g. music, fashion, film, journalism, activism, sports, …)
  2. A “horizontal”: A skill which you have interest in and aptitude for (e.g. technology, writing, art, photography, communication, …)

And then work at the intersection of the two. Basically every vertical needs every horizontal. Every industry needs programmers, communicators, creatives, etc.

This strategy is not foolproof, but can be a good approximate path for those without one. And it worked well for me!

Seek like-minded peers. The first time this happened to me blew my mind — I went to the National Guitar Workshop in 2010 and met a bunch of other teenagers that were interested in writing original metal compositions and recording them on computers. This was a strictly positive experience and gave me friendship, motivation, and a sense of community.

Then in college, I went to NU Hacks and the same thing happened. I found a great network of aspiring hackers, and we became great friends and learned together.

In both cases, all these people are now doing amazing things in the world in their field. And these relationships have turned into the kind of life-long friendships that are one of the best things in life.

Greatness is built iteratively, over a long period of time.

Don’t be afraid to exploit your unfair advantages.

gardenOS Update 2: 12 weeks of streaming

Leave a reply

I’ve been streaming myself doing basic OS hacking for 12 weeks now. Here’s a reflection & some things I’ve learned:

  • I’ve gotten compliments on my audio quality – good to know that my mic setup is good.
  • I got negative feedback that my screen size was too small – I took action and now stream at 720p which seems comically large to me, but produces a more readable video.
  • I got a positive comment on how relaxed I was.
  • I got miscellaneous positivity and encouragement from people, wishing me well for my OS, even though I think it’s unlikely they really watched the whole thing or grasp how wildly far I am away from anything significant.
  • Performance varies from ~70 views on a video to 1.6k. Live streams tend to do better than pre-recorded videos. I believe also posting about it on Twitter helps.
  • I got one chat comment from a person really excited about gardening specifically, and followed specifically because I used this phrasing.
  • I have a couple people that are interested and follow fairly closely – shoutout to l1zard and arash11!
  • I have 10 people in my Discord despite not publicizing it. l1zard and Glenford Williams are the most active.
  • It seems to not really matter that I’m not doing any serious OS dev, or that I was doing boring stuff like working on the build system. Some people were still interested even though I was doing possibly the most boring tasks possible.
  • I slightly regret spending so long doing random Makefile and build system refactors which in retrospect were a bit of a waste of time. But in the end, taking any action at all was the most important thing, and now I have something in motion and can course correct. I already course corrected by quitting the build system and moving to working on C refactors. I might course correct again and entirely switch to JOS which has much better foundational build infrastructure etc, and is also much more minimal – leaving much more work for me to actually do, including accessible beginner projects. It might be worth simply trying the JOS assignments rather than randomly hacking around at will.

Advice for learning the dark arts

Leave a reply

I loved this episode of “My First Million”:

https://www.mfmpod.com/become-a-better-writer-in-60-minutes-masterclass/

It’s about how to communicate effectively for persuasion, which is very useful in business and life.

But I wanted to explicitly state an underlying assumption of their conversation: These techniques are a “dark art”.

These are not general principles for all kinds of writing. It would be mistaken to assume that one must apply these principles in any kind of writing — ie.g. academic, creative, formal. These are techniques to deploy when you have specific goals for your writing, and are writing within specific contexts.

They come with tradeoffs and sacrifices, such as making your writing more “sales-y” or “clickbait-y”, which can decrease credibility or compromise an artistic vision. The exact tradeoffs depend on the context and community, but they exist. That’s why I call them a dark art.

What is mastery?

Leave a reply

To use an analogy from cooking:

Master is when the chef not only knows how to make oatmeal the “right” way, but is able to answer if you ask “What happens if we use half the water? Twice as much butter? No butter at all? 4x the milk?”. Because they’ve tried all these things before.

A master understands where the canonical solution fits in the greater design space of all solutions, and the tradeoffs involved. They can explain, in depth, what happens if you “do it wrong”, and can provide examples of instances where you might want to.

A master understands when the laws of gravity actually break. 1

gardenOS Update 1

Leave a reply

(This is a random collection of thoughts around my new operating systems project, “gardenOS”.)

What’s up with the “gardening” terminology?

This is a phrase that I feel perfectly describes the ethos of this project. In the same way people have gardens as calm places to express themselves, learn things, and have fun doing work, I want to create an analogous place for exploring my interest in operating systems.

Some key tenants of the approach:

  • No stress
  • Have fun
  • Pursue whatever interests you

I don’t have particularly strong OSdev skills at the moment, so I’m especially focusing on doing small, easy tasks, such as cleaning up the build system. I do this in the same way you might spend an afternoon picking weeds in your little garden. It’s easy, well understood, not too complicated, no large decisions to be made — and it concretely improves the project. It’s a concrete win you can lock in, in a fixed amount of time, with fairly little work.

Disclaimer: I’m almost apprehensive to even give this project a name

I’m worried that even naming this project (“gardenOS”) will put too much pressure on me. I am deeply aware that OS’s take monumental amounts of time and energy to even get to basic states. And at my current rate (~2 hours a week), it’s unlikely we will get to even basic levels soon.

I wasn’t expecting things to get this philosophical this quickly. My focus above all, is to have fun, learn things, and make some small progress each week in the stream. Each week where I do a stream or do some work, any any of that happens, is a win.

I explicitly hold very few expectations around a future “goal” of the project or where I want it to end up. I just want to have fun and learn things about operating systems.

The loose vision I have is to create a minimal OS for play and experimentation. It should be a high quality codebase, and I should work on it as if I wanted to present my best self as an aspiring pro systems programmer.

The ethos of the project

Even though the project is in a maximally nascent stage, I already feel a certain ethos evolving. In the project, we emphasize:

  • Relaxed, casual, kind attitude
  • Learning mindset
  • Ambition to use programming best practices, and aspiring to become pro systems programmers
  • Portray an accurate “slice of life” of systems programming. Meaning: Show the day to day, which is not always thrilling tasks (like adding a syscall), but simply just working on the build system or refactoring some code.

My experience after doing four streams

I’ve done four streams so far (2 live, 2 recorded). My big worries are that I’ll run out of stuff to do and have to scramble to find something to do, live.

This has never happened — I’m always surprised at the adventures “we” find ourselves going on

The use of “we”

The project is just me at the moment, but there’s something about using “we” to describe it that makes me feel good.

It’s not just to make myself seem more impressive, or feel less alone. When I use it, I think of the handful of enthusiastic people that have joined me in the live streams, or left comments with questions or encouragement.

Even in these earliest of days, there is some tiny, microscopic community feel forming. So when I say “we”, I speak for this community.

The astonishing cost/benefit asymmetry of a four-day work week

Leave a reply

Work update: I reduced to working 4 days/week and the cost/benefit asymmetry is astonishing.

Just one extra free day might not sound like much, but I feel like I gain >100% more high quality free time (Friday off is even better than Saturday; Sunday is not high quality free time for me – too much adulting to do).

And I lose only 20% of my productive work capacity (Well a bit more; Friday would be a more productive day than average for me b/c it’s quieter & less meetings).

I’ve constantly felt squashed the last few years, but always convinced myself it was a me problem, rather than a possibility that even “normal” working hours didn’t actually leave me with enough free time for everything I had to do*. (Maybe some of both)

But my energy and mood are way better than in a long time, so maybe it goes to show that the latter was the case, and one extra day can go a long way. (But that’s less surprising when you frame it as 100% more time).

I do need to be a bit more conscious of how I use my work week, and I have noticed a tendency to try to fit 5 days of work into 4… but overall it’s going well. I’m curious if I end up filling up the extra personal capacity and end up just as stressed, but I somehow doubt that will happen.

—

*You might ask, well why are you so busy anyway? Are you just piling on voluntary responsibilities?

I’ve thought about this at length and I think my answer is glibly, “expat life”.

Tip: When you’re learning a new programming language, look up prominent open source projects and copy their style

Leave a reply

Tip: When you’re learning a new programming language, look up prominent open source projects and copy their style.

Aside from the core language, there are many conventions & little details to learn: naming (variables, classes, files), file structuring, literal code formatting

These are things few blogs talk about because it’s highly opinionated. But nevertheless when you’re learning, you’ll benefit from at least some reference for these.

Find a few “professional” open source projects and browse to see what various interpretations of “professional style” are. Then pick one you like most.

Be careful of picking projects that are too old — they might use older style for consistency with legacy, even though they might ideally wish to modernize it.

And ideally pick projects whose contributors are experienced engineers who work on it full-time. Since they “live in” the codebase, they’re less likely to tolerate sloppiness – or are at least more invested in cleaning it up.

The last idea is influenced by @awesomekling, who talks about similar things in his classic “My top advice for programmers looking to improve” car-talk video =]