Featured post

Linux Internals: How /proc/self/mem writes to unwritable memory

2 Replies

Introduction

An obscure quirk of the /proc/*/mem pseudofile is its “punch through” semantics. Writes performed through this file will succeed even if the destination virtual memory is marked unwritable. In fact, this behavior is intentional and actively used by projects such as the Julia JIT compiler and rr debugger.

This behavior raises some questions: Is privileged code subject to virtual memory permissions? In general, to what degree can the hardware inhibit kernel memory access?

By exploring these questions1, this article will shed light on the nuanced relationship between an operating system and the hardware it runs on. We’ll examine the constraints the CPU can impose on the kernel, and how the kernel can bypass these constraints.

Continue reading →
Featured post

Double fetches, scheduling algorithms, and onion rings

3 Replies

Most people thought I was crazy for doing this, but I spent the last few months of my gap year working as a short order cook at a family-owned fast-food restaurant. (More on this here.) I’m a programmer by trade, so I enjoyed thinking about the restaurant’s systems from a programmer’s point of view. Here’s some thoughts about two such systems.

Continue reading →
Featured post

What they don’t tell you about demand paging in school

2 Replies

This post details my adventures with the Linux virtual memory subsystem, and my discovery of a creative way to taunt the OOM (out of memory) killer by accumulating memory in the kernel, rather than in userspace.

Keep reading and you’ll learn:

  • Internal details of the Linux kernel’s demand paging implementation
  • How to exploit virtual memory to implement highly efficient sparse data structures
  • What page tables are and how to calculate the memory overhead incurred by them
  • A cute way to get killed by the OOM killer while appearing to consume very little memory (great for parties)

Note: Victor Michel wrote a great follow up to this post here.

Continue reading →
Featured post

How setjmp and longjmp work (2016)

2 Replies

Pretty recently I learned about setjmp() and longjmp(). They’re a neat pair of libc functions which allow you to save your program’s current execution context and resume it at an arbitrary point in the future (with some caveats1). If you’re wondering why this is particularly useful, to quote the manpage, one of their main use cases is “…for dealing with errors and interrupts encountered in a low-level subroutine of a program.” These functions can be used for more sophisticated error handling than simple error code return values.

I was curious how these functions worked, so I decided to take a look at musl libc’s implementation for x86. First, I’ll explain their interfaces and show an example usage program. Next, since this post isn’t aimed at the assembly wizard, I’ll cover some basics of x86 and Linux calling convention to provide some required background knowledge. Lastly, I’ll walk through the source, line by line.

Continue reading →

You need your own workshop

Leave a reply

This is originally Derek Sivers’ idea, from his book “Anything you want”.

We all need a place to play.

Kids need playgrounds and sandboxes. Musicians need an instrument. Mad scientists need a laboratory.

Those of us with business ideas? We need a company.

Not for the money, but because it’s our place to experiment, create, and turn thoughts into reality. We need to pursue our intrinsic motivation.

We have so many interesting ideas and theories. We need to try them!

The happiest people are not lounging on beaches. They’re engaged in interesting work!

Following curiosity is much more fun than being idle. Even if you never have to work a day in your life.

That’s the best reason to have a company. It’s your playground, your instrument, your laboratory. It’s your place to play!

Get the ideas out of your head and into the world.

https://sive.rs/laboratory
  • Gardeners need a garden.
  • Car enthusiasts need a garage.
  • Entrepreneurs need a business.
  • Artists need a studio.

But what about systems programmers?

Systems programmers need a project. A place for them to explore, work, play.

Live streaming myself working on my baby operating system has felt great over the last 14 weeks. And now with Sivers’ idea in mind, I can totally see why. It’s finally my own project where I have full control, and it’s a large enough project where there is infinite potential for the things and can do and learn within it.

No matter what your craft is, if you aspire to be great at it, you need a safe, comfortable “space” to work on your craft.

If you have aspirations, but don’t have a space, you likely haven’t fully committed, or given yourself permission to publicly identify as an enthusiast of the craft. (Actions speak louder than words).

Taking action to make that space for yourself can be scary, because it exposes physical, undeniable proof of your interest, which is vulnerable. But in my experience, it can also be deeply affirming, exciting, and motivating.

Getting into reading again by playing offense

Leave a reply

I used to look at long non-fiction books and immediately wince, thinking of how long it would probably take me to read it, and what a slog it would probably be. Of course, this is a strange point of view that seems to forget that books can actually be captivating and fun.

But beyond that, something that helped me is changing my perspective. Rather than letting the book be in control, I now try to play more on offense.

Instead of allowing a long book to suck a potentially infinite time out of me (which means in practice, I won’t even start), I now give books a budget. If I only have 2 hours of time to give a book, oh well, that’s all it gets. I stop and move on. Hopefully the book can deliver some of its meaning in that time frame, or even better, captivate me and convince me to renegotiate my relationship with it — and give it more time.

Instead of putting the responsibility on you to slog through and make it to the end, put the responsibility on the book to earn your time.

(This does work better when you don’t pay for the book – ideally by lending it from someone else.)

I like WordPress

Leave a reply

Among programmers, it’s very unfashionable to use WordPress for your blog. (“PHP? Yuck.”) Instead, you should be using the latest minimalist static site generator, hosted on the latest free static hosting.

(A decade ago, this was Jekyll on Github Pages — I haven’t bothered to keep up, but I did put in my time. At various points, my blog was based on Jekyll, Pelican, and Octopress).

In 2020 I decided to restart my blog, but just use WordPress.

Three years, later this has been an unambiguously good decision. I keep running into things that save me significant time, compared to me trying to code this myself, or use a static site generator.

I think the proof is in the pudding. If the goal is to actually publish writing on the internet, consistently, over a long period of time, I’ve done that (or at least am well on my way — see the Archive).

Here are some handy things I’ve found myself needing that were just there for me. I’ll add to it as I run into more.

  • Automatic redirects if you change a post slug
  • Rich plugin ecosystem for nearly everything
  • Extensive documentation, both first and third-party on how to do things. Even ChatGPT can advise.
  • Ability to customize with PHP if absolutely necessary
  • Migrating to a different permalink structure was a piece of case with the Redirects plugin
  • Built in RSS feed
  • Built in Recent post
  • Built in Top posts/pages (via Jetpack)
  • Built in downtime monitoring (via Jetpack)
  • Easy mailing list integration (MailChimp, ConvertKit, etc)
  • Built in grouping and taxonomy features (Tags, Categories)
  • Built in Monthly Archives
  • Built in comments
  • WordPress/Jetpack mobile app for easy editing/moderation on the go

Be your most authentic self (and write about whatever you want)

Leave a reply

I love writing about computers, but I also love writing about other topics like creativity, art, and productivity. However, many programmers out there strictly blog about technical topics, which made me feel a bit weird for posting random stuff like poems or my experience with GTD.

This led me to a dilemma: Do I blog all in one place, or do do I perhaps create a separate blog for non-technical content?

My answer is to apply my “golden piece of advice”: Do what feels most authentic to you.

For me, writing about all of my interests is the most authentic expression of myself, so when in doubt, I do this. Curbing this instinct, and making a strictly technical blog just to be like “all the other programmers” wouldn’t be.

It’s totally possible that “all the other programmers” simply don’t feel a desire to write about anything else. So making a strictly technical blog is their maximally authentic expression of themselves — which is great for them! Let’s all do what feels most authentic to us.

Hiding in this case study is a profound lesson about life. The situation applies equally to any other life situation where you feel some pull to act in one way, but feel some hesitation upon observing “everyone around you” seems to act.1

When in doubt, apply the “golden advice”: Do what feels most authentic to you.

I’m very happy with this decision. It feels great to have a single place which all of my thinking, which also has the practical benefit of making it easier for potential followers to submerge themselves in all my content.

I also believe this will win in the long run as it’s more likely to resonate with like-minded people that can respect having a myriad of interests — the kind of people I’m looking to connect with!

Lastly, there’s the non-trivial but subtle benefit that simplicity of infrastructure & accounts actually matters and translated into a lot of time saved.

WIP: Fame is a hamster wheel

Leave a reply

Different forms of capital have different benefits and drawbacks. Money has the benefit that it can grow itself over time, with the downside that it is heavily scrutinized and taxed. Fame has the benefit that it is not taxed, but has the interesting drawback in that it requires maintenance, and lessens over time.

To retain your fame, you need to continually be doing things and delivering value to your audience. Otherwise, your audience will naturally shrink as people forget about you, and their attention is diverted elsewhere by others playing the fame game.

So if you think achieving fame will make you happy, be careful. The moment you achieve the fame you’ve been seeking is not the “end” — it’s actually the beginning of a hamster wheel you’ve just stepped onto. You might stop to celebrate, but don’t stop for too long — now you need to worry about maintaining it (or even exploiting it), lest you lose it and become a “one hit wonder”. The only other alternative is to keep running.

(This post was written based on my small experiences obtaining nano-fame with my offlinemark and comfort projects).

3 weeks of GTD

1 Reply

I read GTD (Getting Things Done) a few weeks ago and have been applying it since then.

I can say since implementing it: my stress is lower, I feel much more in control & at peace, and I’m happier overall. So I’d say it works — or at least there’s really something to it.

Concretely:

The process of taking a fuzzy/vague/unpleasant idea of a project and progressively making it concrete by 1. Identifying the specific outcome, and 2. The specific next action has been critical for me. I wasn’t aware of this and would often get overwhelmed by these fuzzy ideas and let them linger (which made them worse). I often find that after doing the 2 steps, the idea is much less overwhelming, and often much easier than I thought (or even can be done trivially).

I was using Omnifocus wrong — I now work out of my Forecast view which I believe is the intended way to use OF and actually works well. OF 4 is great, in particular for how much better the iOS app is (specifically implementing Focus mode).

I find that I make more forward progress on things in general. Adding an action to a list gives me extra “credit” for having done it, because now I can check it off. But the big thing is I’m now aware of things that I can take action on (and how small and simple they often are). Often I get stuck on actions that involve other people (Ask person X this, post on Slack asking for help with X), and having an action be made concrete and on a list can help me power through it and simply send the text or make the call. I’m more aware that I have no answer to the question “So what’s you’re excuse for not taking the action to move X forward?”, and just do the thing.

A physical inbox has been useful. I’ve discovered that having my physical environment be clean is very important for me, and a physical inbox facilitates this by creating a designated, controlled place for clutter.

I’m amazed at how much of GTD and being productive is about writing things down in such a way that will trick your future brain into not being overwhelmed, and actually doing them.

The first part is writing specific, exciting, and inspiring project names (i.e. “Host best friend for a great weekend” instead of “Best friend visit”).

But the second and more important part is realizing how not to name actions. For example, I’ve learned that I shouldn’t write tasks like “Decide A”, or “Problem solve B situation”. My brain will immediately get overwhelmed and resist the urge to even consider this. Breaking those down into even smaller steps is a must. Like “List pros/cons of A decision” (Not too hard – just listing bullets, not “making a decision”), or “Brain dump B situation on whiteboard”).

There’s an art to writing the next action.

Even with GTD it’s still possible to overload yourself and put more into your system than you have capacity for. I’m still struggling with how to manage when I have tasks I constantly postpone week after week.

But I am learning. I’m finding that if something continually gets postponed, there are a few things I can consider:

  • Is it even important? Can I simply delete it?
  • Is it a matter of phrasing? Can I re-word this to be more palatable to my subconscious brain?
  • It is a matter of breaking it down even further to a smaller increment? But not so small as to be meaningless — my subconscious brain sees right through that. (i.e. “Make a google doc” isn’t quite enough for me).

What I learned in my 20s

Leave a reply

I had the privilege of speaking to my friend Andre’s high school class this week about my career and path to it. I didn’t have time for all the advice I’d give, so I’m putting it here:

It’s ok to not be able to answer “So where do you see yourself in 5 years?”.

That’s a hard question, and it’s ok to not immediately know the answers to hard questions.

In my experience, most of my life was in a state of not really knowing this, with one major exception: When I realized in 2017-2018 that I really wanted to work for Ableton in Germany. Then it became startingly clear where I wanted to be, and approximately what I needed to do.

My advice would be to simply start taking actions while being observant of yourself, and your strengths, interests, and natural inclinations. At what things do you naturally work harder than other people? What things seem like play to you, but work to others? Those are hints at areas you can excel and become world class.

Eventually after enough action (and reflection), you might have an insight about something you deeply want to make happen. And then suddenly it becomes clear.

“As you start to walk on the way, the way appears.” – Rumi

It might seem like life is a race, from start to finish, where checkpoints are things like: university, job, marriage, children. When you “graduate high school” (i.e. become an adult), the gun goes off. Everyone starts running and the first one to make it through, wins.

In my experience, the “race” is actually a custom trail for every single person. When you “graduate high school” (i.e. become an adult), the gun goes off and everyone starts running in different directions. Another person’s progress towards their endpoint has little to no relevance on your progress towards yours.

The only competition is to know yourself as fully as possible, and act with maximum authenticity towards that truth.

A simple strategy towards achieving success and fulfillment is looking for:

  1. A “vertical”: An industry which you have particular interest (e.g. music, fashion, film, journalism, activism, sports, …)
  2. A “horizontal”: A skill which you have interest in and aptitude for (e.g. technology, writing, art, photography, communication, …)

And then work at the intersection of the two. Basically every vertical needs every horizontal. Every industry needs programmers, communicators, creatives, etc.

This strategy is not foolproof, but can be a good approximate path for those without one. And it worked well for me!

Seek like-minded peers. The first time this happened to me blew my mind — I went to the National Guitar Workshop in 2010 and met a bunch of other teenagers that were interested in writing original metal compositions and recording them on computers. This was a life-changing experience and gave me friendship, motivation, and a sense of community.

Then in college, I went to NU Hacks and the same thing happened. I found a great network of aspiring hackers, and we became great friends and learned together.

In both cases, all these people are now doing amazing things in the world in their field. And these relationships have turned into the kind of life-long friendships that are one of the best things in life.

Greatness is built iteratively, over a long period of time.

Don’t be afraid to exploit your unfair advantages.

gardenOS Update 2: 12 weeks of streaming

Leave a reply

I’ve been streaming myself doing basic OS hacking for 12 weeks now. Here’s a reflection & some things I’ve learned:

  • I’ve gotten compliments on my audio quality – good to know that my mic setup is good.
  • I got negative feedback that my screen size was too small – I took action and now stream at 720p which seems comically large to me, but produces a more readable video.
  • I got a positive comment on how relaxed I was.
  • I got miscellaneous positivity and encouragement from people, wishing me well for my OS, even though I think it’s unlikely they really watched the whole thing or grasp how wildly far I am away from anything significant.
  • Performance varies from ~70 views on a video to 1.6k. Live streams tend to do better than pre-recorded videos. I believe also posting about it on Twitter helps.
  • I got one chat comment from a person really excited about gardening specifically, and followed specifically because I used this phrasing.
  • I have a couple people that are interested and follow fairly closely – shoutout to l1zard and arash11!
  • I have 10 people in my Discord despite not publicizing it. l1zard and Glenford Williams are the most active.
  • It seems to not really matter that I’m not doing any serious OS dev, or that I was doing boring stuff like working on the build system. Some people were still interested even though I was doing possibly the most boring tasks possible.
  • I slightly regret spending so long doing random Makefile and build system refactors which in retrospect were a bit of a waste of time. But in the end, taking any action at all was the most important thing, and now I have something in motion and can course correct. I already course corrected by quitting the build system and moving to working on C refactors. I might course correct again and entirely switch to JOS which has much better foundational build infrastructure etc, and is also much more minimal – leaving much more work for me to actually do, including accessible beginner projects. It might be worth simply trying the JOS assignments rather than randomly hacking around at will.

I’ve had a lot more growth than I would have thought: going from 300 to 700 subscribers, and > 1k views on some videos. There are a lot of reasons to think this wouldn’t have achieved those results:

  • It’s long, unedited, and the value in it is very sparse
  • I was just working on the build system, not really doing anything interesting
  • My font size was way too small for most streams
  • I stream on Sundays when people are plausibly out doing things (but on the other hand, maybe they’re relaxing inside)

Advice for learning the dark arts

Leave a reply

I loved this episode of “My First Million”:

https://www.mfmpod.com/become-a-better-writer-in-60-minutes-masterclass/

It’s about how to communicate effectively for persuasion, which is very useful in business and life.

But I wanted to explicitly state an underlying assumption of their conversation: These techniques are a “dark art”.

These are not general principles for all kinds of writing. It would be mistaken to assume that one must apply these principles in any kind of writing — ie.g. academic, creative, formal. These are techniques to deploy when you have specific goals for your writing, and are writing within specific contexts.

They come with tradeoffs and sacrifices, such as making your writing more “sales-y” or “clickbait-y”, which can decrease credibility or compromise an artistic vision. The exact tradeoffs depend on the context and community, but they exist. That’s why I call them a dark art.

What is mastery?

Leave a reply

To use an analogy from cooking:

Master is when the chef not only knows how to make oatmeal the “right” way, but is able to answer if you ask “What happens if we use half the water? Twice as much butter? No butter at all? 4x the milk?”. Because they’ve tried all these things before.

A master understands where the canonical solution fits in the greater design space of all solutions, and the tradeoffs involved. They can explain, in depth, what happens if you “do it wrong”, and can provide examples of instances where you might want to.

A master understands when the laws of gravity actually break. 1