Cargo Cult Programming and Just-in-Time Learning

Cargo Cult Programming

The programming community has a peculiar faith in low-level code.

When someone working at the application layer hears "you still need to know the fundamentals," the response is reflexive. The implication is that you're not a real programmer unless you know the OS, the compiler, and cache lines.

This belief is further reinforced by the fact that most of what computer science and engineering curricula teach through the ACM layers falls exactly in that territory.

And the core of the problem is the phrase "it's good to know." That claim is irrefutable. The problem is that time is finite, while the list of things that would be good to know is infinite.

What is Cargo Cult Programming?

It is the blind faith that mimicking the outward form will produce the essential result.¹
From the observation that skilled engineers often know low-level details, the advice skips straight to a prescription without asking how that knowledge actually translates into outcomes. The reasoning has the same shape as believing that copying a runway will make planes arrive, and it shares another trait: there is no condition that could falsify it. Someone who succeeds without low-level knowledge becomes a person who could have done even better with it; someone who fails despite having it just has some other problem. Advice that no observation can overturn is not advice at all but a statement of belief, and a statement of belief cannot help you set priorities.

So the right question is not what you need to know, but what path that knowledge takes to pay off, because specifying the path should change the shape of the prescription.

Let's think about it using the Queue, the first data structure most people learn.

Contract and Cost Model: Guarantees FIFO ordering
                         Enqueue and dequeue are constant-time operations
                         What happens when it is full: does it block or drop?
                         Is it safe for multiple threads to access concurrently?

Implementation :   Ring Buffer or linked list?
                   How does it copy data when resizing?
                   Does it use locks or Compare-and-Swap (CAS)?

The Contract and Cost Model is about when to use it; the layer below is what you need when you are building it.

What application-layer engineers do all day is make decisions, not manufacture things. Knowing the upper layer is therefore the knowledge that matters, and not knowing the implementation below is not a deficiency.

The Contract and Cost Model here is not a summary of the implementation. The upper layer is complete knowledge in its own right, and it can be precise without the lower layer.

The fact that an operation runs in constant time is not something you can only learn by writing a Ring Buffer yourself; a simple benchmark reveals it just as well. The advice to "know the fundamentals" keeps conflating these two things. Blurring them treats the knowledge needed for decisions as a byproduct of the ability to build things, and at that moment the order of learning gets inverted.

Abstraction Is Capital

Standard libraries and frameworks are closer to costs already paid by others. Sorting, hash tables, TLS handshakes, and schedulers are the accumulated result of decades of validation and optimization. Reimplementing them is not learning; it is capital destruction.

I don't think capital destruction is unconditionally bad, because tearing down inherited foundations and building something new is not a bad thing. It is also one of the easiest ways to gain recognition. The issue is that it only makes sense as a declaration that you intend to move into that library layer yourself.

ACM degrees and curricula, and computer science academia, genuinely require these skills. And if you work in the infrastructure layer of a large company, they will be necessary skills. The problem is that this path is not every programmer's path.

This is precisely why Brooks distinguished Essential Complexity from Accidental Complexity²: what better tooling can reduce is Accidental Complexity, while Essential Complexity, which comes from the problem itself, does not shrink.
A designer's cognitive resources belong directed at the essential.
Whether to introduce a message queue or close everything in a single transaction under this traffic and this domain is a judgment no one else will make for you, whereas writing a queue by hand is something someone has already done.

In Norman's terms, knowledge does not have to live only inside your head; it can be placed out in the world.³ Documentation, signatures, types, and search results are all knowledge placed in the world. The only things worth moving into your head are those where the cost of retrieving them from the world is too high, or where you would miss the moment to retrieve them.

On top of that, learning things in advance often simply doesn't work.

Just-in-Time Learning is not a second-best option. It is closer to how adults actually acquire tools.

This is precisely what Carroll observed in the 1980s.⁴

People do not read structured textbooks in order. They try something immediately, read only at the point where they get stuck, and skip explanations that don't connect to what they were trying to do. Rather than treating this as a bad habit and making the textbook thicker, Carroll found it worked better to lower the cost of starting, attach learning to real tasks, and include a path for recovering from failure.

Going one step further, upfront explanation can actually be harmful. Detailed guidance that helps beginners degrades performance for people who already know the material.⁵ Reconciling an existing mental model with the explanation in front of you is itself a cognitive load. Low-level explanations pushed in before the learner is ready are therefore not retained, and worse, they crowd out what the learner should have been learning at that moment.

When freshmen are taught the OS first, this is usually what happens, because there is no place yet for that knowledge to attach.

The problem is that this follows the educational path of the early computing era unchanged. It is nothing more than path dependence. People who have no intention of becoming kernel or OS developers end up following the path of those who built Linux and made careers writing operating systems.

Weaknesses of Just-in-Time Learning

Of course, this strategy has weaknesses too.

To search, you need a search term, a keyword. A keyword only appears when the symptom names its own cause, and symptoms usually do not.

Suppose you encounter an error while programming.

Symptoms that name themselves: NullReferenceException
                               connection refused
                               Compile error CS8618
                               Searchable right away

Symptoms that don`t name themselves: Usually 40 ms, but occasionally 900 ms
                                     Responses bunching up with latency only under high load
                                     Works in staging but not in production
                                     Knowing what to search for is part of the answer

The right side of the diagram, where the symptom doesn't name itself, is the weakness of Just-in-Time Learning. Whether the cause is GC pause time, connection pool exhaustion, cache line sharing, or a missing index hit must already be among your candidates before a search can begin. Without a candidate list, without keywords, you stall in front of the search box.

There is an old measurement of why this is hard. When people are asked to independently name the same object, the probability that two people choose the same word does not exceed 20 percent.⁶

If you don't know a name in advance, you cannot reach the material indexed under that name. Even as search engines improve, this gap shrinks but never disappears.

But whether this is efficient is a secondary concern. Preparing for 10 possible problems in order to guard against 1 is, in practice, nearly impossible.

Therefore, the actual use of low-level knowledge is not so much solving problems as surfacing candidates, and that is enough.

Acquiring the 'Exact Noun' to type into a search box when you encounter a strange phenomenon while programming: that is the one complete reason an application-layer engineer should invest in low-level knowledge. Simply knowing the name of a phenomenon rather than having the ability to implement it collapses the cost of debugging from 10 to 1.

What you need is not the ability to implement a GC, but for GC to appear on your candidate list when you see a strange tail in your response-time distribution. You only need to survey the layer just below wherever you are programming.

Even so, why do they fabricate the illusion of "real programming"?

Because they are respected.
The work of the pioneers who built operating systems, compilers, and databases from the ground up is overwhelmingly impressive and deserves every bit of that respect.

But the community has twisted this "respect" into a vast 'Symbolic Capital' built on the premise that the closer you are to the machine, the more superior you are.

The problem is that this Symbolic Capital does not map to the real business value it generates in the market, which is a separate question. The sheer "difficulty" of allocating memory directly and manipulating pointers is worn like a badge of honor, ranked above the ability to solve business problems at the application layer and generate revenue. The means (low-level control) and the end (value creation) have been inverted.

Engineers of the early era had no choice but to burn low-level knowledge into their brains in order to survive inside leaky abstractions. Their survival stories became legends and soon hardened into the community's "rite of passage." This is a kind of religiosity and intellectual vanity. It can also be read as a psychology of seeking compensation for the enormous learning costs they once paid, by disparaging those who leverage the excellent abstractions (capital) others have built to construct business logic (Application) quickly. The illusion of "real programming" is, in the end, nothing more than old incumbents pulling up the ladder behind them.

Their kind of programming is programming, but so is the kind that trusts others' work, builds on top of the application layer, and delivers satisfaction to users.

You need to play a different game. The moment you play their game, unless you have exceptional talent, you cannot compete with developers who have 20 or 30 years on you.

Why would you want to play on a tilted field?

This is the Top-down survival strategy an application-layer engineer should adopt.
The idea that you must master every foundational principle before you can create anything is an outdated academic fantasy.

In practice, we first wield large black boxes sitting on top of frameworks or engines and produce results. When we hit the limits of those black boxes and a crack in the abstraction becomes visible, that is the moment to dive deep through that crack into the layer below (the renderer, the graphics API).

Not knowing everything in advance, but going as deep as needed, exactly when the need arises: Just-in-Time Deep Dive.

You must recognize that time is finite.

Breaking Free from Cargo Cult Programming

Claiming that low-level knowledge is useless is itself Cargo Cult Programming. What matters is something different: deciding in which domain you want to make a name for yourself and acquiring the skills appropriate to that layer.

For someone building a queue, Ring Buffers and Compare-and-Swap are the core work; for someone building a compiler, the Application Binary Interface and Memory Model are applied knowledge. For someone building a renderer, the GPU Pipeline is not the bottom of some abstraction stack but the working surface they deal with every day. The notion of "high" or "low" knowledge is not absolute; it shifts depending on the layer you occupy.

The problem lies in turning knowledge specific to one layer into a universal rite of passage that every programmer must go through.

There is infinite material to learn but finite time. Learning therefore requires a selection criterion.

Is this knowledge directly needed for a decision I face right now?
Does it give a precise name to a phenomenon I encounter repeatedly?
Does it let me descend one layer when the abstraction I'm using breaks down?
Or does it merely make me look foundational in the community?

If the answer is one of the first three, it is worth learning. If the answer is the last one, it is closer to symbolic consumption than a technical investment.

If application-layer engineers needed to conquer every lower layer, would the countless websites and applications that exist today ever have been born? Ordinarily, use well-crafted abstractions as capital; when something strange appears, recall the precise noun for that phenomenon; and be able to go only as far down as needed. That is sufficient.

This is not an attitude that glorifies ignorance. On the contrary, it is an attitude that acknowledges your own ignorance and the limits of your time, then consciously chooses where to go deep. That is the attitude a programmer should have.

The foundation of programming is not how close to the machine you work. It is knowing what decisions you must make in the layer you own, and knowing which boundary to suspect when those decisions fail.

So, first decide in which domain you want to make a name for yourself. Whether it is operating systems, databases, game engines, or application products, the fundamentals you need to learn change accordingly.

You do not become a programmer by descending all the way to the bottom. It is enough to create value in the layer where you stand, and to be able to descend precisely when the bottom comes into view.

It is enough not to let others' judgments determine your worth.