So the way I write code is that, my understanding is local. Okay, we need a function that does this (high level). It'll call these functions to do that. And then I just continue until there's nothing left to write and the thing works (after a few rounds of debugging).

I understand each piece and what it talks to. But I can't hold them all in my mind at once, because there's too many pieces. (I think chunking helps here, but it seems to require a certain level of fluency with the entire codebase that I'm not sure it's feasible to hit with anything past a certain line count. I am working on this new memory software though...)

The transformer on the other hand, just loads it into context (they can do about 10K LoC these days without performance degradation), cross references everything against everything (that's how the transformer works! That's why they're so expensive) and just tells me what talks to what, what the full chain is, and also btw you have 3 bugs you didn't notice because they involve how distant parts of the chains interact, you're welcome!

I've been looking for ways to build up that mental model. The Feynman technique seemed like a good place to start. I did it on a section of my codebase. It took half an hour of poking around to connect all the pieces. The transformer was able to do it instantly.

I'm not sure if there was added value to me poking around manually or if those 30 minutes would have been better spent just memorizing what it told me.

(After verification of course! To clarify, I don't think they're infallible, but their perception is broader than ours due to how they're structured, and I'm learning to utilize that more effectively.)

Also, in the absence of that costly verification, the model my Feynman technique produced turned out to be wrong (though it sounded correct!). So I'm leaning in the direction of, the way to actually verify your mental model is to make a modification to the codebase. Make reality push back!

On some projects/subprojects I do build an explicit mental model beforehand, and then I do generally remember it pretty well, at least for a while. Others take a more iterative approach to the design. (I'm on the 5th damn iteration of my netcode right now.)

So there's two distinct issues here, the model building process and the human forgetting curve.

The short leash AI coding method for beating Fable