A bit back, I read Anthropic & Rick Rubin’s “The Way of Code,” and specifically noted: “Free from intellect.” I felt a deep aversion to the idea of shallowing understanding. Most of all, it also felt shameful for a company as noble as (in my view) Anthropic—Given that what they sell is a product that helps you “do just that”.

I have recently gotten a perspective on this, boiled down to two new premises. One birthed from another.

1. At the time, it wasn’t glaringly obvious to me; I was afraid I would lose the ability to understand the thing, and I would be demotivated to keep going. And as models hit their ceiling, I get capped at what they can do, but I forgot that as models increasingly become more agentic, even with fixed input, you should expect to get better results.

2. Science requires holding many parameters in your head; it’s a calibrated combinatorial optimization problem: you rely on working models and hypotheses, which hopefully lead you to find the new working models, run experiments, integrate results, and make a strong inference about the world. The complexity is increasingly exceeding what one mind can hold.

Sure, Humans are very big state machines that can compress very long-running sequences, but surely there is an information-theoretic limitation to that; and models don't have this issue? models can be spun up with thousands of connected processors to process many human lifetimes’ worth of knowledge to infer something about the world.

Example in biology:

Proteins are fundamental building blocks of any living organism / process, if one were to try to work on biology, one must understand how these things work and interact with each other.

Lets say we have 20 different kinds of amino acids used in the process of making proteins. Even at just a sequence lengths of 4 to 300 amino-acids (some span thousands), we can produce 2.1 × 10^390 possible proteins

We can even further reduce our search space to molecules that are on a relatively lower end of the energy state, and further reduce to sequences that can reliable fold stably, as those are generally less reactive and conducive to forming complex interactions needed for evolution to select for them.

This is quite a large space, how would a human possibly deal with all these intricate interactions? Geoffrey Hinton’s “Humans are big analogy machines” is I think quite generous…

I think for quite a while now, we have been building analogies and trying to find some repeatable pattern for them across maybe just one subspace of a much larger interaction tree. We built classes for these interactions, and further we built classes for individual components that are able to have these interactions.

This I think is where models can come in. Models allow us to build representations from incomplete data. Representations that capture vastly larger surface of interactions than one human can hold. As we gather new data, everything that has come before can neatly fold to make everything fit together (through retraining ofc). AlphaFold is a very good example of this, as it allowed researchers to quickly reduce search space of viable protein structures and interactions,

As the protein example shows, The epistemology of science has become deeply complicated. And a question to be next asked would be where information fitting actually ends…

I don’t think what all humans do is information fitting, I think perhaps we are significantly lesser for it… Humans actually did information fitting for maximizing the one objective that is not so immediately obvious to us all.

Evolution is a quirky thing, its a infinite-arrows-hit-atleast-once search algorithm, we don’t know what for. But it was able to discover agents that could collaborate, then further these groups of agents started specializing parts of the collective (neocortex) purely for processing signals from outside world, to be able to fit information better.

Humans are vastly more successful than our closest relatives (Please someone lets make a good benchmark for species success!!), and most of us would credit intelligence. But intelligence was never the point.

I will be ashamed to say this, but, “AI Models are an extension of the neocortex” if you have been relying on the neocortex’s current function as the goal of your existence, you might not be very happy about it!