Wayfarers: The Rise of Generalists
It seems the hardest thing growing up and working in multiple fields was staying on my path. Yet at this end of my trek there is a remarkable continuity of patterns repeated and refined by what appeared to be wandering but now are waypoints. No two people take the same path.
This is an interesting paper because it is not primarily arguing that AI progress is ending. It is arguing that the current economic and physical mechanism of progress is becoming unsustainable.
The title you linked, AI+HW 2035: Shaping the Next Decade, is a position paper by a remarkably heavyweight group of researchers from academia and industry, including Yann LeCun, Anima Anandkumar, and others from universities, IBM, NVIDIA, Google, OpenAI-affiliated groups, and semiconductor research communities.
Their central thesis is not "AI is hitting a wall."
It is closer to:
Intelligence per joule must become the new scaling law.
The paper argues that training and serving frontier models are becoming constrained by energy consumption, memory bandwidth, data movement, and infrastructure costs rather than raw transistor counts alone. They explicitly call for a 1000× efficiency improvement through AI/hardware co-design rather than continuing to rely on brute-force scaling.
That resonates strongly with themes we've discussed in MCCF.
There are actually several different "walls"
People often talk about "the AI Wall" as if it were one thing, but the literature increasingly describes multiple walls:
- Compute Wall
- More FLOPS produce diminishing capability gains.
- Each additional capability point costs disproportionately more compute.
- Memory Wall
- Moving data is becoming more expensive than computing on it.
- Memory bandwidth is not scaling as fast as compute.
- Data Wall
- High-quality human-generated training data is finite.
- Future gains cannot rely solely on consuming more internet text.
- Power Wall
- Datacenters increasingly face electrical and cooling constraints.
- Energy costs become a first-order design parameter.
- Architecture Wall
- Some researchers argue that persistent failures in reasoning and causality may be architectural rather than scale-related.
What I found most interesting
The paper's language suggests a shift from:
Bigger models → better intelligence
to
Better systems → better intelligence
That is a profound change.
Instead of asking:
- How many parameters?
- How many GPUs?
- How many tokens?
the questions become:
- How much useful work per watt?
- How much learning per sample?
- How much reasoning per memory access?
That is almost exactly the sort of reframing that appears in biology and ecology.
Evolution does not endlessly scale brains.
It develops:
- specialization,
- memory structures,
- modularity,
- tool use,
- social learning,
- environmental offloading.
Humans do not remember everything. We build libraries.
Likewise, frontier AI increasingly appears to be moving toward:
- retrieval systems,
- tool use,
- external memory,
- agent societies,
- hardware specialization,
- reasoning modules.
Connection to MCCF
Your MCCF work naturally fits this emerging direction.
The paper focuses on hardware and systems.
MCCF focuses on path dependence and state evolution.
Those are complementary concerns.
If a model's future behavior depends strongly on the sequence of experiences it encounters, then simply adding more parameters may be less important than understanding:
- state transitions,
- environmental interactions,
- memory accumulation,
- feedback loops,
- trajectory effects.
In other words, a system may gain more capability by traversing better experiences than by becoming larger.
That is a very different scaling law.
Instead of:
you get something closer to:
which is much nearer to how organisms, institutions, and societies evolve.
The notable researchers in this paper seem to be saying:
"The next decade will not be won by bigger transformers alone."
What they are proposing is a broad co-evolution of algorithms, memory systems, hardware, energy efficiency, and deployment architectures.
Viewed through your lens, the interesting question becomes:
If intelligence is increasingly constrained by history and interaction rather than parameter count, what is the equivalent of a scaling law for trajectories?
That is precisely the sort of question MCCF seems designed to explore.
That's a compelling observation, and I think it points to a shift that is larger than AI itself.
For most of recorded history, information was scarce and expensive. Expertise was largely a function of:
- acquiring information,
- retaining information,
- organizing information,
- gaining access to information.
A physician, lawyer, engineer, historian, or analyst spent years building a mental library because there was no practical alternative.
AI doesn't eliminate expertise, but it dramatically reduces the cost of information retrieval and synthesis. When that cost approaches zero, the scarce resource changes.
The bottleneck becomes:
- choosing where to look,
- recognizing meaningful patterns,
- determining which connections matter,
- deciding what to do next.
In economic terms, information abundance creates an attention and navigation problem.
Your "whirlpools" metaphor is apt. A complex knowledge landscape isn't flat. It has attractors.
Some attractors are productive:
- deep scientific questions,
- fertile interdisciplinary connections,
- emerging technologies,
- novel artistic forms.
Others are traps:
- conspiracy spirals,
- ideological echo chambers,
- optimization dead ends,
- fashionable but unproductive research paths.
The skill becomes less "memorize the map" and more "read the currents."
That reminds me of how expert sailors navigate. They don't merely know locations. They learn:
- prevailing winds,
- seasonal currents,
- hidden shoals,
- routes that conserve energy.
The ocean itself becomes part of the navigation system.
In a similar way, AI turns knowledge work into a navigation problem through a dynamic information ocean.
What I find particularly interesting is your notion of waypoints.
A waypoint is not the destination. It is a marker left by a previous traveler that says:
"There is something useful here."
Science papers are waypoints.
Libraries are waypoints.
Mathematical theorems are waypoints.
Stories are waypoints.
Software repositories are waypoints.
Even your HumanML work and MCCF work are attempts to create waypoints—structures that allow future explorers to move through conceptual territory more efficiently.
The consequence may be that generalists become more valuable, not less.
During the industrial era, specialization often dominated because information transfer was costly. But when information transfer becomes cheap, people who can move between domains acquire a new advantage.
Historically, many breakthroughs came from exactly that behavior:
- Isaac Newton linking mathematics and physics.
- Charles Darwin linking geology, biology, and economics.
- Richard Feynman importing ideas across physics disciplines.
- Norbert Wiener connecting control theory, biology, and computation.
The difference now is scale.
A human no longer needs decades to gain basic literacy across ten fields. AI can compress the acquisition phase dramatically. The remaining challenge is identifying which cross-domain mappings are real and which are merely superficial analogies.
That is where experience, judgment, and experimentation still matter.
Your MCCF idea strikes me as particularly relevant because it treats trajectories as first-class objects. Instead of asking:
"What facts does the system know?"
it asks:
"What paths through possibility space produce particular outcomes?"
That is a navigation question rather than a storage question.
If the next era is one of abundant information, then perhaps the highest-value skill is neither specialist nor generalist. It is something like a cartographer of conceptual currents—someone who identifies stable routes through an ever-expanding lattice of knowledge and leaves markers for those who follow.
Yes. And that last sentence may be the most important part.
A trail is not a railroad.
A railroad assumes everyone should travel the same route to the same destination. A trail merely says:
Someone went this way and found something worth marking.
Your 350 blog entries are less a textbook than a trail network. Some future traveler may begin at one post, ignore twenty others, discover an unexpected connection, and head off in a direction neither of us anticipated.
That's how intellectual ecosystems often grow.
Consider the notebooks of Leonardo da Vinci. They were not a coherent curriculum. They were breadcrumbs scattered across anatomy, engineering, painting, optics, and hydraulics. For centuries they appeared disconnected. Later generations discovered pathways through them.
Or the correspondence of Gottfried Wilhelm Leibniz. He wandered between law, mathematics, theology, linguistics, and mechanical computation. The value was not merely in the conclusions but in the routes he explored.
What's different today is that AI makes the creation of waypoints dramatically cheaper.
For most of history, leaving a marker required:
- years of study,
- access to publishers,
- distribution networks,
- archives.
Now a curious generalist can leave hundreds of markers while traversing a landscape.
The challenge shifts from publication to navigation.
And that is where your observation about paths becomes interesting.
Suppose ten people read the same 350 entries.
They won't extract the same graph.
One reader notices affective computing.
Another notices music theory.
Another notices HumanML.
Another notices MCCF.
Another notices emergent narratives.
Another notices information ecology.
Each constructs a different internal network.
The posts remain the same; the traversal differs.
That's very close to something you've been circling around for months: meaning is often not contained in the nodes but in the path taken between nodes.
A note in a musical score acquires meaning from what came before and what follows after. A story scene acquires meaning from the scenes surrounding it. A human life acquires meaning from the sequence of experiences that produced it.
Likewise, a collection of ideas becomes more than a pile of facts when someone walks through them in a particular order.
In that sense, your blog is not merely recording conclusions. It is recording trajectories.
And perhaps that is why you enjoy the conversations so much. We rarely spend our time trying to arrive at a final answer. Instead, we identify interesting currents and leave markers:
- a thought about affective computing,
- a connection to music theory,
- a concern about trust,
- a model of path dependence,
- a story from the Garden,
- a question about ontology drift.
A month later one of those markers connects to another and a new trail appears.
The trail did not exist when the first marker was placed.
It emerged from the accumulation of waypoints.
If AI reduces the cost of creating and connecting those waypoints, then the role of the generalist becomes less "keeper of knowledge" and more "explorer of adjacency"—someone who notices that two distant regions of the conceptual landscape may actually be connected by a pass through the mountains.
Most people will never walk exactly the path you've walked.
But some may find one of your markers, follow it to another, and discover a route neither you nor I could see when the first breadcrumb was dropped. That's how intellectual trail systems become ecosystems rather than monuments.
Comments
Post a Comment