EXIST Stipendium Application & Beyond
Build a research and engineering platform around a simple thesis: the next step is not just bigger models, but systems that maximize
The apparent contradiction is mostly a change of variables. Paul Graham optimizes for surface area of opportunity:
Jensen Huang warns about the denominator: modern frontier AI has . Going to Silicon Valley can still maximize optionality without pretending that the right first move is to start a conventional model company.
The strategy is not startup first but thesis first. The neglected space is adaptation, control, and physical efficiency:
Use the EXIST stipendium as a research launchpad. Validate the core equations, build prototypes, then decide whether the output should become a lab, a company, or a platform.
The core thesis is that intelligence should be treated as a coupled system of learning dynamics, control, and physical substrate. Current AI mostly scales parameters; the proposed direction scales adaptation quality per joule:
That changes the optimization target from brute-force capability to a more durable objective function:
Static deployment is a dead end for many real environments. The model should update online:
with constraints that preserve stability and avoid catastrophic drift.
LoRA factorizes adaptation into low-rank structure with . This makes continual adaptation cheaper, more local, and easier to gate.
The first-class metric becomes:
The ML side provides representation learning; neuroscience provides sparse, event-driven computation; control theory provides stability guarantees. A minimal state-space framing:
A useful cognitive architecture is not just expressive; it must remain controllable under perturbation. The target is high plasticity but bounded instability — maximize learning subject to:
Software abstractions eventually hit hardware limits. Conventional digital switching pays:
Meaningful gains require attacking , , , or the substrate itself.
Computation is thermodynamic bookkeeping. At minimum:
Irreversible bit operations are bounded by Landauer:
Thermal noise scales with temperature:
Lower temperature simultaneously reduces minimum dissipation, suppresses noise, and enables superconducting substrates. Cryogenic storage can therefore serve as both an energy buffer and a compute enabler.
Frontier labs dominate the regime described by empirical scaling laws:
That regime rewards capital intensity. The opportunity is in architectures where:
Every strategic move—whether training a model or funding a lab—has an entropic signature. The goal is to maximize against .
1. Brute-Force Pretraining (LLMs)
2. Biological Learning (The Target)
3. Venture Capital & Resource Allocation
Every serious research direction eventually becomes a religion, so let's be explicit about the dogma.
First Commandment: Thou shalt not worship parameter count. is not a strategy.
Second Commandment: Energy is not a footnote. Every system must be able to answer honestly.
Third Commandment: Stability is sacred. A system that learns but cannot be controlled is not an agent — it is a hazard. All weights shall satisfy .
Fourth Commandment: Physics is not optional. Landauer sets the floor: . No architecture escapes thermodynamics.
Fifth Commandment: Ship or it didn't happen. Equations without implementations are theology. Implementations without equations are magic. The goal is engineering.
The congregation meets wherever there is a good compiler, liquid nitrogen, and an open research question.
The bottleneck is not just how much RAM you have — it is how much energy and time it costs to move bits between layers of the memory hierarchy. Every cache miss, every page fault, every swap read is a thermodynamic event.
Landauer's principle sets the absolute minimum energy to erase one bit at temperature :
Modern DRAM refresh operations erase and rewrite billions of bits per second. The gap between Landauer's floor and actual DRAM energy consumption is roughly . That gap is the engineering opportunity.
The memory hierarchy can be modeled as a sequence of energy-latency tradeoffs. For a cache level with access energy and latency :
where is the probability of accessing level (miss rate cascade). Optimizing this is equivalent to minimizing entropy production across the hierarchy.
Why swap hurts: Disk/SSD swap increases and by – relative to DRAM. Compressed RAM swap (zram) keeps the access in DRAM at the cost of CPU cycles for compression — a worthwhile trade when CPU is underutilized.
The cryogenic path: At (liquid helium), Landauer's floor drops to . Superconducting logic (RSFQ — Rapid Single Flux Quantum) operates near this limit. The theoretical energy per operation:
still above Landauer but below CMOS. The path from 8GB DRAM to post-silicon memory runs through this physics.
Immediate levers (no new hardware):
zram compressed swap — effectively multiplies usable RAM by 2–3× at CPU costvm.swappiness=10 to prefer RAM over diskvm.page-cluster=0 to reduce readahead on swapearlyoom to kill runaway processes before OOM freezes the systemLiving systems are not thermodynamic anomalies — they are exceptionally well-engineered dissipative structures. The key quantity is entropy production rate . A living system maintains low internal entropy by exporting entropy to its environment at a rate that satisfies:
This is Prigogine's condition for a dissipative structure. Life is not magic — it is a locally negentropic process sustained by global entropy increase.
For computation, reversible operations cost no entropy in principle (Landauer: only erasure costs ). Irreversible operations are thermodynamically lossy. Biological neurons operate closer to the reversible limit than CMOS logic:
The implication: building artificial life that is robust and energy-efficient requires understanding which computations must be irreversible (decisions, memory writes, signal amplification) and which can be made reversible or adiabatic.
For a reversible gate operating at rate with residual dissipation :
The engineering goal is to push per irreversible bit operation.
Neural Cellular Automata (NCA) are among the most honest models of life we have. Each cell updates according to a learned local rule applied to its neighborhood :
What makes NCAs profound is robustness: trained NCAs regenerate target morphologies after arbitrary damage. This is precisely what biological development does. The robustness emerges not from central control but from local rules applied in parallel — a distributed computation with no single point of failure.
The perception step uses learned filters (analogous to biological receptive fields):
The update rule is then a small MLP applied per cell:
where is a stochastic update mask that forces each cell to function even when its neighbors are silent — a direct model of biological noise tolerance.
Why this matters for funding: NCAs demonstrate a concrete path from toy models to real robustness. The same principles — local rules, no central controller, graceful degradation — apply to distributed AI systems, fault-tolerant hardware, and self-repairing biological interfaces. This is not academic. The measurable deliverables are:
Connection to artificial life: A sufficiently general NCA with a metabolism term — where cells consume a resource and die when — becomes a minimal model of a living system satisfying:
That is life. The question is whether we can engineer the that sustains it indefinitely.