# We Gave AI Language. Maybe It Needs Machine Code. ## Toward Cognitive Opcodes, Semantic Gates, and a Machine-Native Runtime for AI **Concept paper — September 2026** **Arjen Roos** ### Abstract Large language models have become remarkably capable general-purpose components, but contemporary agent systems frequently use the same expensive primitive for many qualitatively different operations: autoregressive language generation. Reasoning, routing, relevance assessment, tool selection, risk assessment, progress tracking, memory access, and inter-agent coordination are often serialized through text even when the consumer is another machine. This paper proposes a systems hypothesis: AI may benefit from a machine-facing computational layer between neural representations and human language. That layer would expose a small vocabulary of composable **cognitive opcodes** — operations such as OBSERVE, RELEVANT, RECOGNIZE, ABSTRACT, RECALL, PREDICT, VERIFY, SELECT, ACT, and ESCALATE — whose outputs are typed state, probability distributions, references, or latent representations rather than prose. Networks of such operations could run in parallel around shared state, allocate compute conditionally, and invoke a general language reasoner only when necessary. The individual ingredients are not new. Cognitive architectures, blackboard systems, modular neural networks, hierarchical reinforcement learning, learned program interpreters, sparse expert routing, tool-using language models, latent reasoning, and programmable LM pipelines all provide important prior art. Particularly close is Soar's *chunking*: expensive subgoal problem solving can be compiled into a production that fires directly in future similar situations. Recent System One models such as TypeSafe AI's Jev add a timely ingredient: fast machine-facing typed probabilistic decisions rather than generated strings. The proposed contribution is therefore not a claim to have invented modular cognition. It is a synthesis and a testable architectural framing: treat learned semantic decisions as something analogous to an instruction set; treat structured or latent state as an intermediate representation; treat the harness as a cognitive runtime; and treat repeated expensive reasoning as a candidate for governed compilation into cheaper reusable paths. The central empirical question is simple: **can an agent preserve task performance while substantially reducing autoregressive reasoning and coordination overhead?** --- ## 1. The Strange Computer We Have Built Imagine a conventional processor that has to narrate its operations in English. Instead of comparing two registers and branching, it produces a paragraph explaining that one value appears larger and that execution should therefore continue elsewhere. Such a machine might be understandable, but it would be an extraordinary way to implement routine control flow. Many contemporary AI agents nevertheless behave in an analogous way. A request such as *review this repository and look for bugs* may cause a model to restate the request, describe a plan, inspect a tool result, narrate what the tool result means, state what it will do next, and repeat. Some of that language reflects real reasoning. Some is coordination overhead: converting machine state into prose so that the same or another model can reconstruct machine state from prose a moment later. A more direct task representation might be: ```text GOAL: BUG_AUDIT TARGET: CURRENT_REPOSITORY MODE: INSPECT OUTPUT_LANGUAGE: NL MUTATION: FALSE ``` The semantic transformation is valuable. The English serialization is not necessarily valuable. This motivates a distinction between **language as an interface** and **language as an internal computational protocol**. The first is indispensable for human interaction. The second should be treated as an engineering choice, not an axiom. ![From Logic Gates to Cognitive Opcodes](images/01_computing_stack.png) ## 2. Natural Language as a Universal Bus The success of next-token prediction has made one primitive extraordinarily general: ```text GENERATE_NEXT_TOKEN(context) ``` With enough scale, it can write, classify, plan, call tools, estimate risk, route requests, summarize memory, judge relevance, and decide whether a task is finished. This universality is one of the great achievements of modern AI. But a universal primitive can also hide architectural inefficiency. Agent loops often look like: ```text state → LLM → reasoning text → tool → textual observation → LLM → ... ``` Every transition may require serialization, decoding, and reconstruction of state. The model is not only solving the hard parts; it is also acting as parser, scheduler, router, state tracker, protocol translator, and control plane. The proposal here is not to eliminate general language models. It is to stop assuming that every internal operation deserves an autoregressive language-generation pass. ## 3. From Semantic Decisions to Semantic Gates TypeSafe AI introduced Jev in September 2026 as a “System One” model: unstructured state in, typed probabilistic decisions out. TypeSafe says Jev uses a new architecture, parallel sampling, and Reinforcement Learning for Calibrated Decisions (RLCD), and positions it as a machine-facing alternative for structured decisions rather than string generation [1]. Those architecture and performance claims are currently vendor claims, not a complete public specification. A crucial distinction is required. A typed model can be prevented from emitting arbitrary output outside its declared schema, but it can still make an incorrect decision. Type safety removes one class of failure; it does not make semantic error impossible. What is interesting for this paper is the primitive itself: ```text STATE ↓ SEMANTIC DECISION ↓ option_A 0.71 option_B 0.24 option_C 0.05 ``` The result is immediately usable by software. No prose needs to be generated and parsed. If such decisions are cheap enough, multiple questions can be evaluated around the same state: ```text RELEVANT? 0.94 RISKY? 0.08 NOVEL? 0.76 NEEDS_REASONING? 0.31 SHOULD_CONTINUE? 0.89 ``` This motivates the term **semantic gate**: a learned component that maps state to a constrained semantic distribution that can directly influence computation. The analogy to logic gates is intentionally limited. Semantic gates are learned, probabilistic, context-dependent, and fallible. The useful similarity is composability: a simple primitive can become more interesting when embedded in a network. ## 4. Cognitive Opcodes Once semantic decisions become composable, a second analogy becomes useful: the instruction set. Traditional processors expose a comparatively small vocabulary of operations such as LOAD, STORE, ADD, COMPARE, and JUMP. Software complexity emerges from their composition and from higher abstraction layers above them. A speculative Cognitive Instruction Set might include: ```text OBSERVE ATTEND RECOGNIZE COMPARE RECALL REMEMBER FORGET ABSTRACT EXPAND PREDICT HYPOTHESIZE VERIFY ASSESS SELECT ACT ESCALATE STOP ``` These names are illustrative, not a proposed standard. The research problem is to discover which operations are sufficiently general, separable, calibratable, and composable to justify becoming primitives. ![A Cognitive Opcode](images/02_cognitive_opcode.png) One particularly important operation is **ABSTRACT**: ```text ABSTRACT(state, purpose) → task_sufficient_state ``` The same underlying world can support different task-conditioned abstractions. A security process may need permissions, trust boundaries, and attack surface. A user-facing explanation may need goals, terminology, and prior understanding. Passing the complete raw state to every downstream process is neither necessary nor desirable. ## 5. Cognitive Intermediate Representation A coding agent might currently produce: > The test failed because the expected key was 9000 but the loop only produced values through 8999. This looks like an off-by-one error, so I should inspect the loop boundary. A machine-oriented intermediate representation could instead contain: ```text OBS[test=T17,status=FAIL] EXPECT[key=9000] OBSERVED[max_key=8999] HYP[off_by_one=.97] NEXT[inspect:loop_boundary] ``` Even this notation is unnecessarily textual. A production system could use typed objects; a neural system could use embeddings or latent state; a hybrid system could use both. The important point is that **human-readable prose is not required at every internal boundary**. This resembles an intermediate representation (IR) in a compiler. Human source code is translated into forms better suited to analysis, optimization, and execution. A cognitive IR would similarly separate the human communication layer from the representation used to coordinate machine cognition. ## 6. Parallel Cognition and the Cognitive Runtime A conventional agent loop is usually depicted as a sequence. The proposed architecture is better understood as a graph around shared state. ![The Cognitive Runtime](images/03_cognitive_runtime.png) On receiving an event, different processes may evaluate intent, relevance, novelty, risk, memory matches, progress, tool requirements, uncertainty, and language in parallel. Their outputs can activate other processes. Some processes may be deterministic code; some may be learned classifiers; some may be vision encoders; some may be retrieval systems; some may be small specialist models; and one may be a large general reasoner. The harness therefore becomes more than orchestration around an LLM. It becomes a **cognitive runtime** responsible for activation, shared state, scheduling, memory, uncertainty, policy, and escalation. The graph, rather than any single model, becomes the intelligent system. ## 7. Uncertainty as a First-Class Value A conventional instruction typically produces a deterministic result. A semantic operation may produce a distribution: ```text door 0.61 window 0.31 unknown 0.08 ``` That uncertainty can itself drive control flow: ```text IF entropy > threshold: OBSERVE_AGAIN ELIF danger_probability > threshold: SAFE_POLICY ELIF confidence < threshold: ESCALATE ELSE: ACT ``` This makes a probability distribution more than metadata. It becomes part of the program state. Calibration is therefore critical. Softmax scores are not automatically calibrated probabilities. Any practical system would need held-out evaluation, reliability analysis, threshold tuning, abstention behavior, and monitoring under distribution shift. ## 8. Reasoning as a Scarce Resource Adaptive computation has a long history. Graves' Adaptive Computation Time allows recurrent networks to learn how many internal steps an input deserves [2]. Mixture-of-Experts architectures route inputs through only a subset of parameters [3], while Mixture-of-Depths allocates transformer compute non-uniformly across tokens and depth [4]. These approaches differ substantially from the proposal here, but they establish an important principle: **not every input deserves the same computation**. At the agent level, the same principle can be applied to reasoning: ```text state ├─ deterministic path ├─ specialist path ├─ retrieval / memory path └─ general reasoner, when required ``` The optimization target then changes. Instead of only asking how capable the largest reasoner can become, we can ask: **How rarely can expensive general reasoning be invoked while preserving task quality?** ## 9. System Two Can Teach System One: Cognitive JIT Compilation Suppose a router initially knows three actions: ```text SUMMARIZE REWRITE OTHER ``` A translation request arrives. `OTHER` escalates to a general reasoner, which solves the task. If translation becomes a repeated, well-understood pattern, the system can consider creating a reusable fast path: ```text SUMMARIZE REWRITE TRANSLATE OTHER ``` ![Cognitive JIT Compilation](images/04_cognitive_jit.png) This is not merely caching. A useful compiled capability should generalize across a defined class of states and should have explicit validity conditions, confidence behavior, and rollback. The idea has deep prior art. Most importantly, Soar's chunking mechanism learns a new production from subgoal problem solving so that future similar situations can produce the result directly, avoiding the previous deliberation [5]. This is strikingly close to the intuition described here. Voyager similarly accumulates executable skills that can be retrieved and reused [6]. Neural Programmer-Interpreters learn compositional programs and build higher-level behavior from lower-level subprograms [7]. DSPy compiles declarative LM programs by optimizing the prompts and weights of modular pipelines [8]. The proposed “cognitive JIT” framing therefore should not be presented as an invention without ancestry. Its narrower claim is that modern probabilistic semantic decision models may make an old cognitive-architecture idea newly practical inside contemporary multimodal agent systems: expensive general reasoning can discover candidate machine-facing routes, while a governed runtime validates, promotes, monitors, and rolls them back. ## 10. Conversation Is Also Perception Perception is not limited to cameras and microphones. Conversation itself produces observations about the task and the other participant. For example, before adding a caveat, a system could estimate: ```text user_already_understands_uncertainty 0.98 new_information_in_disclaimer 0.03 disclaimer_needed 0.04 ``` This is not a proposal for hidden psychological profiling. It is ordinary task state: what has already been established, what remains unresolved, which referents are active, and which assumptions are shared. An LLM context window is a token container. Conversational context is state. Conflating the two forces the model to repeatedly reconstruct state from text history. A cognitive runtime would instead maintain explicit or latent representations of current goals, entities, assumptions, unresolved questions, progress, and uncertainty, and render them into language only when communication requires it. ## 11. Memory Should Not Always Return Prose Retrieval-augmented systems commonly store text and retrieve text back into an LLM context window. This is effective, but it need not be the only memory interface. A memory operation could return an entity, relationship, embedding, previous outcome, probability distribution, state transition, executable skill, or capability reference. A machine should not have to turn its own memory into an essay merely so another machine component can parse it again. This idea has historical parallels in blackboard architectures, where specialized knowledge sources cooperate through shared state [9], and in cognitive architectures such as Soar, which distinguish working memory, procedural knowledge, semantic memory, and episodic memory [10]. ## 12. Prior Art: What Is Actually New Here? A credible version of this proposal must be explicit about its ancestry. Nearly every individual component has substantial prior art. ### 12.1 Cognitive architectures and blackboard systems Blackboard architectures predate modern neural networks as an architectural fashion. Hayes-Roth's 1985 blackboard control architecture addressed the question of which potential action an AI system should perform at each point in problem solving, using a shared structure on which multiple knowledge sources could operate [9]. Soar was introduced as an architecture for general intelligence with working memory, operators, production rules, subgoals, and learning [10]. Its chunking mechanism is especially relevant because it compiles subgoal problem solving into productions that can fire directly in later situations [5]. This means the broad claims “intelligence can emerge from specialized processes around shared state” and “deliberation can be compiled into faster procedural behavior” are emphatically **not new**. ### 12.2 Modular and compositional neural computation Neural Module Networks dynamically compose reusable neural modules according to the structure of a question [11]. Neural Programmer-Interpreters learn to represent and execute compositional programs, with persistent program memory and reusable subprograms [7]. These are strong precedents for learned computation assembled from smaller operations. The proposed cognitive-opcode framing differs mainly in target level: it is intended as a heterogeneous systems interface spanning learned decision models, tools, memory, deterministic code, and general reasoners rather than as a single end-to-end neural architecture. ### 12.3 Hierarchical reinforcement learning and temporal abstraction The options framework in hierarchical reinforcement learning formalizes temporally extended actions with initiation conditions, internal policies, and termination conditions [12]. This is directly relevant to the idea that higher-level “instructions” can encapsulate lower-level behavior. A cognitive opcode should therefore not be confused with an entirely new concept of abstraction over actions. ### 12.4 Sparse routing and conditional compute Mixture-of-Experts and Switch Transformers demonstrate learned routing to subsets of parameters [3]. Adaptive Computation Time and Mixture-of-Depths show that compute can be allocated according to input difficulty or position [2,4]. These approaches operate largely inside models, whereas the cognitive-runtime proposal operates across heterogeneous components, but both share the principle of conditional computation. ### 12.5 Neuro-symbolic and modular LLM systems MRKL explicitly argues for a systems approach combining large language models with discrete reasoning, external knowledge, and specialized modules [13]. ReAct interleaves reasoning and actions so that external observations can update subsequent reasoning [14]. Toolformer trains a model to decide which external APIs to call, when to call them, and how to use their results [15]. These works substantially overlap with the claim that the LLM should be one component inside a larger system. ### 12.6 Programming and compiling LM pipelines LMQL treats interaction with LLMs as a programming-language problem with types, constraints, control flow, and optimized inference [16]. DSPy explicitly describes itself as programming rather than prompting foundation models and compiles declarative modules into optimized LM pipelines [8]. These are important precedents for the idea that agent behavior deserves a programming model and compiler rather than ad-hoc prompt strings. The difference proposed here is that the *internal vocabulary itself* should increasingly consist of machine-facing semantic operations and state transitions, not merely better orchestration of text-generating modules. ### 12.7 Skills that accumulate through experience Voyager maintains an ever-growing library of executable skills, retrieves them for new tasks, and refines programs using environmental feedback [6]. This is a close precedent for reusable capability acquisition. The cognitive-JIT proposal adds an emphasis on promotion governance, probabilistic decision routes, and replacing repeated deliberation with cheaper learned or deterministic paths. ### 12.8 Reasoning outside natural language Coconut is particularly important prior art. It argues explicitly that language space may not be optimal for reasoning and feeds the model's hidden state back as the next input embedding rather than decoding every intermediate step into words [17]. Its reported experiments show that continuous thoughts can encode multiple possible reasoning steps and can outperform chain-of-thought on some planning tasks with fewer thinking tokens. Pause-token work [18] and Quiet-STaR [19] also explore computation that is not identical to ordinary visible answer generation. Therefore, the claim **“AI should not have to think in natural language” is not novel**. ### 12.9 What remains distinctive in this proposal After accounting for this prior art, the potentially distinctive contribution is narrower but, in our view, more defensible: 1. **Instruction-set framing.** Treat machine-facing learned semantic decisions as an explicit analogue of an ISA: a small, typed vocabulary intended for composition across heterogeneous cognitive components. 2. **Probabilistic control values.** Make calibrated distributions, confidence, entropy, and abstention first-class values in the runtime rather than incidental model metadata. 3. **Cognitive IR.** Separate human language from the intermediate representation used for state, goals, hypotheses, evidence, memory references, and control flow. 4. **Heterogeneous cognitive runtime.** Compose deterministic code, retrieval, small decision models, perception models, specialists, and large reasoners in one event-driven graph around shared state. 5. **Governed compilation.** Treat repeated expensive reasoning as evidence for a candidate reusable route, but require testing, calibration, promotion, monitoring, and rollback before it becomes a fast path. 6. **A falsifiable efficiency objective.** Evaluate the architecture not by whether it sounds cognitively plausible, but by whether it preserves task performance with less autoregressive computation. This is a synthesis, not a claim that its ingredients appeared nowhere before. ## 13. A Possible AI Computing Stack The architecture can be summarized as a stack: ```text LEVEL 5 Human language “Review this repository for bugs.” LEVEL 4 Task representation AUDIT(repository) LEVEL 3 Cognitive program OBSERVE → RECOGNIZE → HYPOTHESIZE → VERIFY LEVEL 2 Cognitive opcodes typed semantic operations + uncertainty LEVEL 1 Learned / deterministic primitives decision models • retrieval • tools • policies LEVEL 0 Neural / conventional computation tensors • attention • kernels • CPU/GPU code ``` The historical analogy is useful precisely because it is imperfect. Computing advanced not only through better transistors, but through instruction sets, compilers, operating systems, runtimes, programming languages, and software architecture. Foundation models may likewise be necessary but insufficient building blocks for efficient intelligent systems. ## 14. A Minimal Falsifiable Experiment The proposal does not require a new foundation model to test. Take a coding agent and a fixed suite of repository tasks. Run two conditions. **Baseline:** a conventional reasoner/tool loop. **Experimental:** insert a machine-facing state layer and cheap semantic decisions before and after expensive reasoning calls. Let deterministic code or specialists handle cases they can resolve; escalate only when uncertainty, novelty, or policy requires the general reasoner. ![A Minimal Falsifiable Experiment](images/05_falsifiable_experiment.png) Measure at minimum: - task success and regression rate; - number of expensive model calls; - generated reasoning tokens; - total input tokens; - wall-clock latency; - tool calls; - repeated state narration or duplicated reasoning; - calibration of routing and abstention decisions; - energy or compute proxies where available. The key comparison is not “does the new architecture use fewer tokens?” It is: **At matched task performance, does it require materially less expensive autoregressive computation?** If the answer is no, the architecture has failed to justify its additional complexity. If yes, then some portion of what is currently called reasoning may actually be coordination overhead caused by the interface we chose. ## 15. Research Questions A serious implementation raises difficult questions: - What is the smallest useful cognitive instruction set? - Which primitives should be universal and which should be domain-specific? - How are opcodes versioned and typed? - How do probability distributions compose without destroying calibration? - How should conflicting parallel processes be reconciled? - When is structured state preferable to latent state? - How is provenance retained when abstractions omit information? - How does a runtime detect distribution shift? - When should an observation trigger another observation rather than reasoning? - How should a newly discovered fast path be tested before promotion? - Can cognitive programs themselves be learned? - Can a runtime dynamically create new nodes and edges without becoming unstable? - When does coordination overhead exceed the savings from specialization? These are engineering and empirical questions, not questions that can be resolved by analogy alone. ## 16. Conclusion Large language models gave machines an extraordinarily powerful interface to human knowledge and communication. Their flexibility made language generation the default solution to problems far beyond communication. That success may have hidden a missing layer. Between neural computation and human language there may be room for machine-facing semantic computation: typed decisions, uncertainty, state transitions, memory references, learned specialists, and reusable procedures composed by a cognitive runtime. The historical precedents are substantial. Blackboard systems already organized specialized processes around shared state. Soar already compiled deliberation into faster productions. Hierarchical reinforcement learning already created temporally extended actions. Neural program architectures already composed reusable programs. Modern sparse models already route compute selectively. Tool-using agents already combine language models with external modules. Coconut already demonstrates that reasoning need not be serialized into language at every step. The opportunity is therefore not to rediscover those ideas separately. It is to combine them under a machine-native systems abstraction appropriate to today's models. Perhaps the next important AI primitive is not a larger conversational model. Perhaps it is a small semantic operation that can be composed, measured, calibrated, and executed without speaking first. We gave AI language. **Maybe it needs machine code.** --- ## References [1] TypeSafe AI. “Introducing System One Models & Jev.” 15 Sep 2026. https://typesafe.ai/blog/introducing-system-one-models-and-jev [2] Alex Graves. “Adaptive Computation Time for Recurrent Neural Networks.” 2016. https://arxiv.org/abs/1603.08983 [3] William Fedus, Barret Zoph, Noam Shazeer. “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.” JMLR, 2022. https://arxiv.org/abs/2101.03961 [4] David Raposo et al. “Mixture-of-Depths: Dynamically allocating compute in transformer-based language models.” 2024. https://arxiv.org/abs/2404.02258 [5] Soar Project. “Procedural Knowledge Learning — Chunking.” https://soar.eecs.umich.edu/soar_manual/04_ProceduralKnowledgeLearning/ [6] Guanzhi Wang et al. “Voyager: An Open-Ended Embodied Agent with Large Language Models.” 2023. https://arxiv.org/abs/2305.16291 [7] Scott Reed, Nando de Freitas. “Neural Programmer-Interpreters.” 2015. https://arxiv.org/abs/1511.06279 [8] Omar Khattab et al. “DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines.” 2023. https://arxiv.org/abs/2310.03714 [9] Barbara Hayes-Roth. “A blackboard architecture for control.” Artificial Intelligence 26(3), 1985. https://doi.org/10.1016/0004-3702(85)90063-3 [10] John E. Laird, Allen Newell, Paul S. Rosenbloom. “SOAR: An architecture for general intelligence.” Artificial Intelligence 33(1), 1987. https://doi.org/10.1016/0004-3702(87)90050-6 [11] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein. “Neural Module Networks.” CVPR, 2016. https://openaccess.thecvf.com/content_cvpr_2016/html/Andreas_Neural_Module_Networks_CVPR_2016_paper.html [12] Richard S. Sutton, Doina Precup, Satinder Singh. “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning.” Artificial Intelligence 112, 1999. https://doi.org/10.1016/S0004-3702(99)00052-1 [13] Ehud Karpas et al. “MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning.” 2022. https://arxiv.org/abs/2205.00445 [14] Shunyu Yao et al. “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR, 2023. https://arxiv.org/abs/2210.03629 [15] Timo Schick et al. “Toolformer: Language Models Can Teach Themselves to Use Tools.” 2023. https://arxiv.org/abs/2302.04761 [16] Luca Beurer-Kellner et al. “Prompting Is Programming: A Query Language for Large Language Models.” / LMQL project. https://lmql.ai/ [17] Shibo Hao et al. “Training Large Language Models to Reason in a Continuous Latent Space.” 2024; ICLR 2025 Workshop. https://arxiv.org/abs/2412.06769 [18] Sachin Goyal et al. “Think before you speak: Training Language Models With Pause Tokens.” 2023. https://arxiv.org/abs/2310.02226 [19] Eric Zelikman et al. “Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking.” 2024. https://arxiv.org/abs/2403.09629 ### Note on scope and novelty This document is a concept paper and architectural hypothesis, not a report of completed experimental validation. The literature review is intended to identify the strongest conceptual precedents, not to claim exhaustive coverage of every modular-agent or cognitive-architecture publication. The proposed terminology — *semantic gates*, *cognitive opcodes*, *cognitive IR*, *cognitive runtime*, and *cognitive JIT compilation* — is used as a unifying systems vocabulary. Similar phrases may exist elsewhere; novelty should ultimately be judged by the precise architecture and empirical results rather than terminology.