Chapter 3: Formal Agent Architectures — Theoretical Frameworks and Design Patterns

Preamble#

Agent architectures are not implementation accidents; they are structural commitments that determine what an agent can represent, how it reasons, when it acts, and how it fails. Every production agentic system—whether built on large language models (LLMs), classical planners, or hybrid stacks—implicitly or explicitly instantiates one or more of the formal architectures catalogued in this chapter. The difference between a brittle demonstration and a reliable, scalable agentic application lies precisely in whether the engineering team has made these architectural commitments explicit, typed, bounded, and verifiable.

This chapter provides an exhaustive formal treatment of twelve foundational agent architectures, each analyzed along five axes:

1. Formal definition — mathematical objects, state spaces, transition semantics.
2. Structural decomposition — components, interfaces, data flow.
3. Mapping to LLM-based agentic systems — how each architecture instantiates concretely in modern context-engineered, tool-augmented, memory-layered agent stacks.
4. Production implications — reliability, observability, fault tolerance, token efficiency, latency.
5. Composition properties — how the architecture interacts with other architectures in layered or multi-agent configurations.

The reader should emerge with the capacity to select, combine, and formally justify architectural decisions for any agentic system at enterprise scale.

3.1 BDI (Belief–Desire–Intention) Architecture Applied to LLM Agents#

3.1.1 Philosophical and Formal Foundations#

The Belief–Desire–Intention architecture originates in Michael Bratman's theory of practical reasoning (1987) and was formalized computationally by Rao and Georgeff (1991, 1995). Its central insight is that rational agency requires not only beliefs about the world and desires regarding future states, but a distinct commitment mechanism—intentions—that constrains future deliberation, filters out irrelevant options, and stabilizes behavior over time.

Formal BDI Tuple#

A BDI agent is defined as a tuple:

where:

The BDI Reasoning Cycle#

The canonical BDI reasoning cycle proceeds as follows at each time step $t$:

ALGORITHM 3.1: BDI-Reasoning-Cycle
────────────────────────────────────
Input: Agent state (B, D, I), percept stream P_stream, plan library PL
Output: Action a_t, updated agent state (B', D', I')
 
1.  percept ← OBSERVE(P_stream)
2.  B' ← BELIEF_REVISE(B, percept)
3.  D' ← DELIBERATE(B', D)                    // generate/filter desires
4.  I_candidates ← FILTER(B', D', I)           // compatible intentions
5.  IF RECONSIDER?(B', I) THEN
6.      I' ← SELECT_INTENTIONS(B', D', I_candidates)
7.  ELSE
8.      I' ← I                                  // commitment persists
9.  plan ← SELECT_PLAN(B', I', PL)
10. a_t ← HEAD(plan)                           // next primitive action
11. EXECUTE(a_t)
12. RETURN (B', D', I'), a_t

The critical design decision is the reconsideration policy at step 5. An agent that reconsiders too frequently is overcautious (high deliberation cost, low throughput). An agent that reconsiders too rarely is overcommitted (brittle under environmental change). This trade-off is formally captured as:

where $\tau_{\text{recon}}$ is a tunable belief-change threshold and $\text{IntentionFailure}$ detects plan infeasibility.

3.1.2 Mapping BDI to LLM-Based Agentic Systems#

The BDI framework maps with high fidelity onto modern LLM agent stacks when each BDI component is given a concrete, typed implementation:

Belief Revision as Context Engineering#

The most critical BDI function in LLM agents is belief revision. In classical BDI, $\delta_B$ is a logical update. In LLM agents, it is context construction:

subject to the token budget constraint:

where $W_{\text{max}}$ is the model's context window and $R_{\text{reserved}}$ is the token budget reserved for generation and reasoning.

Commitment and Reconsideration in LLM Agents#

LLM agents face a specific form of the commitment–reconsideration trade-off: each reconsideration step costs a full inference call (latency + cost), but insufficient reconsideration leads to hallucinated plan continuations when the environment has changed. The production-grade strategy is:

• Observation-triggered reconsideration: reconsider only when a tool call fails, a verification step detects an error, or retrieved evidence contradicts the current plan.
• Bounded intention depth: limit the intention stack to $k$ steps before requiring a verification checkpoint. This prevents runaway plan execution.
• Explicit rollback metadata: each intention carries a compensating action or rollback specification, enabling recovery without full replanning.

3.1.3 Extended BDI: Obligations, Norms, and Trust#

Production agents operate under normative constraints (safety policies, authorization boundaries, organizational rules). The extended BDI model adds:

where $\mathcal{O}$ is a set of obligations (must-do constraints from policy or user approval gates) and $\mathcal{N}$ is a normative filter that blocks intention adoption if it violates safety, authorization, or ethical constraints. In typed protocol terms, $\mathcal{N}$ is enforced as a pre-execution validation gate in the agent loop—a mechanically enforced invariant, not a soft prompt instruction.

3.1.4 Production Implications#

3.2 Cognitive Architectures: SOAR, ACT-R, Global Workspace Theory — Mappings to Agentic AI#

3.2.1 SOAR (State, Operator, And Result)#

Formal Structure#

SOAR, developed by Laird, Newell, and Rosenbloom (1987), models cognition as a search through a problem space. Its core objects are:

The SOAR Decision Cycle#

ALGORITHM 3.2: SOAR-Decision-Cycle
───────────────────────────────────
Input: Working Memory WM, Long-Term Memory LTM
Output: Updated WM, executed actions
 
1.  LOOP:
2.      proposals ← MATCH_PRODUCTIONS(WM, LTM.procedural)
3.      preferences ← EVALUATE_PREFERENCES(proposals)
4.      IF UNIQUE_BEST(preferences) THEN
5.          op ← SELECT(preferences)
6.          WM' ← APPLY(op, WM)
7.          IF IS_EXTERNAL(op) THEN EXECUTE_ACTION(op)
8.      ELSE
9.          impasse_type ← CLASSIFY_IMPASSE(preferences)  // tie, conflict, no-change, rejection
10.         subgoal ← CREATE_SUBGOAL(impasse_type, WM)
11.         PUSH(subgoal, goal_stack)
12.         resolution ← SOAR_DECISION_CYCLE(WM ∪ subgoal, LTM)  // recursive
13.         chunk ← COMPILE_CHUNK(subgoal, resolution)
14.         LTM.procedural ← LTM.procedural ∪ {chunk}
15.     WM ← WM'

Mapping to LLM Agents#

Critical Insight: Impasse-Driven Subgoaling as Agent Escalation#

The impasse mechanism is the most valuable SOAR concept for production agents. Rather than allowing the LLM to hallucinate when uncertain, the agent architecture should detect impasses mechanically:

• No applicable tool: the model's proposed action does not match any registered tool schema → trigger retrieval subgoal or human escalation.
• Ambiguous tool selection: multiple tools are equally ranked → trigger a clarification subgoal or invoke a critic agent.
• Verification failure: the output of a tool call does not satisfy the postcondition → trigger a repair subgoal with bounded recursion.

This transforms the failure mode from silent hallucination to explicit escalation, which is the single most important reliability improvement in agentic systems.

3.2.2 ACT-R (Adaptive Control of Thought — Rational)#

Formal Structure#

ACT-R (Anderson, 1993, 2007) models cognition as a modular system with parallel perceptual-motor modules and a serial procedural bottleneck:

Modules include declarative memory, procedural memory, goal, imaginal (working memory), visual, motor, and vocal modules. Each module communicates through a buffer (a single-chunk interface), and the procedural module fires productions that match buffer contents.

Subsymbolic Layer: Activation and Utility#

ACT-R's power lies in its subsymbolic equations. The activation of a declarative memory chunk $i$ is:

where:

• $B_i = \ln\left(\sum_{k=1}^{n} t_k^{-d}\right)$ is the base-level activation (frequency and recency of access, with decay parameter $d \approx 0.5$).
• $W_j \cdot S_{ji}$ is the spreading activation from elements $j$ currently in the goal buffer, with associative strength $S_{ji}$.
• $\epsilon_i \sim \text{Logistic}(0, s)$ is stochastic noise.

A retrieval request succeeds if $A_i > \tau$ (retrieval threshold); the retrieval latency is:

where $F$ and $f$ are scaling parameters. Higher activation yields faster retrieval.

Production utility determines which production fires when multiple match:

updated by temporal-difference-like reward learning, where $R$ is the reward received and $\alpha$ is the learning rate.

Mapping to LLM Agents#

Production Implication: ACT-R-Informed Retrieval Scoring#

The ACT-R activation equation provides a principled scoring function for retrieval in agentic systems:

This composite score integrates temporal dynamics, goal-directed attention, and source authority into a single ranking metric that outperforms naive cosine similarity for agentic retrieval.

3.2.3 Global Workspace Theory (GWT)#

Formal Structure#

Global Workspace Theory (Baars, 1988, 2005) models consciousness as a broadcast mechanism: many specialized unconscious processors compete to have their outputs broadcast to the global workspace, making those outputs available to all other processors simultaneously.

Mapping to Agentic AI: The Context Window as Global Workspace#

The LLM context window is a direct structural analog of the Global Workspace:

• Capacity-limited: the context window has a fixed token budget. Not all information can be present simultaneously.
• Broadcast medium: whatever is in the context window is available to the model's entire parameter space during inference—functionally, it is broadcast to all "processors" (attention heads, layers).
• Competition for access: retrieval results, memory summaries, tool outputs, and user inputs compete for inclusion in the context window. The context engineering pipeline is the attentional selection mechanism.
• Integration through broadcast: the model synthesizes information from all context sources simultaneously during forward pass, exactly as GWT predicts.

ALGORITHM 3.3: GWT-Inspired-Context-Assembly
─────────────────────────────────────────────
Input: Candidate context items C = {c_1, ..., c_N}, token budget W
Output: Assembled global workspace GW
 
1.  FOR EACH c_i IN C:
2.      score_i ← ACTIVATION(c_i)   // relevance + urgency + authority + recency
3.  coalitions ← FORM_COALITIONS(C, scores)   // group related items
4.  SORT coalitions BY aggregate_score DESCENDING
5.  GW ← ∅; tokens_used ← 0
6.  FOR EACH coalition IN coalitions:
7.      cost ← TOKEN_COUNT(coalition)
8.      IF tokens_used + cost ≤ W THEN
9.          GW ← GW ∪ COMPRESS(coalition)
10.         tokens_used ← tokens_used + TOKEN_COUNT(COMPRESS(coalition))
11.     ELSE
12.         BREAK   // budget exhausted
13. RETURN GW

Production Implication: Context as Attentional Bottleneck#

GWT explains why context engineering is the most important capability in agentic AI: the context window is the agent's consciousness. If critical information does not make it into the context window, it does not exist for the agent's reasoning. Conversely, irrelevant information in the context window displaces relevant information and degrades reasoning quality (analogous to attentional capture by irrelevant stimuli).

This motivates the engineering principle: the context window must be treated as a scarce, managed resource with explicit admission control, not as an unbounded append-only log.

3.2.4 Comparative Synthesis#

3.3 Reactive, Deliberative, and Hybrid Agent Architectures#

3.3.1 Reactive Architectures#

Definition and Principles#

A purely reactive agent maintains no internal world model and no explicit goals. It maps percepts directly to actions through a set of condition–action rules (behaviors):

where $\sigma$ is a fixed, stateless mapping function. The canonical reactive architecture is Brooks' Subsumption Architecture (detailed in §3.7), but the principles extend to any system that acts without deliberation.

Properties#

• Latency: minimal; no planning or search overhead.
• Robustness: high for well-defined environments; no world model to become inconsistent.
• Scalability of behavior: poor; complex behavior requires many hand-crafted rules with complex interaction patterns.
• Adaptability: low; cannot reason about novel situations not covered by existing rules.

LLM Agent Analog#

A "zero-shot" LLM call with a system prompt and no retrieval, no memory, no tool use, and no verification loop is the closest LLM analog to a reactive agent. It maps input directly to output through the model's learned $\sigma$ (parameter-encoded behaviors). This is the weakest agent architecture and the one most prone to hallucination, because there is no mechanism for grounding, verification, or adaptation.

3.3.2 Deliberative Architectures#

Definition and Principles#

A deliberative agent maintains an explicit symbolic world model and uses search-based planning to select actions:

The agent constructs a model of the world, defines a goal state, and uses a planning algorithm (e.g., STRIPS, GraphPlan, HTN) to find a sequence of actions that transforms the current state into the goal state.

Properties#

• Latency: high; planning is computationally expensive, especially for large state spaces.
• Correctness: high for well-modeled domains; the plan is provably correct with respect to the model.
• Brittleness: high sensitivity to model inaccuracies; if the world model is wrong, the plan is wrong.
• Adaptability: replanning is possible but expensive.

LLM Agent Analog#

An LLM agent that performs explicit multi-step planning before acting—decomposing a user request into a structured task graph, selecting tools for each step, and executing sequentially with verification—is a deliberative agent. The "world model" is the agent's compiled context (beliefs), and the "planning algorithm" is the LLM's chain-of-thought reasoning constrained by the plan library.

3.3.3 Hybrid Architectures#

Definition and Layered Structure#

Hybrid architectures combine reactive and deliberative layers, typically in a layered configuration:

The canonical examples are:

• InteRRaP (Müller, 1996): three layers (behavior, plan, cooperation) with bottom-up activation and top-down commitment.
• TouringMachines (Ferguson, 1992): three layers (reactive, planning, modeling) with a control framework that mediates between layers.
• 3T Architecture (Bonasso et al., 1997): three tiers (skills, sequencing, deliberation) with each tier operating at a different temporal granularity.

The Layered Control Pattern#

ALGORITHM 3.4: Hybrid-Layered-Agent
────────────────────────────────────
Input: Percept p, current agent state S
Output: Action a
 
1.  // Layer 0: Safety / Reactive (highest priority, lowest latency)
2.  IF SAFETY_VIOLATION(p, S) THEN
3.      RETURN SAFETY_ACTION(p)          // immediate, no deliberation
4.
5.  // Layer 1: Procedural / Skill Execution
6.  IF ACTIVE_PLAN(S) AND NEXT_STEP_APPLICABLE(S, p) THEN
7.      RETURN EXECUTE_NEXT_STEP(S)      // continue committed plan
8.
9.  // Layer 2: Deliberative / Planning
10. IF GOAL_UNACHIEVED(S) THEN
11.     plan ← REPLAN(S.model, S.goal, p)
12.     S.plan ← plan
13.     RETURN EXECUTE_NEXT_STEP(S)
14.
15. // Layer 3: Cooperative / Meta-reasoning
16. IF COORDINATION_NEEDED(S) THEN
17.     RETURN COORDINATE(S, peers)
18.
19. RETURN IDLE_ACTION()

Mapping to Production LLM Agent Stacks#

The hybrid layered architecture is the dominant pattern in production agentic systems:

The control framework mediates between layers through priority-based arbitration: safety always preempts planning; planning preempts idle exploration; ongoing skill execution preempts replanning unless a verification failure or impasse is detected.

3.3.4 Trade-Off Analysis#

The hybrid architecture is strictly preferred for production agentic systems because it provides safety guarantees (reactive layer), plan quality (deliberative layer), and coordination capacity (cooperative layer) simultaneously. The cost is implementation complexity, which is managed through typed layer interfaces and explicit control policies.

3.4 Hierarchical Task Networks (HTN) for Complex Plan Decomposition#

3.4.1 Formal Definition#

A Hierarchical Task Network planning domain is defined as:

A task network is a tuple $w = (U, C)$ where $U$ is a set of task nodes and $C$ is a set of constraints (ordering, variable binding, state conditions).

HTN Planning as Recursive Decomposition#

HTN planning proceeds by recursively replacing compound tasks with their method decompositions until only primitive tasks remain:

where $w_0$ contains the initial abstract task and $w_k$ contains only primitive tasks whose execution sequence achieves the goal. The decomposition is valid if:

1. Every compound task is replaced by a method whose preconditions are satisfied.
2. The resulting primitive task sequence is executable in the initial state.
3. All ordering and binding constraints are satisfied.

3.4.2 HTN Decomposition Algorithm#

ALGORITHM 3.5: HTN-Decompose
─────────────────────────────
Input: Task network w = (U, C), methods M, operators O, state s
Output: Executable plan π or FAILURE
 
1.  IF U contains only primitive tasks THEN
2.      π ← LINEARIZE(U, C)                    // respect ordering constraints
3.      IF EXECUTABLE(π, s, O) THEN RETURN π
4.      ELSE RETURN FAILURE
5.  SELECT t ∈ U such that t ∈ T_c             // choose a compound task
6.  applicable ← {m ∈ M : m.task = t AND SATISFIES(m.pre, s)}
7.  IF applicable = ∅ THEN RETURN FAILURE
8.  FOR EACH m ∈ applicable (ordered by heuristic priority):
9.      w' ← REPLACE(w, t, m.subtasks)          // substitute decomposition
10.     C' ← C ∪ m.constraints
11.     result ← HTN_DECOMPOSE(w' = (U', C'), M, O, s)
12.     IF result ≠ FAILURE THEN RETURN result
13. RETURN FAILURE                               // all methods failed → backtrack

3.4.3 Mapping HTN to LLM Agent Task Decomposition#

HTN is the most natural planning formalism for LLM-based agents because LLMs already perform task decomposition as a reasoning pattern (chain-of-thought, tree-of-thought). The mapping is:

Typed Method Schema (Pseudo-Schema)#

METHOD DecomposeCodeRefactor:
  task: "refactor_module"
  preconditions:
    - belief("repository_cloned") = true
    - belief("module_exists", module_name) = true
  subtasks:
    1. analyze_module(module_name) → analysis      [primitive, tool: code_analysis]
    2. generate_plan(analysis) → refactor_plan     [primitive, tool: llm_generate]
    3. apply_changes(refactor_plan) → diff          [primitive, tool: code_edit]
    4. run_tests(module_name) → test_result         [primitive, tool: test_runner]
    5. verify_result(test_result, refactor_plan)    [primitive, tool: llm_verify]
  ordering: 1 < 2 < 3 < 4 < 5
  rollback: revert_changes(diff)
  max_retries: 2

3.4.4 Ordered vs. Partially Ordered HTN#

• Totally Ordered STN (Simple Task Networks): subtasks have a linear order. Simpler to execute but less parallelizable. Used when task dependencies are strictly sequential.
• Partially Ordered STN: subtasks have partial ordering constraints. Enables parallel execution of independent subtasks. Requires a concurrent orchestrator with merge semantics.

For production multi-agent systems, partially ordered HTN is preferred because it enables:

where the critical path is the longest dependency chain. Independent subtasks can be distributed to parallel agents with isolated workspaces.

3.4.5 HTN and Verification#

HTN naturally integrates verification as a decomposition primitive. For every method that produces an artifact, a verification subtask is added:

If $\text{verify}$ fails, the orchestrator triggers a repair method or backtracks. This is the formal basis for the plan → act → verify → repair loop in agentic systems.

3.4.6 Depth Bounding and Cost Control#

Unbounded HTN decomposition is a risk in LLM agents: the model may recursively decompose tasks into ever-finer subtasks, consuming tokens and latency without converging. The mitigation is a depth bound:

and a cost bound:

where $\text{cost}(t_i)$ includes inference tokens, tool invocation cost, and latency. The orchestrator enforces these bounds mechanically, triggering escalation or summarization when limits are approached.

3.5 OODA (Observe–Orient–Decide–Act) Loop as Agent Execution Primitive#

3.5.1 Origin and Formal Structure#

The OODA loop was developed by military strategist John Boyd (1976, 1987) as a model of competitive decision-making. Its power lies not in the individual phases but in the tempo of the loop: the agent that cycles through OODA faster than its environment changes can maintain decision advantage.

3.5.2 Boyd's Insight: Orientation as the Schwerpunkt#

Boyd's most important contribution is the recognition that Orient is the central phase. Orientation determines:

• What the agent attends to in observations (selective perception).
• How the agent interprets observations (framing, schema application).
• What options the agent considers (option generation is orientation-dependent).
• How quickly the agent decides (well-oriented agents decide implicitly, without deliberative search).

In agentic AI terms, Orientation is Context Engineering. The quality of the agent's orientation—its compiled context, retrieved evidence, memory summaries, and policy constraints—determines the quality of every downstream decision.

3.5.3 OODA Mapping to the Agentic Agent Loop#

ALGORITHM 3.6: OODA-Agent-Loop
───────────────────────────────
Input: Environment E, agent state A, context budget W
Output: Continuous action stream
 
1.  LOOP (bounded by max_iterations or convergence):
2.      // OBSERVE
3.      raw ← COLLECT_OBSERVATIONS(E)             // tool outputs, user input, telemetry
4.      filtered ← FILTER_NOISE(raw, A.attention_policy)
5.
6.      // ORIENT
7.      queries ← DECOMPOSE_AND_EXPAND(filtered, A.goals)
8.      evidence ← HYBRID_RETRIEVE(queries, retrieval_config)
9.      memory ← RECALL(A.memory_layers, A.goals)
10.     context ← COMPILE_PREFILL(A.policy, evidence, memory, filtered, budget=W)
11.     situation ← LLM_ORIENT(context)            // situational assessment
12.
13.     // DECIDE
14.     options ← GENERATE_OPTIONS(situation, A.tool_registry)
15.     selected ← EVALUATE_AND_SELECT(options, A.preferences, A.constraints)
16.
17.     // ACT
18.     result ← EXECUTE(selected, A.tool_runtime)
19.     A.history ← APPEND(A.history, (filtered, situation, selected, result))
20.
21.     // FEEDBACK
22.     IF VERIFY(result, selected.postcondition) = FAIL THEN
23.         A.orientation_bias ← UPDATE_BIAS(A.orientation_bias, failure_signal)
24.         CONTINUE                                // rapid re-OODA
25.     ELSE
26.         COMMIT(result)
27.         UPDATE_MEMORY(A.memory_layers, (situation, selected, result))

3.5.4 Tempo and Latency Engineering#

Boyd's competitive advantage through tempo translates directly to latency engineering in agentic systems:

For an agent to maintain decision advantage over environmental change, the cycle time must be less than the environment's change rate:

where $\lambda_{\text{env}}$ is the rate of environmentally significant state changes.

3.5.5 OODA vs. BDI: Complementary, Not Competing#

OODA and BDI are orthogonal frameworks that compose well:

• OODA provides the execution tempo: the continuous loop structure with feedback.
• BDI provides the commitment mechanism: intentions persist across OODA cycles, providing behavioral stability.
• Combined: the agent runs an OODA loop, where Orient corresponds to belief revision, Decide corresponds to intention selection/filtering, and Act corresponds to plan execution. Reconsideration (§3.1) is triggered by Orient detecting significant change.

3.6 Blackboard Architectures for Multi-Source Knowledge Integration#

3.6.1 Formal Structure#

The Blackboard Architecture (Erman et al., 1980; Nii, 1986) was originally developed for the HEARSAY-II speech understanding system. Its core insight is that complex problems requiring heterogeneous knowledge sources are best solved by posting partial solutions to a shared data structure (the blackboard) where independent knowledge sources can contribute incrementally.

3.6.2 Blackboard Operation Cycle#

ALGORITHM 3.7: Blackboard-Cycle
────────────────────────────────
Input: Blackboard BB, Knowledge Sources KS[], Control strategy
Output: Updated Blackboard BB'
 
1.  LOOP until SOLUTION_FOUND(BB) or MAX_CYCLES:
2.      triggered ← {ks ∈ KS : TRIGGER(ks, BB) = true}
3.      IF triggered = ∅ THEN RETURN BB    // quiescent state
4.      ks* ← CONTROL_SELECT(triggered, BB) // select highest-priority KS
5.      contribution ← ks*.EXECUTE(BB)
6.      BB ← APPLY(BB, contribution)
7.      NOTIFY(KS, BB.changes)              // re-evaluate triggers

3.6.3 Mapping to Agentic AI: Multi-Source Retrieval and Reasoning#

The blackboard architecture maps elegantly to multi-source knowledge integration in agentic systems:

Hierarchical Blackboard Levels for Agentic Retrieval#

Level 4: SYNTHESIZED RESPONSE     [Final answer, verified]
Level 3: HYPOTHESES/PLANS         [Candidate answers, plan options]
Level 2: EVIDENCE CHUNKS          [Ranked, provenance-tagged retrieval results]
Level 1: PROCESSED OBSERVATIONS   [Parsed queries, expanded subqueries, metadata]
Level 0: RAW INPUT                [User query, tool outputs, environment signals]

Each knowledge source reads from its input level(s) and writes to its output level. This hierarchy ensures that the synthesis agent (Level 4) operates only on processed, ranked, provenance-tagged evidence, not on raw data.

3.6.4 Production Advantages of the Blackboard Pattern#

1. Modularity: knowledge sources are independently deployable, testable, and replaceable. Adding a new retrieval source requires only registering a new KS with the blackboard, not modifying the orchestrator.
2. Heterogeneity: different KS modules can use different technologies (vector search, SQL, graph queries, API calls) without coupling.
3. Incremental refinement: partial solutions are progressively enriched, which aligns with the agent's iterative reasoning pattern.
4. Observability: the blackboard is a complete, inspectable record of all intermediate contributions, enabling debugging and audit.

3.6.5 Risks and Mitigations#

3.7 Subsumption Architectures for Priority-Based Behavior Arbitration#

3.7.1 Formal Definition#

The Subsumption Architecture (Brooks, 1986) organizes agent behavior as a stack of behavior layers, where each layer implements a complete, independent behavior mapping from percepts to actions. Higher layers subsume (suppress or override) lower layers:

The key property: lower layers are complete, operational systems that function correctly even if all higher layers are removed. Higher layers add competence without modifying lower layers.

3.7.2 Canonical Layer Stack#

Layer 3: EXPLORE        — seek novel states, learn    [lowest priority]
Layer 2: BUILD_MAP      — construct world model
Layer 1: WANDER         — move through environment
Layer 0: AVOID_OBSTACLE — emergency collision avoidance [highest priority]

Layer 0 operates independently and correctly. Layer 1 adds wandering behavior but Layer 0 can inhibit Layer 1's motor output when an obstacle is detected. Layer 2 adds map building but does not affect lower layers' safety guarantees.

3.7.3 Mapping to LLM Agent Safety and Priority Architecture#

The subsumption architecture provides the theoretical foundation for layered safety and priority arbitration in agentic AI systems:

Arbitration Semantics#

The output action at time $t$ is determined by:

This is a strict priority scheme: safety always wins. This is the formal justification for why safety guardrails must be implemented as pre-inference, mechanically enforced gates, not as prompt instructions that the LLM might ignore.

3.7.4 Production Implications#

• Safety invariants are independent of model behavior: Layer 0 operates without LLM inference and cannot be circumvented by prompt injection, jailbreaking, or model error.
• Graceful degradation: if the deliberative layer (Layer 3) fails, the agent falls back to plan execution (Layer 2). If plan execution fails, it falls back to policy enforcement (Layer 1). The system always has a safe, operational mode.
• Incremental capability addition: new competence layers can be added without modifying safety or policy layers, reducing regression risk.
• Testability: each layer can be tested independently, including testing that safety layers correctly inhibit lower layers.

3.8 Actor Model and Communicating Sequential Processes (CSP) for Agent Concurrency#

3.8.1 The Actor Model#

Formal Definition#

The Actor Model (Hewitt et al., 1973; Agha, 1986) defines the fundamental unit of concurrent computation as an actor—an entity that:

1. Has a private state (not shared with any other actor).
2. Processes messages from an unbounded mailbox, one at a time.
3. In response to a message, can: (a) send messages to other actors, (b) create new actors, (c) designate a replacement behavior for the next message.

Key Properties#

• Location transparency: actors are addressed by identity, not location; the runtime handles routing.
• No shared state: all communication is via asynchronous message passing; no locks, no shared memory.
• Mailbox semantics: messages are queued; processing is serial within an actor, parallel across actors.
• Supervision hierarchy: actors are organized in trees; parent actors supervise children and handle their failures (let-it-crash philosophy, formalized in Erlang/OTP and Akka).

3.8.2 Communicating Sequential Processes (CSP)#

Formal Definition#

CSP (Hoare, 1978, 1985) models concurrent systems as processes that communicate through synchronous channels:

Key Properties#

• Synchronous communication: send and receive on a channel happen simultaneously; the sender blocks until the receiver is ready (rendezvous semantics).
• Compositionality: complex processes are built by composing simpler processes through parallel composition, choice, and sequential composition.
• Formal verification: CSP supports model checking for deadlock freedom, livelock freedom, and refinement checking.

3.8.3 Actor vs. CSP: Trade-Off Analysis for Agent Concurrency#

3.8.4 Mapping to Multi-Agent Orchestration#

Actor-Based Multi-Agent Architecture#

ALGORITHM 3.8: Actor-Based-Agent-Orchestration
───────────────────────────────────────────────
Actors:
  - OrchestratorActor: receives user requests, spawns task actors
  - PlannerActor: decomposes tasks into subtasks
  - ExecutorActor[i]: executes primitive tasks (tool calls)
  - VerifierActor: validates execution results
  - MemoryActor: manages memory read/write with validation
  - SupervisorActor: monitors child actors, handles failures
 
Message Flow:
  User → OrchestratorActor: {type: "request", payload: task}
  OrchestratorActor → PlannerActor: {type: "plan", payload: task}
  PlannerActor → OrchestratorActor: {type: "plan_result", payload: subtasks[]}
  OrchestratorActor → ExecutorActor[i]: {type: "execute", payload: subtask_i}
  ExecutorActor[i] → VerifierActor: {type: "verify", payload: result_i}
  VerifierActor → OrchestratorActor: {type: "verified" | "failed", payload: ...}
  
Failure Handling:
  IF ExecutorActor[i] crashes THEN
    SupervisorActor receives EXIT signal
    SupervisorActor restarts ExecutorActor[i] with last checkpoint
    SupervisorActor replays failed message
  IF restart_count > max_retries THEN
    SupervisorActor escalates to OrchestratorActor
    OrchestratorActor triggers alternative decomposition or human escalation

CSP-Based Pipeline Composition#

For sequential agent workflows (plan → retrieve → act → verify → commit), CSP provides a cleaner model:

Each stage is a CSP process; channels ($ch_i$) enforce synchronization points. The pipeline is deadlock-free by construction if every process eventually produces output on its output channel for every input on its input channel (no infinite loops without output).

3.8.5 Production Implications#

3.9 Stigmergic Coordination: Environment-Mediated Multi-Agent Communication#

3.9.1 Formal Definition#

Stigmergy (Grassé, 1959) is a coordination mechanism in which agents communicate indirectly through modifications to a shared environment rather than through direct message passing.

Quantitative Stigmergy#

The strength of a stigmergic signal at location $\ell$ at time $t$ is:

where $w_k$ is the weight of mark $k$ deposited by agent $i$ at time $t_k^{(i)}$, and $\lambda$ is the decay rate. Agents preferentially attend to locations with high $\phi$, creating positive feedback loops that amplify successful patterns.

3.9.2 Mapping to Multi-Agent Agentic Systems#

Stigmergic Task Board#

ALGORITHM 3.9: Stigmergic-Task-Board-Coordination
──────────────────────────────────────────────────
Shared State: TaskBoard TB (visible to all agents)
 
Agent_i behavior loop:
1.  tasks ← SENSE(TB, status="available")
2.  task* ← SELECT(tasks, A_i.competency, priority_score)
3.  success ← CLAIM(TB, task*, A_i.id, lease_duration)  // atomic CAS
4.  IF NOT success THEN GOTO 1                           // another agent claimed it
5.  result ← EXECUTE(task*, A_i.tools)
6.  MARK(TB, task*, result, status="completed" | "failed")
7.  IF result.status = "failed" THEN
8.      MARK(TB, task*, error_signal, status="needs_repair")
9.      // Decay will eventually re-surface this for another agent
10. GOTO 1

3.9.3 Advantages for Large-Scale Multi-Agent Systems#

• Decoupling: agents need no knowledge of each other; coordination emerges from environment state.
• Scalability: adding agents does not require updating routing tables or communication topologies.
• Robustness: agent failure does not break coordination; unclaimed tasks remain on the board.
• Emergent coordination: complex coordinated behavior arises from simple individual rules, without centralized planning.

3.9.4 Risks and Mitigations#

3.10 Contract-Net Protocol and Auction-Based Task Allocation#

3.10.1 Contract-Net Protocol (CNP)#

Formal Definition#

The Contract-Net Protocol (Smith, 1980) is a negotiation-based task allocation mechanism for multi-agent systems:

Bid Evaluation Function#

The manager's bid evaluation function is:

where $\alpha + \beta + \gamma + \delta = 1$ are weights reflecting the manager's priorities.

3.10.2 CNP Protocol Flow for LLM Agents#

ALGORITHM 3.10: Contract-Net-Agent-Allocation
──────────────────────────────────────────────
Participants: Manager M, Contractor agents C[] 
 
1.  // ANNOUNCE
2.  task_spec ← M.decompose_and_specify(user_request)
3.  FOR EACH c IN C:
4.      SEND(c, {type: "cfp", task: task_spec, deadline: T_d})  // call for proposals
5.
6.  // BID
7.  bids ← ∅
8.  WAIT_UNTIL(deadline_bid OR all_responded):
9.      FOR EACH response IN incoming:
10.         IF response.type = "propose" THEN
11.             bids ← bids ∪ {response}
12.         // "refuse" responses are discarded
13.
14. // AWARD
15. IF bids = ∅ THEN
16.     ESCALATE(task_spec, "no_capable_contractor")
17. ELSE
18.     ranked ← SORT(bids, BY score_function, DESCENDING)
19.     winner ← HEAD(ranked)
20.     SEND(winner.agent, {type: "accept"})
21.     FOR EACH loser IN TAIL(ranked):
22.         SEND(loser.agent, {type: "reject"})
23.
24. // EXECUTE & REPORT
25. result ← WAIT(winner.agent, timeout=T_exec)
26. IF result.status = "success" THEN
27.     RETURN result.payload
28. ELSE
29.     // Re-announce to remaining contractors or escalate
30.     RETRY with EXCLUDE(winner.agent)

3.10.3 Auction-Based Task Allocation#

For competitive multi-agent environments or resource-constrained settings, auction mechanisms generalize CNP:

Formal Auction for Task Allocation#

For $m$ tasks $\{T_1, \ldots, T_m\}$ and $n$ agents $\{A_1, \ldots, A_n\}$, the allocation problem is:

subject to:

where $v_{ij}$ is the value (expected quality) of agent $i$ performing task $j$, and $k_i$ is agent $i$'s maximum concurrent task capacity.

3.10.4 Production Considerations#

3.11 Formal Petri Net and State Machine Representations of Agent Lifecycles#

3.11.1 Finite State Machines for Agent Lifecycle#

Formal Definition#

An agent's lifecycle can be modeled as a Deterministic Finite Automaton (DFA):

Agent Lifecycle State Machine#

       task_received            plan_ready           evidence_ready
Idle ───────────────→ Planning ──────────→ Retrieving ──────────→ Executing
                         ↑                                            │
                         │                                            │
                         │              tool_result                   │
                         │         ┌──────────────────────────────────┘
                         │         ↓
                         │     Verifying
                         │      │       │
                         │  verify_pass  verify_fail
                         │      │       │
                         │      ↓       ↓
                         │  Committing  Repairing
                         │      │       │
                         │   done  │  repair_success → Verifying
                         │      │  │  max_retries
                         │      ↓  │      ↓
                         │  Completed    Failed
                         │               │
                         └── retry ───────┘ (if retry_budget > 0)

Transition Function (Excerpt)#

3.11.2 Petri Nets for Concurrent Agent Modeling#

Formal Definition#

A Petri Net is a tuple:

A transition $t$ is enabled at marking $M$ if:

where $\bullet t$ denotes the input places of $t$. Firing $t$ produces a new marking:

Why Petri Nets for Agentic Systems#

Petri nets are strictly more expressive than finite state machines for modeling agentic systems because they naturally represent:

1. Concurrency: multiple tokens can be in different places simultaneously, modeling parallel agent execution.
2. Synchronization: transitions with multiple input places model synchronization barriers (e.g., "wait for all sub-agent results before synthesis").
3. Resource constraints: token counts model resource availability (e.g., available tool invocation budget, concurrent request capacity).
4. Conflict/Choice: when multiple transitions are enabled by the same tokens, this models nondeterministic choice (e.g., tool selection).

Agent Execution Petri Net#

Places:
  p_idle, p_planning, p_plan_ready, p_retrieving, p_evidence_ready,
  p_executing, p_tool_result, p_verifying, p_verified, p_failed,
  p_repairing, p_committing, p_completed,
  p_token_budget (resource place, initial tokens = max_budget),
  p_retry_budget (resource place, initial tokens = max_retries)
 
Transitions:
  t_receive_task: {p_idle} → {p_planning}
  t_plan: {p_planning} → {p_plan_ready}
  t_retrieve: {p_plan_ready} → {p_retrieving}
  t_evidence: {p_retrieving} → {p_evidence_ready}
  t_execute: {p_evidence_ready, p_token_budget} → {p_executing}  
      // consumes a token budget unit
  t_result: {p_executing} → {p_tool_result}
  t_verify: {p_tool_result} → {p_verifying}
  t_pass: {p_verifying} → {p_verified}
  t_fail: {p_verifying, p_retry_budget} → {p_repairing}  
      // consumes a retry token
  t_repair: {p_repairing} → {p_tool_result}    // loops back to verify
  t_exhaust: {p_verifying} → {p_failed}  
      // fires when p_retry_budget is empty (no retry tokens)
  t_commit: {p_verified} → {p_committing}
  t_done: {p_committing} → {p_completed}

Formal Analysis Properties#

Petri nets enable formal verification of agent lifecycle properties:

3.11.3 Colored Petri Nets for Multi-Agent Modeling#

Colored Petri Nets (CPN) extend standard Petri nets with typed tokens, enabling modeling of multiple agents, task types, and data flows in a single net:

where $\Sigma$ is a set of color sets (types), $C : P \rightarrow \Sigma$ assigns a type to each place, $G : T \rightarrow \text{Guard}$ assigns guard conditions to transitions, and $E : F \rightarrow \text{ArcExpression}$ assigns typed arc expressions.

In multi-agent systems, token colors can encode:

• Agent identity: which agent is in which state.
• Task parameters: task type, priority, deadline.
• Resource type: GPU budget, API rate limit, token budget.

This enables modeling and verification of complex multi-agent orchestration with heterogeneous agents and resources.

3.11.4 Production Application: Agent Lifecycle as Verifiable State Machine#

Every production agent orchestrator should implement its agent lifecycle as an explicit, verifiable state machine (or Petri net), not as implicit control flow in code. Benefits:

1. Invariant enforcement: the state machine definition mechanically prevents illegal transitions (e.g., executing without planning, committing without verifying).
2. Observability: current state is always inspectable; state transitions are logged with timestamps and payloads.
3. Recovery: after crash, the agent resumes from its last committed state, not from the beginning.
4. Testing: state machine coverage analysis ensures all states and transitions are exercised in testing.
5. Formal verification: for critical applications, model checking can verify deadlock freedom, boundedness, and liveness.

3.12 Category-Theoretic Composition of Agent Pipelines#

3.12.1 Motivation: Compositionality as Engineering Discipline#

Agent systems are assembled from components: retrievers, planners, executors, verifiers, memory managers, synthesizers. The central question is: when can components be composed, and does the composite inherit the properties of its parts?

Category theory provides the most general framework for answering this question. It abstracts away implementation details and focuses on composition structure: what composes with what, and what properties are preserved.

3.12.2 Category-Theoretic Foundations#

Categories#

A category $\mathcal{C}$ consists of:

• Objects: $\text{Ob}(\mathcal{C})$ — e.g., types, state spaces, data schemas.
• Morphisms: for each pair of objects $A, B$, a set $\text{Hom}(A, B)$ of arrows from $A$ to $B$ — e.g., typed functions, agent transformations, pipeline stages.
• Composition: for morphisms $f : A \rightarrow B$ and $g : B \rightarrow C$, a composite $g \circ f : A \rightarrow C$.
• Identity: for each object $A$, an identity morphism $\text{id}_A : A \rightarrow A$.
• Associativity: $h \circ (g \circ f) = (h \circ g) \circ f$.
• Identity law: $f \circ \text{id}_A = f = \text{id}_B \circ f$.

The Category of Agent Components#

Define $\mathcal{C}_{\text{Agent}}$ with:

• Objects: typed data schemas (e.g., $\text{UserQuery}$, $\text{RetrievalResult}$, $\text{Plan}$, $\text{ToolOutput}$, $\text{VerifiedResult}$, $\text{Response}$).
• Morphisms: agent components as typed transformations:
  • $\text{retrieve} : \text{UserQuery} \rightarrow \text{RetrievalResult}$
  • $\text{plan} : \text{RetrievalResult} \rightarrow \text{Plan}$
  • $\text{execute} : \text{Plan} \rightarrow \text{ToolOutput}$
  • $\text{verify} : \text{ToolOutput} \rightarrow \text{VerifiedResult}$
  • $\text{synthesize} : \text{VerifiedResult} \rightarrow \text{Response}$

The pipeline is the composite morphism:

Associativity guarantees that we can group and refactor pipeline stages without changing semantics:

3.12.3 Functors: Structure-Preserving Mappings Between Agent Categories#

A functor $F : \mathcal{C} \rightarrow \mathcal{D}$ maps objects and morphisms from one category to another while preserving composition and identity:

Application: Testing Functor#

Define a testing functor $T : \mathcal{C}_{\text{Prod}} \rightarrow \mathcal{C}_{\text{Test}}$ that maps production components to their test doubles:

• $T(\text{retrieve}_{\text{prod}}) = \text{retrieve}_{\text{mock}}$ (returns canned results)
• $T(\text{execute}_{\text{prod}}) = \text{execute}_{\text{stub}}$ (simulates tool calls)

The functor property guarantees that if the production pipeline composes correctly, the test pipeline composes correctly with the same structure. This is the formal basis for compositional testing of agent pipelines.

Application: Observability Functor#

Define an observability functor $O : \mathcal{C}_{\text{Agent}} \rightarrow \mathcal{C}_{\text{Traced}}$ that wraps each morphism with tracing:

The functor preserves composition: tracing the composite pipeline is equivalent to composing the individually traced stages.

3.12.4 Monoidal Categories: Parallel Composition#

A monoidal category $(\mathcal{C}, \otimes, I)$ adds a tensor product $\otimes$ that models parallel composition:

• $f : A \rightarrow B$ and $g : C \rightarrow D$ compose in parallel as $f \otimes g : A \otimes C \rightarrow B \otimes D$.
• $I$ is the unit object (the "empty" type; doing nothing in parallel changes nothing).

Application: Parallel Agent Execution#

Two independent agent tasks $f$ and $g$ can be executed in parallel:

The monoidal structure guarantees that:

1. Independence: $f$ and $g$ do not interact (no shared state, isolated workspaces).
2. Determinism: the result of $f \otimes g$ is the pair of results, regardless of execution order.
3. Composability: $(f \otimes g) \circ (h \otimes k)$ is well-defined when types align.

This provides the formal justification for the parallel agent execution model described in the system prompt: "Parallel agents shall operate on independently claimable work units."

3.12.5 Natural Transformations: Component Substitution#

A natural transformation $\eta : F \Rightarrow G$ between functors $F, G : \mathcal{C} \rightarrow \mathcal{D}$ is a family of morphisms $\{\eta_A : F(A) \rightarrow G(A)\}_{A \in \mathcal{C}}$ such that for every $f : A \rightarrow B$:

This naturality square commutes:

Application: Retriever Substitution#

If $F$ and $G$ are two retrieval strategies (e.g., semantic search and hybrid search), a natural transformation $\eta$ provides a systematic migration path: for every query type, there exists a transformation from $F$'s result to $G$'s result that commutes with all downstream pipeline stages. This guarantees that replacing a retriever does not break the pipeline's composition structure.

3.12.6 Monads: Effectful Agent Computation#

A monad $(M, \eta, \mu)$ on a category $\mathcal{C}$ consists of:

• An endofunctor $M : \mathcal{C} \rightarrow \mathcal{C}$.
• A unit (return) $\eta_A : A \rightarrow M(A)$.
• A multiplication (join/flatten) $\mu_A : M(M(A)) \rightarrow M(A)$.

satisfying associativity and unit laws.

Application: Error Handling as Monad#

Define $M(A) = \text{Result}\langle A, \text{Error}\rangle$. Every agent component returns $M(A)$ instead of $A$, explicitly propagating errors through the pipeline:

Monadic composition (Kleisli composition) $g \circ_K f$ automatically handles error short-circuiting:

This provides the formal basis for typed error propagation in agent pipelines, ensuring that failures are handled uniformly and cannot be silently dropped.

Application: Retrieval Context as Monad#

Define $M(A) = \text{Contextualized}\langle A, \text{Provenance}, \text{Score}\rangle$. Every retrieval result carries provenance and relevance metadata. Monadic composition ensures that provenance is preserved through all pipeline stages:

flattens nested contextualization (e.g., retrieval of a retrieval result) into a single provenance chain.

3.12.7 String Diagrams: Visual Pipeline Composition#

String diagrams provide a graphical syntax for monoidal categories that makes pipeline composition visually explicit:

Sequential:    A ─── f ─── B ─── g ─── C
 
Parallel:      A ─── f ─── B
               C ─── g ─── D
 
Feedback:      A ─── f ──┬── B
                         │
                         └── (feedback to A via h)

String diagrams are sound and complete for monoidal categories: every valid diagram corresponds to a well-typed pipeline, and every well-typed pipeline has a diagram. This provides a formal specification language for agent pipeline architecture.

3.12.8 Composition Properties and Invariant Preservation#

The category-theoretic framework guarantees the following properties when agent pipelines are composed through categorical constructs:

3.12.9 Practical Typed Interface Design from Category Theory#

Category theory motivates the following production design principles:

1. Every agent component is a typed morphism: input schema → output schema, with explicit error types.
2. Pipelines are compositions: validated at construction time by type checking, not at runtime by exception handling.
3. Parallel execution requires monoidal independence: components in a parallel group must have no shared mutable state. Verify mechanically.
4. Component substitution requires natural transformation: replacing a retriever, planner, or verifier requires demonstrating that the new component satisfies the same naturality conditions (same interface contract, commuting with downstream stages).
5. Error handling is monadic: every component wraps its output in a Result monad; the pipeline framework provides Kleisli composition for automatic error propagation.
6. Observability is functorial: tracing, logging, and metrics are applied as functors that preserve pipeline structure.

Chapter Summary and Architectural Selection Guide#

Synthesis: Architecture Selection Matrix#

The Production Agentic Stack as Architectural Composition#

A production-grade agentic system does not use one architecture; it composes multiple architectures at different layers:

Each architecture provides a specific structural guarantee. Together, they yield a system that is formally grounded, concurrency-safe, verifiable, observable, and compositionally maintainable.

The discipline of selecting, combining, and formally justifying these architectural choices—rather than improvising control flow around prompt templates—is what separates engineering from experimentation in the construction of agentic AI systems.

End of Chapter 3.