Designing Lightweight AI Agents for Edge Deployment

This first part of the thesis establishes the foundational motivation, problem context, and methodology. It begins by identifying the increasing need for lightweight, deployable AI agents in edge environments (Chapter 1), and articulates a clear research gap: the absence of design-first, minimal frameworks for agent construction.

Chapter 2 reviews the literature underpinning this gap, focusing on key architectural domains: lightweight modeling, prompt engineering, memory constraints, and over-engineering in agent stacks. These findings motivate the Minimal Capability Design (MCD) framework introduced in later chapters.

Chapter 3 then outlines the methodology used to construct and validate the MCD framework—grounded in literature synthesis, design principles, and validation via simulation and walkthroughs. Together, these chapters define the scope, motivation, and research logic for the work that follows.

In recent years, the rise of transformer-based agents has led to a duality between performance-oriented orchestration frameworks and task-specific, domain-bounded deployments (Vaswani et al., 2017; Brown et al., 2020). This thesis pursues the latter: the design of agents that operate effectively within tight constraints, even at the cost of generality.

Existing AI agents are typically constructed under assumptions of abundant memory, orchestration infrastructure, and access to external toolchains (Brown et al., 2020; Shinn et al., 2023; Zhou et al., 2023). These defaults introduce avoidable cost, latency, and fragility — especially in edge-aligned applications where devices have tight resource budgets and may operate offline (Xu et al., 2023). Beyond performance considerations, edge deployment introduces critical safety implications where agent failure modes must be predictable and transparent (Amodei et al., 2016). Traditional agents often exhibit dangerous failure patterns—confident but incorrect responses—while minimal agents can be designed for safe degradation, acknowledging limitations rather than providing misleading outputs (Kadavath et al., 2022). The rise of AI agents on constrained devices like phones, browsers, and microcontrollers means cloud-based orchestration is often overkill for simple tasks (Li et al., 2024).

Recent work on lightweight model deployment (Dettmers et al., 2022; Frantar et al., 2023) focuses on computational optimization but does not address interaction minimalism or stateless reasoning as first-class design principles. This gap motivates the Minimal Capability Design (MCD) framework, which treats minimalism, statelessness, and prompt resilience as foundational concerns (Sahoo et al., 2024). Unlike performance-optimized models, minimal agents prioritize interpretability and robustness under tight constraints, making them ideal for edge-aligned design (Ribeiro et al., 2016).

MCD addresses a critical gap: while existing frameworks optimize for peak performance under ideal conditions, they often degrade unpredictably when resource constraints intensify (Strubell et al., 2019). This thesis positions constraint-resilience as a primary design objective, acknowledging that edge deployment scenarios require agents that maintain stable functionality as computational budgets decrease, rather than maximizing performance in resource-abundant environments (Schwartz et al., 2020).

Recent research on Small Language Models (SLMs) also demonstrates parallel trends toward specialization and efficiency, providing additional validation for constraint-first design approaches (Belcak et al., 2025). While SLMs achieve efficiency through domain specialization and parameter reduction, MCD achieves similar goals through architectural constraints and stateless design—suggesting these approaches are complementary rather than competing (Magnini et al., 2025). This convergence of model-level and architectural minimalism validates the broader industry shift toward constraint-aware AI deployment strategies. The framework's model-agnostic design principles (Section 4.9.1) ensure compatibility with emerging optimization strategies including quantization, pruning, and domain-specialized SLMs.

Thereby, MCD tries to represent a important shift from “build complex, then optimize” to “design minimal, verify sufficiency” (Mitchell, 2019). This affects not just prompt engineering, but entire agent architectures including memory systems, tool orchestration, and execution environments (Wei et al., 2022). The framework establishes constraint-first design principles that apply across all architectural layers—from token-level prompting decisions to system-wide capability selection—ensuring that minimalism is embedded at the design stage rather than retrofitted during deployment (Liu et al., 2023).

For clarity, the thesis uses the following operational definitions:

Table 1.1 - Operational Definitions Table

Despite advances in AI agent design and model optimization strategies such as PEFT or distillation, most frameworks implicitly assume abundant memory, persistent state, orchestration layers, or retraining access (Hu et al., 2021; Hinton et al., 2015). These architectural defaults increase cost, latency, and fragility—and are unnecessary for many real-world edge deployments (Brown et al., 2020; Kojima et al., 2022; Zhou et al., 2023).

However, the field lacks principled frameworks that (Qin et al., 2023):
- Treat minimalism and statelessness as foundational design constraints.
- Systematically evaluate agent robustness under these constraints.
- Detect over-engineering or capability collapse before deployment.

This study addresses the gap by proposing and validating the Minimal Capability Design (MCD) framework, which provides a structured approach to designing and diagnosing lightweight, interpretable agents for edge environments (Zhang et al., 2024). The framework specifically addresses scenarios where traditional approaches fail due to resource limitations, providing reliable baseline performance under constraint conditions where alternative architectures degrade significantly or fail unpredictably (Chen et al., 2023).

To address this problem, the thesis investigates:
- RQ1: What design principles enable stateless, low-resource AI agents to function reliably? (Wang et al., 2024)
- RQ2: How can architectural complexity be minimized to provide predictable baseline performance under resource constraints, even when this requires sacrificing peak performance in optimal conditions? (Tay et al., 2022)
- RQ3: How can agent behavior be systematically evaluated for robustness under constraints such as prompt compression, fallback handling, and statelessness? (Min et al., 2022)
- RQ4: What diagnostic signals reveal over-engineering, excessive capabilities, or fragility in minimal agents? (Perez et al., 2022)

Aim:
To propose and validate a generalizable design framework—Minimal Capability Design (MCD)—for constructing lightweight, interpretable AI agents suitable for real-world edge deployment (Bommasani et al., 2021).

Objectives:
- Formalize design principles that prioritize minimalism, robustness, and prompt resilience (Zhou et al., 2022).
- Validate the framework via literature synthesis, simulation in a constrained browser-based runtime (which serves as an effective proxy for edge deployment constraints), and walkthroughs across diverse agent domains (Thoppilan et al., 2022).
- Extract a diagnostic toolkit to detect symptoms of over-engineering, fragility, or prompt failure modes in minimal agents (Ouyang et al., 2022).
- Anticipate hardware-based benchmarking extensions using edge boards such as Raspberry Pi or Jetson Nano in future iterations (Singh et al., 2023).
- Justify the choice of quantization as the primary optimization strategy through comparative architectural review, considering resource alignment, reproducibility, and deployment feasibility (Zafrir et al., 2019)

This thesis makes the following contributions:
- A formal, literature-derived design framework — Minimal Capability Design (MCD) — that prioritizes constraint-resilience and predictable degradation patterns over peak performance, treating minimalism, statelessness, and prompt resilience as primary design constraints rather than post hoc optimizations.
- A principled diagnostic methodology for detecting over-engineering, capability excess, and prompt fragility in AI agents, grounded in both theoretical synthesis and controlled simulation.
- A browser-based, reproducible simulation testbed that emulates edge constraints (no memory, limited token budgets, stateless execution) to stress-test agent designs.
- Defined and implemented a quantization-aware agent architecture using 1-bit (simulated), 4-bit, and 8-bit model tiers, selected after comparative consideration of alternative optimization approaches (e.g., distillation, PEFT) in terms of edge suitability.
- Demonstrated the feasibility of deploying fallback-capable lightweight agents in browser and edge settings.
- Domain-specific walkthroughs demonstrating the application of MCD principles to real-world agent use cases, highlighting both strengths and trade-offs.
- A taxonomy of heuristic indicators and failure patterns that can be applied across domains to evaluate and refine lightweight agent designs.
- Design heuristics operationalized through agent checklists and failure diagnostics (Appendix E).
- Agent architecture diagrams (Appendix D) support reproducibility and instantiation clarity.
- A unifying validation arc combining theoretical stress tests and applied agent walkthroughs to operationalize minimal design.
- Empirical validation that MCD maintains stable performance under progressive constraint pressure (quantization degradation, token limitations, memory restrictions) where traditional approaches show significant performance loss, providing evidence for constraint-first design philosophy.

While numerous optimization strategies exist—such as pruning, distillation, parameter-efficient fine-tuning (PEFT), and adaptive computation—this thesis focuses explicitly on quantization (1-bit, 4-bit, and 8-bit tiers) (Jacob et al., 2018; Nagel et al., 2021). This focus stems from
- The practical relevance of quantization to runtime deployment in browser and microcontroller contexts, validated through comparative analysis demonstrating superior constraint-resilience characteristics - maintaining functionality when alternatives degrade under resource pressure - even when sacrificing optimal-condition performance,
- Its minimal hardware dependency and compatibility with edge toolchains (e.g., WebAssembly, ONNX runtimes), and
- The relative simplicity of its integration without retraining or fine-tuning.

Scope Clarification: This work does not benchmark or fine-tune LLMs for downstream performance. Among various optimization strategies, only quantization is pursued as it enables runtime minimization without retraining or parameter tuning.

Other optimization approaches (e.g., LoRA, adapters, distillation) are acknowledged and briefly discussed in Chapter 3, but are excluded from implementation due to either increased training dependency, storage footprint, or poor alignment with stateless agent goals.

With the problem defined and the research questions articulated, the next chapter reviews relevant literature on lightweight agent design, prompt-based reasoning, memory architectures, and over-engineering in AI systems. Rather than following a chronological review structure, this examination is organized by core architectural concerns—lightweight modeling, prompt reasoning, memory constraints, and modular complexity—to systematically evaluate how current agent design approaches attempt to address edge constraints. This analysis highlights where existing solutions fall short of supporting edge-native, minimal-capability agents and identifies gaps that necessitate a new design-oriented framework—specifically one that prioritizes reliable constraint-handling over peak performance optimization, motivating the Minimal Capability Design (MCD) framework proposed in Chapter 4.