**Research Problem and Approach:** The rapid proliferation of Large Language Models (LLMs) has revolutionized artificial intelligence but introduced significant computational barriers that hinder deployment on resource-constrained edge devices. This research addresses the critical inefficiency of deploying modern Transformers on integer-only hardware architectures, where a fundamental mismatch between advanced quantization algorithms and hardware capabilities creates a substantial latency gap. The study adopts a hardware-software co-design approach to reconcile the high dynamic range requirements of activation outliers with the strict limitations of low-power, integer-based processing units found in FPGAs and RISC-V microcontrollers.

**Methodology and Findings:** By analyzing the mathematical formulation of quantization and the architectural constraints of edge computing, this investigation identifies the root causes of performance degradation in post-training quantization (PTQ). The research demonstrates that standard methods, such as GPTQ and AWQ, often fail to translate theoretical compression into wall-clock speedups on strict edge hardware due to their reliance on floating-point fallback operations. The study evaluates optimized quantization strategies designed for configurable systolic arrays, showing that efficient inference is achievable without the expensive silicon footprint required for mixed-precision arithmetic.

**Key Contributions:** This thesis makes three primary contributions: (1) A systematic analysis of the hardware-algorithm mismatch in state-of-the-art quantization techniques when applied to integer-only architectures, (2) An evaluation of strategies to mitigate the impact of activation outliers without reverting to floating-point operations, and (3) A framework for optimizing quantization specifically for hardware accelerators like systolic arrays to maximize resource utilization and throughput.

**Implications:** The findings have profound implications for the democratization of AI, enabling privacy-preserving, local inference for sensitive applications in healthcare and security. By bridging the gap between massive model complexity and edge hardware constraints, this research offers a pathway toward sustainable AI development that significantly reduces the carbon footprint of inference while extending the operational lifespan of battery-powered autonomous systems.

**Keywords:** Large Language Models, Edge AI, Quantization, Hardware-Software Co-Design, Integer-Only Inference, Post-Training Quantization, Activation Outliers, FPGA, RISC-V, Energy Efficiency, Systolic Arrays, Model Compression, Latency Optimization, Sustainable AI, Deep Learning