# Scout Output - Academic Citation Discovery

## Summary

**Total Valid Citations**: 43
**Success Rate**: 43.4%
**Failed Topics**: 71

### Sources Breakdown

- **Crossref**: 23 (53.5%)
- **Semantic Scholar**: 17 (39.5%)
- **Gemini Grounded**: 3 (7.0%)
- **Gemini LLM**: 0 (0.0%)

---

## Citations Found

### From Crossref (23 citations)

#### 1. Addressing Activation Outliers in LLMs: A Systematic Review of Post-Training Quantization Techniques
**Authors**: Czakó, Kertész, Szénási
**Year**: 2025
**DOI**: 10.1109/access.2025.3568702
**URL**: https://doi.org/10.1109/access.2025.3568702

#### 2. HADQ-Net: A Power-Efficient and Hardware-Adaptive Deep Convolutional Neural Network Translator Based on Quantization-Aware Training for Hardware Accelerators
**Authors**: Oflamaz, Yalçın
**Year**: 2025
**DOI**: 10.3390/electronics14183686
**URL**: https://doi.org/10.3390/electronics14183686

**Abstract**: With the increasing demand for implementing deep-learning models on devices on resource-constrained devices, the development of power-efficient neural networks has become imperative. This paper introduces HADQ-Net, a novel framework for optimizing deep convolutional neural networks (CNNs) through Quantization-Aware Training (QAT). By compressing 32-bit floating-point (FP32) precision weights and activation values to lower bit-widths, HADQ-Net significantly reduces memory footprint and computational complexity while maintaining high accuracy. We propose adaptive quantization limits based on the statistical properties of each layer or channel, coupled with normalization techniques, to enhance quantization efficiency and accuracy. The framework includes algorithms for QAT, quantized convolution, and quantized inference, enabling efficient deployment of deep CNN models on edge devices. Extensive experiments across tasks such as super-resolution, classification, object detection, and semantic segmentation demonstrate the trade-offs between accuracy, model size, and computational efficiency under various quantization levels. Our results highlight the superiority of QAT over post-training quantization methods and underscore the impact of quantization types on model performance. HADQ-Net achieves significant reductions in memory footprint, computational complexity, and energy consumption, making it ideal for resource-constrained environments without sacrificing performance.

#### 3. Fixed- and Floating-point Arithmetic
**Authors**: GEORGE
**Year**: 1966
**DOI**: 10.1016/b978-0-08-011463-7.50012-7
**URL**: https://doi.org/10.1016/b978-0-08-011463-7.50012-7

#### 4. NeuronMM: High-Performance Matrix Multiplication for LLM Inference on AWS Trainium
**Authors**: Song, Yang
**Year**: 2025
**DOI**: 10.22541/au.176463814.43517852/v1
**URL**: https://doi.org/10.22541/au.176463814.43517852/v1

#### 5. Flatten-DiT: Mitigating Activation Outliers for Accurate Post-Training Quantization of Diffusion Transformers
**Authors**: Zhou, Hou, Huang, Zhou, Xiao, Wang, Chenwei
**Year**: 2025
**DOI**: 10.2139/ssrn.5413755
**URL**: https://doi.org/10.2139/ssrn.5413755

#### 6. Zero-shot Mathematical Problem Solving via Generative Pre-trained Transformers
**Authors**: Galatolo, Cimino, Vaglini
**Year**: 2022
**DOI**: 10.5220/0011032400003179
**URL**: https://doi.org/10.5220/0011032400003179

#### 7. Scalable Scheduling and Intelligent Resource Optimization for Efficient Large Language Model Inference Acceleration
**Authors**: Cheung
**Year**: 2025
**DOI**: 10.36227/techrxiv.176581822.29363022/v1
**URL**: https://doi.org/10.36227/techrxiv.176581822.29363022/v1

#### 8. Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
**Authors**: Shen, Dong, Yao, Gholami, Mahoney, Keutzer
**Year**: 2020
**DOI**: 10.1609/aaai.v34i05.6409
**URL**: https://doi.org/10.1609/aaai.v34i05.6409

**Abstract**: Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accuracy gain for GLUE tasks, CoNLL-03 and SQuAD. However, BERT based models have a prohibitive memory footprint and latency. As a result, deploying BERT based models in resource constrained environments has become a challenging task. In this work, we perform an extensive analysis of fine-tuned BERT models using second order Hessian information, and we use our results to propose a novel method for quantizing BERT models to ultra low precision. In particular, we propose a new group-wise quantization scheme, and we use Hessian-based mix-precision method to compress the model further. We extensively test our proposed method on BERT downstream tasks of SST-2, MNLI, CoNLL-03, and SQuAD. We can achieve comparable performance to baseline with at most 2.3% performance degradation, even with ultra-low precision quantization down to 2 bits, corresponding up to 13× compression of the model parameters, and up to 4× compression of the embedding table as well as activations. Among all tasks, we observed the highest performance loss for BERT fine-tuned on SQuAD. By probing into the Hessian based analysis as well as visualization, we show that this is related to the fact that current training/fine-tuning strategy of BERT does not converge for SQuAD.

#### 9. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration
**Authors**: Lin, Tang, Tang, Yang, Xiao, Han
**Year**: 2025
**DOI**: 10.1145/3714983.3714987
**URL**: https://doi.org/10.1145/3714983.3714987

**Abstract**: Large language models (LLMs) have transformed numerous AI applications. On-device LLM is becoming increasingly important: running LLMs locally on edge devices can reduce cloud computing costs and protect users' privacy. However, the astronomical model size and the limited hardware resources pose significant deployment challenges. To solve these issues, we propose Activation-aware Weight Quantization (AWQ) and TinyChat, an algorithm-system full-stack solution for efficient on-device LLM deployment. AWQ is a novel quantization method that identifies and protects salient weights based on activation distribution, significantly reducing model size while preserving performance. TinyChat, an optimized inference framework, translates AWQ's theoretical memory savings into practical speedups through techniques such as on-the-fly dequantization, SIMD-aware weight packing, and kernel fusion. Together, they enable 4x model size reduction and 3-4x acceleration across various edge platforms, from high-end desktop GPUs to resource-constrained IoT devices. This solution democratizes on-device LLM deployment, offering privacy-preserving, low-latency AI capabilities across a wide range of applications.

#### 10. Enhancing Large Vision-Language Models via Quantized Grounded Reasoning
**Authors**: Reed, Barnes
**Year**: 2025
**DOI**: 10.20944/preprints202510.2397.v1
**URL**: https://doi.org/10.20944/preprints202510.2397.v1

**Abstract**: Large Vision-Language Models (LVLMs) have achieved strong results in general visual understanding but remain limited in fine-grained visual reasoning. This paper introduces LVLM-GR, a framework designed to improve detailed visual grounding and robust multimodal reasoning. The proposed Visual Concept Quantizer (VCQ) encodes images into discrete visual tokens through context-aware pooling and a semantic hierarchical codebook, effectively preserving fine-grained semantics. These visual tokens are then aligned with language via a lightweight Grounded Reasoning Adapter (GRA) based on LoRA-tuned adaptation atop a frozen LLaVA 1.5 13B backbone. Experiments on GQA, RefCOCO+, and A-OKVQA show that LVLM-GR achieves superior performance in fine-grained visual understanding, reasoning, and grounding, highlighting its potential for complex multimodal reasoning tasks in material-level and detailed visual analysis.

#### 11. A Survey of Quantization Methods for Efficient Neural Network Inference
**Authors**: Gholami, Kim, Dong, Yao, Mahoney, Keutzer
**Year**: 2022
**DOI**: 10.1201/9781003162810-13
**URL**: https://doi.org/10.1201/9781003162810-13

#### 12. Efficient methods for deep learning
**Authors**: Cai, Lin, Han
**Year**: 2022
**DOI**: 10.1016/b978-0-12-822109-9.00013-8
**URL**: https://doi.org/10.1016/b978-0-12-822109-9.00013-8

#### 13. Hardware-Software Co-Design for Efficient LLM Inference on PCIe-Based FPGAs Using Coarse-Grained Systolic Arrays
**Authors**: Chang
**Year**: 2025
**DOI**: 10.1109/socc66126.2025.11235351
**URL**: https://doi.org/10.1109/socc66126.2025.11235351

#### 14. Model Compression for Efficient Generative AI
**Authors**: Guo
**Year**: 2025
**DOI**: 10.1145/3746262.3761971
**URL**: https://doi.org/10.1145/3746262.3761971

#### 15. Hardware Acceleration of Graph Neural Networks
**Authors**: Auten, Tomei, Kumar
**Year**: 2020
**DOI**: 10.1109/dac18072.2020.9218751
**URL**: https://doi.org/10.1109/dac18072.2020.9218751

#### 16. Scalar Arithmetic Multiple Data: Customizable Precision for Deep Neural Networks
**Authors**: Anderson, Doyle, Gregg
**Year**: 2019
**DOI**: 10.1109/arith.2019.00018
**URL**: https://doi.org/10.1109/arith.2019.00018

#### 17. Intel OpenVino: A Must-Know Deep Learning Toolkit
**Authors**: Vasudevan, Pulari, Vasudevan
**Year**: 2021
**DOI**: 10.1201/9781003185635-11
**URL**: https://doi.org/10.1201/9781003185635-11

#### 18. Qualcomm snapdragon "bullet train"
**Authors**: Pipes
**Year**: 2015
**DOI**: 10.1145/2745234.2746856
**URL**: https://doi.org/10.1145/2745234.2746856

#### 19. FPGA-Accelerated Real-Time DCGANs via Xilinx DPUs and Vitis AI
**Authors**: Sadr, Haghighat, Pakniyat, Rahmati, Gorgin
**Year**: 2025
**DOI**: 10.21203/rs.3.rs-7263274/v1
**URL**: https://doi.org/10.21203/rs.3.rs-7263274/v1

**Abstract**: Abstract
        Generative Adversarial Networks (GANs) produce high-quality images but are computationally intensive, especially due to transposed convolution operations, limiting their real-time performance on traditional hardware. To address this, we propose an optimized FPGA-based acceleration framework leveraging Xil-inx Deep Learning Processing Units (DPUs) and the Vitis AI toolchain to enable real-time inference of Deep Convolutional GANs (DCGANs) for image reconstruction. The proposed approach applies a two-stage quantization method that profiles layer-wise dynamic ranges and fine-tunes scale factors via host-side retraining. This enables quantization of both generator and discriminator from 32-bit floating-point to INT8 precision with minimal accuracy degradation. Additionally, structured pruning through the Vitis AI Optimizer removes redundant weights and filters, producing a compact model that fits entirely in on-chip memory and maximizes DPU efficiency. The architecture uses a multi-threaded ARM processor to manage preprocessing and DMA operations, while a lightweight scheduler in programmable logic sequences the execution of convolu-tion kernels across multiple DPU cores. Double buffering is employed to overlap data movement with computation. Experimental results on a Zynq UltraScale+ MPSoC ZCU104 show over 105 FPS throughput, achieving up to 3.5× better performance and 7.3× energy efficiency than GPU/CPU baselines, with Fréchet Inception Distance (FID) scores within 5% of floating-point models.

#### 20. Quantization of a Llama Language Model for improved Efficiency and Inference
**Authors**: Madhanegha, Vishnuvaradhan, Arun, Surenther
**Year**: 2025
**DOI**: 10.21203/rs.3.rs-6021454/v1
**URL**: https://doi.org/10.21203/rs.3.rs-6021454/v1

**Abstract**: Abstract
        Despite their transformational potential, large language models (LLMs) like Llama are difficult to implement on devices with limited computational power due to their high computational requirements. This study explores the quantization of the Lamba model, a method that minimizes memory footprint and model size for effective deployment. In order to obtain significant model compression with acceptable performance, we investigate different quantization techniques. At various quantization levels, the study will assess the trade-off between efficiency and accuracy. We will also look into how quantization affects the target devices' power consumption and inference speed. By enabling deployment on resource-constrained platforms and effectively quantifying the Llama model, this initiative seeks to democratize access to potent AI tools, encouraging greater innovation and practical applications. 
Additionally, a smaller model results in cheaper implementation costs and enhanced sustainability due to lower inference power usage. 
In order to quantified the Llama model, this research explores a number of technical approaches, assesses performance trade-offs, and optimizes deployment for effective hardware use. This project's objective is to successfully quantify the Llama model in order to show that it is feasible to implement it in contexts with limited resources. The results will help create LLMs that are easier to use and more effective.

#### 21. Optimasi Proses Klasifikasi Topik Berita Berbahasa Indonesia Menggunakan Model NLP Berbasis Onnx Runtime
**Authors**: Risnanto, Poerwandono
**Year**: 2025
**DOI**: 10.36040/jati.v9i6.15707
**URL**: https://doi.org/10.36040/jati.v9i6.15707

**Abstract**: Perkembangan berita digital yang pesat memunculkan tantangan baru dalam klasifikasi topik berita secara akurat, terutama ketika satu artikel dapat memiliki lebih dari satu kategori (multi-label). Proses klasifikasi manual terbukti tidak efisien dan rawan kesalahan. Penelitian ini mengusulkan sistem klasifikasi multi-label berita berbasis Term Frequency–Inverse Document Frequency (TF-IDF) dan Logistic Regression yang dioptimalkan menggunakan ONNX Runtime untuk meningkatkan kecepatan inferensi. Dataset penelitian terdiri dari 10.058 artikel berita dari Kompas.com yang telah melalui tahap pra-pemrosesan dan pelabelan multi-label. Model dievaluasi menggunakan metrik F1-score mikro dan makro, serta perbandingan waktu inferensi antara model Scikit-learn (.pkl) dan model yang telah dioptimasi ONNX Runtime (.onnx). Hasil menunjukkan bahwa model ONNX mempertahankan akurasi yang sama (F1-mikro = 0,648; F1-makro = 0,402) dengan pengurangan waktu inferensi dari 0,88 detik menjadi 0,44 detik. Hal ini membuktikan bahwa ONNX Runtime dapat secara efektif meningkatkan efisiensi model NLP ringan untuk implementasi skala besar.

#### 22. Hardware Optimization for Generative AI
**Authors**: Dua, Patel
**Year**: 2024
**DOI**: 10.1007/979-8-8688-0917-0_3
**URL**: https://doi.org/10.1007/979-8-8688-0917-0_3

#### 23. Experiential neural architecture selection: dynamic cross-layer memory for real-time inference optimization
**Authors**: RODRIGUEZ
**Year**: 2025
**DOI**: 10.21203/rs.3.rs-7378044/v1
**URL**: https://doi.org/10.21203/rs.3.rs-7378044/v1

**Abstract**: Abstract
        Neural networks suffer from operational amnesia: they process each input as if it were the first time, without remembering which neuron combinations proved effective in similar contexts. We introduce ExNAS (Experiential Neural Architecture Selection), a system that performs real-time, neuron-granular architectural adaptation during the same inference by leveraging a distributed experiential memory. ExNAS records layer-wise neural fingerprints and lightweight contextual metadata and then performs transversal selection across non-consecutive layers under explicit per-layer and global budgets.
On a CPU proof-of-concept using a small CNN (2×Conv+FC), ExNAS delivers measurable time reductions (≈3.7–7.9%) and throughput gains (≈3.8–8.5%) at low active fractions (≈4.7–10.9%), without retraining. We detail the design, provide formal definitions, and discuss sensitivity to budgets and a negative case where heavier adaptation adds overhead. These results substantiate experience-guided, neuron-level conditional computation as a practical tool for real-time inference.

### From Semantic Scholar (17 citations)

#### 1. HyQ: Hardware-Friendly Post-Training Quantization for CNN-Transformer Hybrid Networks
**Authors**: Kim, Lee, Kim
**Year**: 2024
**DOI**: 10.24963/ijcai.2024/474
**URL**: https://doi.org/10.24963/ijcai.2024/474

**Abstract**: Hybrid models that combine CNNs and ViTs have recently emerged as state-of-the-art computer vision models. To efficiently deploy these hybrid models on resource-constrained mobile/edge devices, quantization is emerging as a promising solution. However, post-training quantization (PTQ), which does not require retraining or labeled data, has not been extensively studied for hybrid models. In this study, we propose a novel PTQ technique specialized for CNN-transformer hybrid models by considering the hardware design of hybrid models on AI accelerators such as GPUs and FPGAs. First, we introduce quantization-aware distribution scaling to address the large outliers caused by inter-channel variance in convolution layers. Furthermore, in the transformer block, we propose approximating the integer-only softmax with a linear function. This approach allows us to avoid costly FP32/INT32 multiplications, resulting in more efficient computations. Experimental results show that the proposed quantization method with INT8 precision demonstrated a 0.39% accuracy drop compared with the FP32 baseline on MobileViT-s with the ImageNet-1k dataset. Furthermore, when implemented on the FPGA platform, the proposed linear softmax achieved significant resource savings, reducing the look-up table and flip-flop usage by 1.8 ~ 2.1x and 1.3 ~ 1.9x, respectively, compared with the existing second-order polynomial approximation. The code is available at https://github.com/IDSL-SeoulTech/HyQ.

#### 2. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
**Authors**: Dettmers, Lewis, Belkada, Zettlemoyer
**Year**: 2022
**DOI**: 10.48550/arXiv.2208.07339
**URL**: https://doi.org/10.48550/arXiv.2208.07339

**Abstract**: Large language models have been widely adopted but require significant GPU memory for inference. We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance. With our method, a 175B parameter 16/32-bit checkpoint can be loaded, converted to Int8, and used immediately without performance degradation. This is made possible by understanding and working around properties of highly systematic emergent features in transformer language models that dominate attention and transformer predictive performance. To cope with these features, we develop a two-part quantization procedure, LLM.int8(). We first use vector-wise quantization with separate normalization constants for each inner product in the matrix multiplication, to quantize most of the features. However, for the emergent outliers, we also include a new mixed-precision decomposition scheme, which isolates the outlier feature dimensions into a 16-bit matrix multiplication while still more than 99.9% of values are multiplied in 8-bit. Using LLM.int8(), we show empirically it is possible to perform inference in LLMs with up to 175B parameters without any performance degradation. This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our software.

#### 3. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
**Authors**: Frantar, Ashkboos, Hoefler, Alistarh
**Year**: 2022
**DOI**: 
**URL**: https://www.semanticscholar.org/paper/7da0f2501034522e3d50af7e9b8fa7ec9d7b65b6

**Abstract**: Generative Pre-trained Transformer models, known as GPT or OPT, set themselves apart through breakthrough performance across complex language modelling tasks, but also by their extremely high computational and storage costs. Specifically, due to their massive size, even inference for large, highly-accurate GPT models may require multiple performant GPUs, which limits the usability of such models. While there is emerging work on relieving this pressure via model compression, the applicability and performance of existing compression techniques is limited by the scale and complexity of GPT models. In this paper, we address this challenge, and propose GPTQ, a new one-shot weight quantization method based on approximate second-order information, that is both highly-accurate and highly-efficient. Specifically, GPTQ can quantize GPT models with 175 billion parameters in approximately four GPU hours, reducing the bitwidth down to 3 or 4 bits per weight, with negligible accuracy degradation relative to the uncompressed baseline. Our method more than doubles the compression gains relative to previously-proposed one-shot quantization methods, preserving accuracy, allowing us for the first time to execute an 175 billion-parameter model inside a single GPU for generative inference. Moreover, we also show that our method can still provide reasonable accuracy in the extreme quantization regime, in which weights are quantized to 2-bit or even ternary quantization levels. We show experimentally that these improvements can be leveraged for end-to-end inference speedups over FP16, of around 3.25x when using high-end GPUs (NVIDIA A100) and 4.5x when using more cost-effective ones (NVIDIA A6000). The implementation is available at https://github.com/IST-DASLab/gptq.

#### 4. Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
**Authors**: Shen, Dong, Ye, Ma, Yao, Gholami, Mahoney, Keutzer
**Year**: 2019
**DOI**: 10.1609/AAAI.V34I05.6409
**URL**: https://doi.org/10.1609/AAAI.V34I05.6409

**Abstract**: Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accuracy gain for GLUE tasks, CoNLL-03 and SQuAD. However, BERT based models have a prohibitive memory footprint and latency. As a result, deploying BERT based models in resource constrained environments has become a challenging task. In this work, we perform an extensive analysis of fine-tuned BERT models using second order Hessian information, and we use our results to propose a novel method for quantizing BERT models to ultra low precision. In particular, we propose a new group-wise quantization scheme, and we use Hessian-based mix-precision method to compress the model further. We extensively test our proposed method on BERT downstream tasks of SST-2, MNLI, CoNLL-03, and SQuAD. We can achieve comparable performance to baseline with at most 2.3% performance degradation, even with ultra-low precision quantization down to 2 bits, corresponding up to 13× compression of the model parameters, and up to 4× compression of the embedding table as well as activations. Among all tasks, we observed the highest performance loss for BERT fine-tuned on SQuAD. By probing into the Hessian based analysis as well as visualization, we show that this is related to the fact that current training/fine-tuning strategy of BERT does not converge for SQuAD.

#### 5. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
**Authors**: Lin, Tang, Tang, Yang, Dang, Han
**Year**: 2023
**DOI**: 10.48550/arXiv.2306.00978
**URL**: https://doi.org/10.48550/arXiv.2306.00978

#### 6. Co-design of a TinyLLM using Programmable Logic and Software on an FPGA
**Authors**: Muller, Tyshka, Theisen, Hanna
**Year**: 2024
**DOI**: 10.1109/MWSCAS60917.2024.10658754
**URL**: https://doi.org/10.1109/MWSCAS60917.2024.10658754

#### 7. MixDiT: Accelerating Image Diffusion Transformer Inference With Mixed-Precision MX Quantization
**Authors**: Kim, Hwang, Oh, Park
**Year**: 2025
**DOI**: 10.1109/LCA.2025.3560786
**URL**: https://doi.org/10.1109/LCA.2025.3560786

**Abstract**: Diffusion Transformer (DiT) has driven significant progress in image generation tasks. However, DiT inferencing is notoriously compute-intensive and incurs long latency even on datacenter-scale GPUs, primarily due to its iterative nature and heavy reliance on GEMM operations inherent to its encoder-based structure. To address the challenge, prior work has explored quantization, but achieving low-precision quantization for DiT inferencing with both high accuracy and substantial speedup remains an open problem. To this end, this paper proposes MixDiT, an algorithm-hardware co-designed acceleration solution that exploits mixed Microscaling (MX) formats to quantize DiT activation values. MixDiTquantizes the DiT activation tensors by selectively applying higher precision to magnitude-based outliers, which produce mixed-precision GEMM operations. To achieve tangible speedup from the mixed-precision arithmetic, we design a MixDiTaccelerator that enables precision-flexible multiplications and efficient MX precision conversions. Our experimental results show that MixDiTdelivers a speedup of 2.10–5.32× over RTX 3090, with no loss in FID.

#### 8. X-SA: An Efficient Configurable Systolic Array Computing Architecture for GPGPU
**Authors**: Wang, Jia, Yang, Tan, Chang, Tie, Huang
**Year**: 2025
**DOI**: 10.1109/HPCC67675.2025.00039
**URL**: https://doi.org/10.1109/HPCC67675.2025.00039

**Abstract**: GPGPUs are pivotal for edge AI, but resource constraints demand efficient low-precision computation. Conventional GPGPUs face challenges in resource utilization, particularly with irregular matrices common in AI, and memory bandwidth limitations on edge devices. Traditional fixed-size systolic arrays often suffer from underutilization under varying workloads. This paper introduces X-SA, a configurable systolic array architecture tailored for INT8 matrix multiplication on GPGPUs in resource-constrained edge environments. X-SA distinctively employs a parameterized $2 \times N$ processing element design enabling dynamic computational scaling, unlike fixed systolic arrays. It integrates an interleaved matrix buffer to alleviate memory bottlenecks and optimize dataflow. Experimental results demonstrate X-SA achieves a $2.83 \times$ performance speedup over the Vortex baseline with minimal Look-Up Table overhead of 2.8% and Flip-Flops overhead of 1.4%.. It offers comparable performance to a standard $4 \times 4$ systolic array but with significantly reduced area by 46.26% and power by 39.42%, and superior processing element utilization for irregular matrices. X-SA provides a approach to help improve the performance of some AI applications running on edge GPGPUs relatively in resource-constrained environments.

#### 9. Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency
**Authors**: Husom, Goknil, Astekin, Shar, KÃ¥sen, Sen, Mithassel, Soylu
**Year**: 2025
**DOI**: 10.1145/3767742
**URL**: https://doi.org/10.1145/3767742

**Abstract**: Deploying Large Language Models (LLMs) on edge devices presents significant challenges due to computational constraints, memory limitations, inference speed, and energy consumption. Model quantization has emerged as a key technique to enable efficient LLM inference by reducing model size and computational overhead. In this study, we conduct a comprehensive analysis of 28 quantized LLMs from the Ollama library, which applies by default Post-Training Quantization (PTQ) and weight-only quantization techniques, deployed on an edge device (Raspberry Pi 4 with 4 GB RAM). We evaluate energy efficiency, inference performance, and output accuracy across multiple quantization levels and task types. Models are benchmarked on five standardized datasets (CommonsenseQA, BIG-Bench Hard, TruthfulQA, GSM8K, and HumanEval), and we employ a high-resolution, hardware-based energy measurement tool to capture real-world power consumption. Our findings reveal the trade-offs between energy efficiency, inference speed, and accuracy in different quantization settings, highlighting configurations that optimize LLM deployment for resource-constrained environments. By integrating hardware-level energy profiling with LLM benchmarking, this study provides actionable insights for sustainable AI, bridging a critical gap in existing research on energy-aware LLM deployment.

#### 10. Efficient Deep Learning Framework for Fire Detection in Complex Surveillance Environment
**Authors**: Dilshad, Khan, Song
**Year**: 2023
**DOI**: 10.32604/csse.2023.034475
**URL**: https://doi.org/10.32604/csse.2023.034475

#### 11. Hardware-Software Co-Design for Efficient LLM Inference on PCIe-Based FPGAs Using Coarse-Grained Systolic Arrays
**Authors**: Chang
**Year**: 2025
**DOI**: 10.1109/SOCC66126.2025.11235351
**URL**: https://doi.org/10.1109/SOCC66126.2025.11235351

**Abstract**: Inference of large language models (LLMs) requires extensive matrix-multiplication operations that place heavy demands on both compute and memory resources. Traditional systolic arrays (SAs), typically built from scalar multiply-accumulate processing elements (PEs), exhibit low compute density and high routing overhead when scaled to support the large matrix dimensions of transformer-based models. To address these limitations, we present a hardware-software co-design approach that introduces a coarse-grained "Big PE" architecture for FPGA acceleration. Each Big PE performs a complete 4 × 4 matrix multiplication per clock cycle using sixteen parallel multipliers and local accumulators, significantly reducing inter-PE communication and flip-flop usage. We implement a 2×32 array of these Big PEs on a Xilinx Alveo U55 FPGA and tightly integrate the design into the open-source Llama2 inference framework using an XDMA PCIe–AXI bridge. Experimental evaluation with an 8-bit quantized TinyStories-15M LLM shows more than 2.15× improvement in token processing speed during the prompt stage compared to a CPU-only baseline, with no degradation in output quality. Under a fixed budget of 4,096 DSP multipliers, our SA reduces the cycle count for 64 × 64 matrix multiplication from 190 to approximately 120 cycles and cuts total flip-flop usage from 194,560 to 45,056. These results highlight the benefits of hardware-software co-design in deploying LLM inference on resource-constrained FPGA platforms and suggest promising directions for optimizing on-chip memory hierarchies and enabling multi-FPGA scalability.

#### 12. Guest Editorial Introduction to the Special Section on the 2025 IEEE International Solid-State Circuits Conference (ISSCC)
**Authors**: Akita, Miyaji, Ryu, Babaie, Reiskarimian
**Year**: 2025
**DOI**: 10.1109/jssc.2025.3626462
**URL**: https://doi.org/10.1109/jssc.2025.3626462

#### 13. Semantic Segmentation of Brain Tumors: A Performance Evaluation Using DeepLabV3+, UNet, and Intel’s OpenVINO Toolkit
**Authors**: Kapo, Akagić, Buza
**Year**: 2024
**DOI**: 10.1109/CoDIT62066.2024.10708196
**URL**: https://doi.org/10.1109/CoDIT62066.2024.10708196

**Abstract**: Artificial intelligence, Machine Learning, and Deep Learning are increasingly making significant contributions to the field of medicine. Individual patient conditions, disease localization, and various influencing factors underscore the complexity of disease diagnosis and treatment planning. Introducing new technologies can revolutionize medical diagnostics, facilitating swift and accurate assessments. Among the noninvasive diagnostic methods, Magnetic Resonance Imaging (MRI) stands out, particularly in tumor diagnosis. UNet, renowned for its effectiveness in medical image analysis, serves as a robust model for semantic segmentation, as does DeepLabV3+. However, these models are inherently complex, and their inference process can be time-consuming. By leveraging the OpenVINO toolkit, the inference process is significantly reduced. In this study, nearly a 2-fold acceleration is achieved in inference time with the DeepLabV3+ model and a roughly 1.2-fold improvement with the UNet model on CPU. Moreover, when employing GPU with FP16 precision, the acceleration reached almost 2.5fold for UNet and nearly 3-fold for DeepLabV3+, showcasing the substantial performance enhancements attainable through optimized hardware utilization.

#### 14. Edge Artificial Intelligence for Real-Time Decision Making using NVIDIA Jetson Orin, Google Coral Edge TPU and 6G for Privacy and Scalability
**Authors**: Dr.J.V.Anchitaalagammai, Dr.S.Kavitha, R.Buurvidha, T.S.Santhiya, Roopa, Sankari
**Year**: 2025
**DOI**: 10.1109/ICVADV63329.2025.10960953
**URL**: https://doi.org/10.1109/ICVADV63329.2025.10960953

**Abstract**: Edge AI or Edge Artificial Intelligence is the provision of AI within edge devices itself without needing a connection to the cloud. AI gets better with more information and time, and so a lot of AI models “live in the cloud.” However, AI made Edge devices is more smarter by deploying models onboard. There are several factors like NVIDIA's Jetson Orin, Google's Coral Edge TPU, and Qualcomm's AI optimized Snapdragon platforms, which has accelerated the adoption of Edge AI and made these onboard Edge devices with modests resources, robust processing capabilities. Experts anticipate that the advent of 6G technology will further expand speeds, making it possible to utilize augmented (AR) and virtual (VR) reality as well as the Internet of Things (IoT). Privacy of the end user is still safeguarded through federated and differential learning which enables intelligent distribution while data is kept close to the source. The concept of optimization techniques such as quantization, pruning, and NAS are resources that help in considerations for deploying AI models at the Edge as highlighted in the literature. Containerization and microservices also provide a great ecosystem for Edge AI deployments supported by Docker, Kubernetes and KubeEdge which provide scalable infrastructure. Evaluations conducted on various use cases rangingfrom predictive maintenance of industrial IoT[l], real- time health monitoring as well as autonomous systems, show the performance of these models to improve in terms of model accuracy, inference computation as well as energy efficiency through the enhanced methodologies. The recent advancements in hardware and software capabilities have placed Edge AI as a preferred option for many applications that have strict latency requirements in addition to needing to make real time decisions. Edge AI is set to revolutionize industries likehealth care, smart cities, industrial automation despite hitches like data unification and barrier from privacylaws. Further studies is suggested to improve on energy management, security measures and reduce barriers to interoperability so as to optimize the chancesof Edge AI in many applications.

#### 15. Performance Analysis and Fronthaul Quantization Bits Allocation in Cell-free ISAC System
**Authors**: Zhu, Deng, Xin, Xia, Han, Wang, Wang
**Year**: 2025
**DOI**: 10.1109/ICCC65529.2025.11148976
**URL**: https://doi.org/10.1109/ICCC65529.2025.11148976

**Abstract**: In cell-free integrated sensing and communication system, the substantial information transmission need from access points to central processing unit for centralized communication decoding and sensing data fusion places a heavy burden on the fronthaul links and the finite fronthaul capacity in practice may cause severe performance degradation. To address this, we first analyze sensing and communication performance under fronthaul constraint and then formulate a multi-objective optimization problem to maximize both sensing and communication performance by allocating the fronthaul quantization bits. Considering about the non-convex nature of the bit allocation problem, we adopt a meta-heuristic algorithm, the non-dominated sorting genetic algorithm III to solve it. Numerical results validate our theoretical analysis and demonstrate the effectiveness of the bit allocation algorithm. Also, the results highlight the tradeoffs between sensing and communication under finite fronthaul capacity and suggest optimal sensing strategies for varying fronthaul capacities.

#### 16. Search-optimized quantization in biomedical ontology alignment
**Authors**: Bouaggad, Grabar
**Year**: 2025
**DOI**: 10.3389/frai.2025.1662984
**URL**: https://doi.org/10.3389/frai.2025.1662984

**Abstract**: In the fast-moving world of AI, as organizations and researchers develop more advanced models, they face challenges due to their sheer size and computational demands. Deploying such models on edge devices or in resource-constrained environments adds further challenges related to energy consumption, memory usage and latency. To address these challenges, emerging trends are shaping the future of efficient model optimization techniques. From this premise, by employing supervised state-of-the-art transformer-based models, this research introduces a systematic method for ontology alignment, grounded in cosine-based semantic similarity between a biomedical layman vocabulary and the Unified Medical Language System (UMLS) Metathesaurus. It leverages Microsoft Olive to search for target optimizations among different Execution Providers (EPs) using the ONNX Runtime backend, followed by an assembled process of dynamic quantization employing Intel Neural Compressor and IPEX (Intel Extension for PyTorch). Through our optimization process, we conduct extensive assessments on the two tasks from the DEFT 2020 Evaluation Campaign, achieving a new state-of-the-art in both. We retain performance metrics intact, while attaining an average inference speed-up of 20x and reducing memory usage by 70%.

#### 17. Latency-Critical Quantized Inference With Transformer Decoders on ARM and RISC-V CPUs
**Authors**: Martínez, Catalán, Castelló, Mestre, Quintana-Ortí
**Year**: 2025
**DOI**: 10.1109/JIOT.2025.3560382
**URL**: https://doi.org/10.1109/JIOT.2025.3560382

**Abstract**: Large language models are transforming industries but face challenges due to their high computational and energy demands. Model compression via quantization mitigates these barriers by reducing the bit precision of parameters and arithmetic operations, enabling deployment on resource-constrained devices like smartphones and edge platforms. This article focuses on quantization applied to transformer decoders, which are critical for tasks, such as text generation and conversational artificial intelligence. Unlike encoders, decoders are constrained by memory due to their sequential processing nature and low arithmetic intensity. We propose optimizations targeting inference on low-power CPUs, emphasizing efficient linear layers with quantized data/arithmetic and cache optimization. Using two representative ARM and RISC-V platforms, we present optimized mixed-precision implementations of the matrix multiplication that outperform the instance of that computational kernel in popular libraries, such as basic linear algebra subprograms infrastructure software, XNNPACK and ARMCL. This work thus advances the understanding of the impact of quantization on transformer decoder efficiency, energy consumption and precision in edge environments.

### From Gemini Grounded (3 citations)

#### 1. mit.edu
**Authors**: MIT
**Year**: 2026
**DOI**: 
**URL**: https://mitibmwatsonailab.mit.edu/people/song-han/

#### 2. mit.edu
**Authors**: MIT
**Year**: 2026
**DOI**: 
**URL**: https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00704/125482/A-Survey-on-Model-Compression-for-Large-Language

#### 3. gartner.com
**Authors**: Gartner
**Year**: 2026
**DOI**: 
**URL**: https://www.gartner.com/en/newsroom/press-releases/2024-05-29-gartner-forecasts-worldwide-artificial-intelligence-chips-revenue-to-grow-33-percent-in-2024

---

## Failed Topics

The following topics did not return valid citations:

- peer-reviewed studies on integer-only quantization for LLMs
- INT8 vs INT4 quantization accuracy trade-offs in LLMs
- SmoothQuant: accurate and efficient post-training quantization for LLMs
- integer-only inference for vision transformers and LLMs
- memory bandwidth optimization for quantized transformer models
- ZeroQuant: efficient and affordable post-training quantization for large-scale transformers
- dyadic arithmetic for integer-only deep learning inference
- title:integer-only inference transformers
- mixed-precision vs integer-only arithmetic for LLM inference
- calibration datasets for post-training quantization of LLMs
- ARM Ethos-U NPU quantization support technical reference
- NVIDIA TensorRT integer quantization best practices white paper
- Google TensorFlow Lite quantization specification for microcontrollers
- BCG analysis of semiconductor market for AI inference chips
- Deloitte insights on sustainable AI and energy efficient computing
- OECD policy paper on eco-friendly AI computing infrastructure
- European Commission regulations on AI energy consumption
- NIST framework for artificial intelligence hardware evaluation
- ISO/IEC JTC 1/SC 42 Artificial Intelligence standards quantization
- MLPerf inference benchmarks for integer quantized models
- TinyML foundation white papers on LLMs for microcontrollers
- comparison of floating-point vs fixed-point inference energy cost
- challenges of deploying quantized LLMs on mobile hardware
- lossless vs lossy quantization for large language models
- logarithmic number system for LLM quantization
- bit-serial computing for transformer acceleration
- impact of quantization on LLM reasoning capabilities
- sparsity and quantization co-optimization for LLMs
- hardware-aware neural architecture search for quantized transformers
- lookup table based quantization for non-linear functions in LLMs
- bias correction in integer-only quantization
- stochastic rounding in low-precision training
- block-wise quantization techniques for large models
- vector quantization for LLM weight compression
- dynamic vs static quantization for LLM inference
- per-channel vs per-tensor quantization for transformers
- simulated quantization vs hardware-in-the-loop evaluation
- latency analysis of quantized LLMs on CPU vs GPU vs NPU
- cost analysis of serving quantized LLMs at scale
- environmental impact of large model quantization studies
- security implications of quantized neural networks
- robustness of quantized LLMs to adversarial attacks
- patent landscape for neural network quantization technologies
- startups developing integer-only AI accelerator chips
- Sambanova systems reconfigurable dataflow for quantized AI
- Groq LPU architecture and deterministic integer performance
- Cerebras wafer-scale engine quantization support
- Mythic analog compute-in-memory vs digital integer logic
- IBM research low-precision AI hardware roadmap
- Samsung NPU architecture for on-device LLM
- Apple Neural Engine quantization specifications
- Huawei Da Vinci architecture quantization capabilities
- MediaTek APU quantization support for generative AI
- Open source libraries for LLM quantization (bitsandbytes, auto-gptq)
- RISC-V vector extension for quantized neural networks
- GGML library integer quantization format
- llama.cpp quantization implementation details
- ExLlamaV2 kernel optimization for quantized inference
- TVM compiler stack for integer-only hardware
- MLIR compiler infrastructure for quantization
- Apache TVM quantization automated search
- deep learning compiler optimizations for INT8
- survey of quantization for generative AI on mobile
- fine-tuning quantized LLMs (QLoRA) methodology
- post-training quantization for mixture-of-experts (MoE) models
- impact of quantization on hallucination rates in LLMs
- quantization of attention mechanisms in transformers
- integer-only implementation of LayerNorm and RMSNorm
- precision-scalable hardware architectures for AI
- World Economic Forum report on green computing and AI
- Brookings Institution report on AI hardware supply chain