Efficient Test-Time Compute Scaling for Resource-Constrained Large  Language Models

GAURAV KUMAR SINHA

Efficient Test-Time Compute Scaling for Resource-Constrained Large Language Models

Authors

GAURAV KUMAR SINHA

Keywords:

Test-Time Compute Scaling, Resource-Constrained Inference, Large Language Models, Adaptive Sampling, Verifier-Guided Search, Surrogate Modeling, Federated Learning, Edge Intelligence

Abstract

Spending more computation at inference time, by sampling several reasoning chains, voting, or searchingwith a verifier, is now a proven way to make language models more accurate without making them larger. The technique is least affordable where it is most needed. Small models running

References

1: A. Mahendra Vardhan and S. Sridhar, "Determining False Positive Analysis of Software Vulnerabilities with Predefined Scan Rules using Random Forest Classifier and Decision Tree Technique," 2022 4th International Conference on Advances in Computing, Communication Control and Networking (ICAC3N), Greater Noida, India, 2022, pp. 622-625, DOI: https://doi.org/10.1109/ICAC3N56670.2022.10074458

Downloads

Published

2024-01-20

How to Cite

GAURAV KUMAR SINHA. (2024). Efficient Test-Time Compute Scaling for Resource-Constrained Large Language Models. Journal of Computational Analysis and Applications (JoCAAA), 32(1), 1589–1605. Retrieved from https://eudoxuspress.com/index.php/pub/article/view/5917