relation: https://khub.utp.edu.my/scholars/20421/
title: Application Integration Framework for Large Language Models
creator: Ho, Joe Ee
creator: Ooi, Boonyaik Yaik
creator: Westner, Markus K.
description: Large Language Models (LLMs) have unlocked new opportunities in processing non-structured information. However, integrating LLM into conventional applications poses challenges due to their non-deterministic nature. This paper introduces a framework designed to effectively integrate LLM into intermediate modules by ensuring more consistent and reliable outputs. The framework includes three key components: the Sieve, which captures and retries processing of incorrect outputs; the Circuit Breaker, which stops processing persistently incorrect outputs; and the Optimizer, which enhances processing efficiency by combining inputs into single prompts. Experimental results employing structured methodology demonstrate the framework's effectiveness, achieving significant improvements a 71.05 reduction in processing time and an 82.97 reduction in token usage while maintaining high accuracy. The proposed framework, agnostic to specific LLM implementations, aids the integration of LLMs into diverse applications, enhancing automation and efficiency in fields such as finance, healthcare, and education. © 2024 IEEE.
publisher: Institute of Electrical and Electronics Engineers Inc.
date: 2024
type: Conference or Workshop Item
type: PeerReviewed
identifier:   Ho, Joe Ee and Ooi, Boonyaik Yaik and Westner, Markus K.  (2024) Application Integration Framework for Large Language Models.  In: UNSPECIFIED.     
relation: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85209631691&doi=10.1109%2FAiDAS63860.2024.10730541&partnerID=40&md5=e6c7b763073dd52c258251f2542e1756
relation: 10.1109/AiDAS63860.2024.10730541
identifier: 10.1109/AiDAS63860.2024.10730541