…
comment: The repository administrator has not yet configured an RDF license.
primary topic: Application Integration Framework for Large Language Models

Application Integration Framework for Large Language Models
type: AcademicArticle
type: Article
abstract: Large Language Models (LLMs) have unlocked new opportunities in processing non-structured information. However, integrating LLM into conventional applications poses challenges due to their non-deterministic nature. This paper introduces a framework designed to effectively integrate LLM into intermediate modules by ensuring more consistent and reliable outputs. The framework includes three key components: the Sieve, which captures and retries processing of incorrect outputs; the Circuit Breaker, which stops processing persistently incorrect outputs; and the Optimizer, which enhances processing efficiency by combining inputs into single prompts. Experimental results employing structured methodology demonstrate the framework's effectiveness, achieving significant improvements a 71.05 reduction in processing time and an 82.97 reduction in token usage while maintaining high accuracy. The proposed framework, agnostic to specific LLM implementations, aids the integration of LLMs into diverse applications, enhancing automation and efficiency in fields such as finance, healthcare, and education. © 2024 IEEE.
date: 2024
status: peerReviewed
status: published
author list: authors
type: EPrint
type: ConferenceItemEPrint
is part of: repository

Institute of Electrical and Electronics Engineers Inc.
type: Organization

Markus K. Westner
type: Person
given name: Markus K.
family name: Westner

Boonyaik Yaik Ooi
type: Person
given name: Boonyaik Yaik
family name: Ooi

Joe Ee Ho
type: Person
given name: Joe Ee
family name: Ho

HTML Summary of #20421 Application Integration Framework for Large Language Models
format: text/html
primary topic: Application Integration Framework for Large Language Models