Motivation

There has been huge excitement around large language models since recently, and as the pioneer of civil technologies, Boston’s MONUM wants to explore the potential use of LLMs in the public sector. The flexibility and scalability of LLMs makes it a natural candidate for text retrieval and Q&A for government workers, when large and sometimes divided public organizations fail to communicate information between themselves.

Due to the nature of the work in the public sector, the responses generated by the model must be credible and the source must be traceable. Part of the problem can be solved by customizing the knowledge base of the LLM. However, even with a custom knowledge base, the language models can still generate inaccurate/misleading responses. This describes the main challenge of the project: building a retrieval-augmented generative AI app that guarantees a high level of accuracy and credibility.