Four Sample LLM Applications The products best suited for LLM adoption have one aspect in common: textual data is a major factor. Here I look at four examples, inspired by real-world LLM applications, that serve to further illustrate how companies can move from problem to LLM solution. You’ll see that they span different industries, pain points, flavors of LLM, and user groups.
- Sophisticated Recommendations for Legal Professionals For lawyers working on a case, it’s critical to review all relevant legal cases and documents. Consider a company that has a large database of legal texts—including past cases and decisions—available to its subscribers, most of whom are legal professionals. The company wants to enhance its service by adding a feature that displays a list of related cases and documents on its platform. This tool is designed to help lawyers find and review relevant cases more quickly and effectively, thereby allowing them to take on more cases or improve the quality of their work by catching details they might otherwise miss. However, because new cases are constantly being added to the database, this list of relevant cases cannot be static, but must be defined dynamically.
The solution is a recommendation system based on semantic similarity. Semantic similarity uses small or large language models to determine the closeness of meaning between two documents based on an abstract representation of their content, rather than the vocabulary they use. This helps to identify two semantically similar documents even if they use very different words. In addition to the search functionality and its integration into a user-facing frontend, the team tasked with building the application also needs to architect and schedule the preprocessing of incoming documents. This process, known as indexing, is accomplished by the same language model.
- Conversational AI for Technical Documentation For developers, clear and accessible software documentation is critical—it is the guide that helps them use software or libraries effectively. But navigating documentation can be daunting. Consider the massive scale of cloud service providers like Amazon Web Services (AWS) or Azure: their documentation can run to thousands of pages, covering a wide range of services, features, and policies. Faced with such a sea of information, users quickly feel overwhelmed and often turn to browsing the web for answers rather than sifting through official documentation. To address this problem, a software company wants to revolutionize the way customers interact with their documentation. They plan to introduce an intuitive search interface that allows users to ask questions about the codebase in natural language and quickly find accurate answers. This not only streamlines the process, but also ensures that developers are getting the most accurate and relevant information directly from the source.
The natural language capabilities of an LLM are ideal for this use case. However, the team doesn’t want the LLM to generate answers based on the knowledge it learned during training: the documentation was probably not part of that training data—and even if it was, that information could become outdated with the next software update. To compound the problem, LLMs are notoriously bad at understanding the limits of their own knowledge. Thus, when asked about things absent from their training data, they often answer regardless—by inventing facts. This is known as “hallucinating,” and it is a major problem in LLM adoption.
The remedy to outdated information in the LLM’s parameters and ensuing inflation of hallucinations is a method known as retrieval-augmented generation (RAG). In such a setup, the LLM is preceded by a retrieval module, which extracts the candidate documents it deems most suitable to answer the user query from your database. Upon receiving a query, the RAG system first identifies suitable documentation pages. It then embeds those in the prompt to the LLM, instructing it to base its answers on the fact-checked information from the database. RAG as an LLM technique is extremely popular because of its ability to create a factual knowledge base on the fly.
- Automating the Collection of Information from Earnings Reports The advent of LLMs has enabled machines to process unstructured data. The term “unstructured” refers to data types such as images, audio, video, or text: formats that don’t follow a strictly predefined structure. Structured data, on the other hand, is data that comes in a predictable format, such as tables and graphs, and can be processed using less resource-intensive methods. For example, a large table can be queried using SQL, which is faster, more accurate, and infinitely cheaper than running an LLM for the same task.
Let’s say a company wants to identify information in its unstructured textual data that can be fed into such tables. For example, they might want to extract specific numerical and other factual data points from a collection of earnings reports.
The solution: using a smaller language model to mine text for information, not by generating answers, but by highlighting it in the underlying source document. Such models are called “extractive” language models. They’re not only lighter and cheaper than LLMs but also safer for highly sensitive areas such as finance. That’s because they are incapable of LLM-like hallucinations, and they are necessarily more faithful to the underlying dataset.
- Condensing Political Discourse for News Consumers Democracy thrives on the active participation of its citizens. But political debates in parliament are often long and difficult to access. For this use case, let’s imagine a government application that wants to make parliamentary debates more accessible to citizens by summarizing the debates’ transcripts according to the user’s interest.
This use case is similar to the second in that we’re dealing with a database of texts that is updated periodically. And indeed, to address it, we would again use a retrieval module that extracts the relevant transcripts upon receiving a user query. But instead of generating answers, this system would use an LLM to summarize all the underlying texts. In this way, users could get timely overviews of political debates tailored to their individual interests.
Resources: