ai-orchestration.md

The main components are:

1. Models. The model is simply the type of LLM we decide to embed in our application. There are two main categories of models:
2. Proprietary LLMs->models that are owned by specific companies or organizations. Examples include GPT-3 and GPT-4 developed by OpenAI or Bard, developed by Google. As their source code and architecture is not available, those models cannot be re-trained from scratch on custom data, yet they can be fine-tuned if needed.
3. Open-source->models with code and architecture freely available and distributed, hence they can also be trained from scratch on custom data. Examples include Falcon LLM, developed by Abu Dhabi’s Technology Innovation Institute (TII), or LLaMA, developed by Meta.
4. Memory. LLM applications commonly use a conversational interface, which requires the ability to refer back to earlier information within the conversation. This is achieved through a "memory" system that allows the application to store and retrieve past interactions. Note that past interactions could also constitute an additional non-parametric knowledge to be added to the model. To achieve that, it is important to store all the past conversations – properly embedded – into the VectorDB which is at the core of applications data.
5. A VectorDB is a type of database that stores and retrieves information based on vectorized embeddings, the numerical representations that capture the meaning and context of text. By using VectorDB, you can perform semantic search and retrieval based on the similarity of meanings rather than keywords. VectorDB can also help LLMs generate more relevant and coherent text by providing contextual understanding and enriching generation results. Some examples of vectorDB are Chroma, FAISS, Elastic Search, Milvus, Pinecone, Qdrant, and Weaviate.
6. Plug-ins. They can be seen as additional modules or components that can be integrated into the LLM to extend its functionality or adapt it to specific tasks and applications. These plug-ins act as add-ons, enhancing the capabilities of the LLM beyond its core language generation or comprehension abilities. The idea behind plug-ins is to make LLMs more versatile and adaptable, allowing developers and users to customize the behavior of the language model for their specific needs. Plug-ins can be created to perform various tasks, and they can be seamlessly incorporated into the LLM's architecture.
7. Prompts. This is probably the most interesting and pivotal component of an LLM-powered application.
8. “Frontend”, or what the user sees. A “prompt” refer to the input to the model. It is the way the user interact with the application, asking things in natural language;
9. “Bakcend”, or what the user does not see. Natural Language not only is the way to interact, as an user, with the front-end, but also is it the way we “program” the backend. In fact, on top of the user’s prompt, there are many natural language instructions, or meta-promts, that we give to the model so that it can properly address user’s query. Meta-prompts are meant to instruct the model to act as it is meant to. For example, if we want to limit our application to answer only to questions related to the documentatios we provided in the VectorDB, we will specify in our meta-promts to the model: “Answer only if the question is related to the provided documentation”.