Variables implementation

[tomlynchRNA]

re #490

[tomlynchRNA]

Some important scenarios we need to consider for variables, all closely related:

• Dynamic session variables

Dynamic variables would be variables that can have their value set or updated during a session. For example, a process app may have a task description that uses a variable e.g. {question_answer} where that answer is taken from a previous tasks human input question, see next point.

• Human input variable output

Human input tasks should allow the selection of a named variable (or a variable per question for "form" human inputs) that the response to the human input will be saved into, and be accessible throughout the rest of the sessions prompts (tasks, agent role/goal, tool descriptions etc).

• Handling dynamic variables in agent-backend

Variables may need to change after a crew is "built" or after tools, etc are instantiated. i.e we can't just whatever.prompt.format(**variables) once, we need to do it when ever the variables are changed, or perhaps every time that prompt, task, tool, etc is used.

• (specifically for RAG tools) Ability to define "filters" in the tool parameters section. This is different from "parameters" (input to the function), and environment variables, but these are filters e.g. WHERE column = '{our_variable}', so we need to define key, type of filter, value OR a json input box to input it raw, also must support variables. Variables would not be mandatory for this. We might break this into a separate ticket, but I mention it here because it's closely related.

---

And as a side note, we should consider how to improve human input in chat apps to allow more formatted inputs and the ability to direct outputs for some responses to variables. This is difficult because of the lack of tasks in chat apps. @anada10

[tomlynchRNA]

Agreed that we would need to store the original prompt w/ placeholders still in there as a first step.

And we can think about where to call interpolate_inputs again later 👍

[iandjx]

It would make sense to update the variables during the task callback. For us to easily determine which variable to update, we would have to use structured output feature of expected output so that we can easily parse what variables need to be updated

[tomlynchRNA]

We would already have the exact answer to the human input question(s) and variable associated with it from the form, so we can handle those however we want without necessarily requiring structured output, I think 🙂

[iandjx]

Okay. But what about task output as arguments to change session variables?

[tomlynchRNA]

Ah, I see what you mean. Well I guess we could have both 😅 UX is another story, but both are possible 👍

[iandjx]

What is the advantage of using dynamic session variables vs using human input/task output as context for the next tasks? Can we have some examples on when we would want to update the session variables mid-session?

[anada10]

Context is a crew/process app specific feature @iandjx.. we are designing a feature that will work across the platform across all parts of the app in a reliable and consistent manner, not just something for one subcomponent.

[tomlynchRNA]

another thing, right now if we ask a user to answer human input for e.g. "what is your name", in a later prompt, we rely 100% on LLM to know the users name and interpolate it into some answer, tool call parameter, etc later on. If this were a variable we have better control over it and could precisely take the a variable and use it as the input in later stages.

[anada10]

The advantage of a feature like this is quite evident when attempting to conduct a reliable RAG search.
For example, storing of variables would allow users to set form fields to be equal to metadata filters on a RAG search.
This is more reliable than using approaches like self query retrieval which tries to invent what value should be passed to the Vector store.

[iandjx]

another thing, right now if we ask a user to answer human input for e.g. "what is your name", in a later prompt, we rely 100% on LLM to know the users name and interpolate it into some answer, tool call parameter, etc later on. If this were a variable we have better control over it and could precisely take the a variable and use it as the input in later stages.

this makes sense. @anada10 showed me issues before where the AI asked redundant questions even when context was provided earlier

[tomlynchRNA]

RE:

Can we use the extra search_params/kwargs (forgot the exact name) in langchain vector stores to use these variables in filters in a simple way, or d owe have to construct the filter in a vector-db-specific way? @tomlynchRNA looking into this

I did some experimenting of where to pass the filters and got it working. It was a little tricky because of multiple versions of docs with similar/confusing param names, and that we don't call the method which takes the filter directly in the self query retriever, but we can fix that.

For "similarity search", this POC works:

diff --git a/agent-backend/src/tools/retrievers/similarity_search.py b/agent-backend/src/tools/retrievers/similarity_search.py
index 214231a1..3120391f 100644
--- a/agent-backend/src/tools/retrievers/similarity_search.py
+++ b/agent-backend/src/tools/retrievers/similarity_search.py
@@ -15,5 +15,13 @@ class SimilaritySearchRetriever(BaseRetriever):
     def _get_relevant_documents(self, query: str, *, run_manager: CallbackManagerForRetrieverRun):
         embedded_question = self.embedding.embed_query(query)
         if isinstance(self.vector_store, Qdrant):
-            return self.vector_store.similarity_search_with_score_by_vector(embedded_question, k=self.k)
+            from qdrant_client.http import models
+            return self.vector_store.similarity_search_with_score_by_vector(embedded_question, k=self.k, filter=models.Filter(
+                must=[
+                    models.FieldCondition(
+                        key="Retweets",
+                        match=models.MatchValue(value="11097"),
+                    )
+                ]
+            ))
         return self.vector_store.similarity_search_by_vector_with_score(embedded_question, k=self.k)

Just hardcodes to find a specific value, uses Must. So we could construct those filter field conditions based on some configuration in the tool that incorporates variables 😎

[iandjx]

I tried using context in this example and it worked as expected. The final output took into consideration my name, address, and email when generating the final output

import json
from crewai import Agent, Task, Crew
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

# Define agents
user_info_agent = Agent(
    role='User Information Collector',
    goal='Collect user information including name, address, and email',
    backstory='An agent dedicated to gathering essential user information for further processing.',
)

email_generator_agent = Agent(
    role='Email Generator',
    goal='Generate an email to invite the user to talk about climate change',
    backstory="An agent dedicated to generating emails for the user's climate change inquiry.",
)

proofreader_agent = Agent(
    role='Proofreader',
    goal='Proofread and improve the email',
    backstory='An agent dedicated to proofreading and improving the email for the user.',
)

geopolitical_agent = Agent(
    role='Geopolitical Analyst',
    goal='Append geopolitical climate information based on the user’s country',
    backstory='An agent dedicated to appending geopolitical climate information based on the user’s country.',
)

# Define tasks
collect_user_info_task = Task(
    description='Ask the user for their name, address, and email',
    expected_output='User information including name, address, and email',
    agent=user_info_agent,
    human_input=True,
)

generate_email_task = Task(
    description='Generate an email to invite the user to talk about climate change. Add information about climate change.',
    expected_output='Generated email with climate change information',
    agent=email_generator_agent,
    context=[collect_user_info_task],
)

proofread_email_task = Task(
    description='Proofread the email to check for errors and improve it further',
    expected_output='Proofread and improved email',
    agent=proofreader_agent,
    context=[generate_email_task, collect_user_info_task],
)

append_geopolitical_info_task = Task(
    description='Append the geopolitical climate based on the user’s country fetched from their address',
    expected_output='Email with appended geopolitical climate information',
    agent=geopolitical_agent,
    context=[proofread_email_task, generate_email_task, collect_user_info_task],
)

# Create the crew
crew = Crew(
    agents=[user_info_agent, email_generator_agent, proofreader_agent, geopolitical_agent],
    tasks=[collect_user_info_task, generate_email_task, proofread_email_task, append_geopolitical_info_task],
    process='sequential',
)

# Kickoff the crew
crew_output = crew.kickoff()
print(crew_output)