Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Method add_to_existing implemented #45

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

havent-invented
Copy link

Retrieval with functionality of adding new documents implemented. Method add_to_existing implemented in RetrievalAugmentation

…hod add_to_existing implemented in RetrievalAugmentation
@jalotra
Copy link

jalotra commented Jun 17, 2024

@havent-invented
The update would not be that simple, the tree building phase does this :

1. Starts with leaf nodes. 
2. Creates summarisations on top of those leaf nodes and do this recursively upwards.
3. Stops at some layer `L` which it > 0, 0 being the leaf nodes layer.

Now let's see what would happen in merge operation, say merge(Tree A, Tree B)

1. The summarisation op would have to be done again.
2. Because the tree creation is inherently pivoted on combining some `Nodes : N` based on max of some metric using UMAP.

@jytheawesome
Copy link

jytheawesome commented Nov 20, 2024

Hey, @parthsarthi03, the add_to_existing feature would be insanely useful for my use case (adding documents to an existing RAG tree). I see that this branch is implementing this feature and that @ET-yzk has already approved it. Not sure if it is ready to be merged :)

Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants