Hybrid Queries: What is the default value of Theta, in Theta.K

[rsrs2013]

Hello Milvus Community,

In the paper titled 'Milvus: A Purpose-Built Vector Data Management System',
it is specified that, to support Hybrid Queries, and fetch Top-K results, θ-K vectors are searched for.
May I know what is the

1. default value of θ,
2. how can I change it and
3. where can I find the default value of θ in the code base.

Also I have given the link to the paper as well for any reference. Milvus paper

[yhmo]

In the paper, θ-K is for "vector scan first then attribute full scan". In this approach, the search engine performs a two-step opertation:

1. scan the vectors to find θ-K nearest items, θ is an experience value which is greater than 1.0
2. do attribute filtering in the θ-K nearest items, return the items that meet the filtering condition

Let's say the expected result is K=100 items.
In the first step, we set θ=2.0, it will return 200 nearest items.
In the second step, not all the items can meet the filtering condition. Sometimes there are 150 items can meet the filtering condition, then we pick 100 items from the 150 as the final result. Sometimes there are only 75 items can meet the filtering condition, then we directly return the 75 items as the final result.

θ is not a const value, it must be greater than 1.0. In my opinion, the runtime value should be configured by users.

Anyway, it is a paper written in 2021. Now 2025 is coming.

Now the milvus is mainly using "attribute full scan first then vector scan" for filtering search:
https://milvus.io/docs/filtered-search.md#Filtered-Search

In filtering search, a bitset is generated by the first step "attribute full scan", each bit represents a position of a vector, and the second step "vector scan" will skip the vectors that are marked as 0 in the bitset.
https://milvus.io/docs/bitset.md

[rsrs2013]

Thank you @yhmo for your response. Does that mean that, milvus doesn't use the concept of cost metric calculation with various physical plans as described in the paper ?

Also, how can the user configure the θ

[yhmo]

The paper described some possible methods for filtering search. Not all the methods are good in production. Currently, Milvus only uses "attribute full scan first then vector scan" for filtering search. So, no θ.

[rsrs2013]

Thank you very much @yhmo