[Bug]: Failed to delete data resulting in repeated insertion of data(删除数据未成功导致重复插入数据)

[uniqueSkeeter]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:V2.40
- Deployment mode(standalone or cluster):standalone
- SDK version(e.g. pymilvus v2.0.0rc2): 2.4.4
- OS(Ubuntu or CentOS): Ubuntu

Current Behavior

Failed to delete data resulting in repeated insertion of data

code as below:

先删除已有的商品

    for product in param.productList:
        query_filter = f"display_prd_id=={product.disPlayPrdId}"
        product_kb.delete(expr=query_filter)
    # 将商品信息转为Document
    docs = to_product_docs(param.productList)
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=600, chunk_overlap=0)
    split_docs = text_splitter.split_documents(docs)
    # 文档向量化，并创建数据库
    product_kb.add_documents(split_docs)

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

[github-actions]

The title and description of this issue contains Chinese. Please use English to describe your issue.

[yanliang567]

@uniqueSkeeter quick questions:

1. is the field disPlayPrdId primary key? could you please share the schema? how do you ensure it is unique?
2. what is the Count:5 means in the attached screenshots. could you please share the code snippet about how to tell that the deletion is failed?

/assign @uniqueSkeeter
/unassign

[uniqueSkeeter]

yes, "disPlayPrdId" is my logical primary key （but not table's real primary key ）. and as code see, I add data always delete by disPlayPrdId then insert a new column

[yanliang567]

could u please share the code snippets of insert and query? i need to you how you define the deletion failure

[uniqueSkeeter]

try:
    collection_name = "tbl_hshop_product_kb"
    # 创建 Collection 对象
    collection = get_milvus_collection(collection_name)
    if param.type == 1:
        # 更新商品
        logger.info("update_product 更新商品开始")
        # 先删除已有的商品
        for product in param.productList:
            query_filter = f"display_prd_id=={product.disPlayPrdId}"
            delete_row = collection.delete(query_filter)
            logger.info(f'delete display_prd_id=={product.disPlayPrdId}, rows: {delete_row}')
        collection.flush()
        # 将商品信息转为Document
        metadata_list = to_product_kb_list(param.productList)
        # 文档向量化，并创建数据库
        add_rows = collection.insert(metadata_list)
        collection.flush()
        logger.info(f'add_rows: {add_rows}')

        # 更新商品数据入redis
        add_product_param(param.beCode, param.productList)
        logger.info("update_product 更新商品结束")
        return UpdateProductResponse(result=True, resultMessage="更新商品成功")

[uniqueSkeeter]

i use "collection.flush()"，and it a liittle work but didn't work out completely

[xiaofan-luan]

use 2.4.19. We fix multiple bugs on delete.

If you still hit the same issue please let us know