[Feature]: Parquet import can support column-based import like numpy

[zhuwenxing]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe.

I have multiple collections that need data import, but the vector fields are all the same, and I don't want to save the vector fields in multiple files.

collection 1

id.parquet
text.parquet
emb.parquet

collection 2

id.parquet
word.parquet
emb.parquet

Each parquet file has only one column, and the column name is the file name

According to the current format requirements, if we need to save (id,text,emb) as one file and (id,word,emb) as another file, this will bring additional storage

Describe the solution you'd like.

No response

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

[xiaofan-luan]

parquet is already a columnar storage.
usually people store all the column in one file rather than seperate it?