ArrowInvalid: JSON parse error: Column() changed from object to array in row 0

[CLL112]

Describe the bug

Encountering an error while loading the liuhaotian/LLaVA-Instruct-150K dataset.

Steps to reproduce the bug

from datasets import load_dataset
fw =load_dataset("liuhaotian/LLaVA-Instruct-150K")

Error:

ArrowInvalid                              Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/json/json.py](https://localhost:8080/#) in _generate_tables(self, files)
    136                                 try:
--> 137                                     pa_table = paj.read_json(
    138                                         io.BytesIO(batch), read_options=paj.ReadOptions(block_size=block_size)

20 frames
ArrowInvalid: JSON parse error: Column() changed from object to array in row 0

During handling of the above exception, another exception occurred:

ArrowTypeError                            Traceback (most recent call last)
ArrowTypeError: ("Expected bytes, got a 'int' object", 'Conversion failed for column id with type object')

The above exception was the direct cause of the following exception:

DatasetGenerationError                    Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/datasets/builder.py](https://localhost:8080/#) in _prepare_split_single(self, gen_kwargs, fpath, file_format, max_shard_size, job_id)
   1895             if isinstance(e, DatasetGenerationError):
   1896                 raise
-> 1897             raise DatasetGenerationError("An error occurred while generating the dataset") from e
   1898 
   1899         yield job_id, True, (total_num_examples, total_num_bytes, writer._features, num_shards, shard_lengths)

DatasetGenerationError: An error occurred while generating the dataset

Expected behavior

I have tried loading the dataset both on my own server and on Colab, and encountered errors in both instances.

Environment info

- `datasets` version: 3.2.0
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- `huggingface_hub` version: 0.26.3
- PyArrow version: 17.0.0
- Pandas version: 2.2.2
- `fsspec` version: 2024.9.0