Hi,

1. Thanks for the great tool for managing datasets.

2. I have got an issue when doing some test to create a local hub dataset from a local large dataset. The process stopped and said Killed.

3. I tried to upload the dataset in batches, i.e. within with ds: only appending a certain number of data, so that it fits in memory.

4. Then call this with ds: iteratively until the whole dataset is uploaded.

5. However, after the first batch (1800 samples with multiple tensors) is written, the program was killed during the second batch.

6. I noticed this might be an indication of running out of resource. But don't know where the issue could be.

7. If I can have any suggestion from you, it would be very appreciated.

Many thanks
TS