python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - huggingface-datasets

Is there a way to load local csv.gz file in huggingface dataloader?

I'm using datasets library by huggingface to load csv dataset stored locally. The problem is, the dataset is compressed and is stored as a csv.gz file. Therefore, I'm not able to load it using load_da...
test-img

Mahavir I

huggingface-datasets

huggingface

Votes: 0

Answers: 1

Latest Answer

Just decompress it with gunzip: gunzip -v yourdataset.csv.gz
test-img

cronoik

HuggingFace Datasets to PyTorch

I want to load the dataset from Hugging face, convert it to PYtorch Dataloader. Here is my script. dataset = load_dataset('cats_vs_dogs', split='train[:1000]') trans = transforms.Compose([transforms.R...
test-img

Ahmad Anis

python

machine-learning

pytorch

computer-vision

huggingface-datasets

Votes: 0

Answers: 1

Latest Answer

I think you might just need to use set_format(), e.g. ... dataset.set_transform(annot) dataset.set_format("torch") dataloader = torch.utils.data.DataLoader(dataset, batch_size=32)
test-img

Darren Cook

Huggingface datasets ValueError

I am trying to load a dataset from huggingface organization, but I am getting the following error: ValueError: Couldn't cast string -- schema metadata -- pandas: '{"index_columns": [{"k...
test-img

TMN

python

nlp

huggingface-datasets

huggingface

Votes: 0

Answers: 2

Latest Answer

According to https://github.com/huggingface/datasets/issues/3700#issuecomment-1035400186, you actually want to use load_from_disk: from datasets import load_from_disk dataset = load_from_disk("da...
test-img

Serge Chastel

ArrowInvalid: Could not convert with type Image: did not recognize Python value type when inferring an Arrow data type

I am trying to work with Hugging Face Datasets to for a sample problem. I want to turn some random images upside down, and change the label based on it. The approach I used is the following. from data...
test-img

Ahmad Anis

python

machine-learning

python-imaging-library

huggingface-datasets

Votes: 0

Answers: 1

Latest Answer

I encountered the same thing. The problem is that you need to cast the Pillow image back to the Image feature of the Datasets library. from datasets import Image feature = Image() # assuming image is...
test-img

Niels

Posts

Questions

Blogs

Jobs