python (65.2k questions)

javascript (44.3k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (13k questions)

Questions - fastparquet

fastparquet error when saving pandas df to parquet: AttributeError: module 'fastparquet.parquet_thrift' has no attribute 'SchemaElement

import pandas as pd from flatten_json import flatten actual_column_list = ["_id", "external_id", "email", "created_at","updated_at", "dob.timest...
test-img

Dulshan

pandas

python-3.6

parquet

nullable

fastparquet

Votes: 0

Answers: 1

Latest Answer

Use fastparquet 0.7.2 Even though fastparquet 0.8.0 is compatible with python 3.6, looks like it requires a pyarrow version greater 5.0.0 to function properly. So had to downgrade fastparquet to 0.7.2...
test-img

Dulshan

AttributeError: 'numpy.ndarray' object has no attribute '_ndarray' while reading parquet files in pandas

I am trying to read the parquet files using pandas and fastparquet engine, like below. Getting the error. import pandas as pd df = pd.read_parquet("path/to/parquet_file/", engine='fastparqu...
test-img

Vineel

python-3.x

pandas

fastparquet

Votes: 0

Answers: 0

while creating a parquet file using dask (fastparquet) with an append option the first partitions file were missing from the folder

When we create a parquet file with append option the first partition file of the parquet is missing from the final result. Any one know the reason. we are using Dask 2.30. And this happens only in one...
test-img

Arun

python

dask

dask-dataframe

fastparquet

Votes: 0

Answers: 0

unable to export dataframe due to overflow error: Python int too large to convert to C long

I am trying to export a pandas dataframe as a parquet file. This dataframe has a memory usage of 4GB+ with 76 million rows and 6 columns (int64(3) columns, object(3) columns). When I write this out as...
test-img

veg2020

python

pandas

dataframe

fastparquet

Votes: 0

Answers: 0

Posts

Questions

Blogs

Jobs