1 year ago

#80247

test-img

Arun

while creating a parquet file using dask (fastparquet) with an append option the first partitions file were missing from the folder

When we create a parquet file with append option the first partition file of the parquet is missing from the final result. Any one know the reason. we are using Dask 2.30. And this happens only in one environment but in another completely different environment the issue doesnt occurs. Any one experience such an issue. I found a work around instead of creating the folder by dask , i created using a makedir option before it writes and give a few mins of sleep its works fine.

python

dask

dask-dataframe

fastparquet

0 Answers

Your Answer

Accepted video resources