1 year ago
#80247
Arun
while creating a parquet file using dask (fastparquet) with an append option the first partitions file were missing from the folder
When we create a parquet file with append option the first partition file of the parquet is missing from the final result. Any one know the reason. we are using Dask 2.30. And this happens only in one environment but in another completely different environment the issue doesnt occurs. Any one experience such an issue. I found a work around instead of creating the folder by dask , i created using a makedir option before it writes and give a few mins of sleep its works fine.
python
dask
dask-dataframe
fastparquet
0 Answers
Your Answer