I am trying to export a pandas dataframe as a parquet file. This dataframe has a memory usage of 4GB+ with 76 million rows and 6 columns (int64(3) columns, object(3) columns).

When I write this out as a parquet file, I am getting an OverflowError: “Python int too large to convert to C long message.”

I am using fastparquet as follows:

    import fastparquet
    from fastparquet import write
    write('ref_util_table.parq', df)

This works just fine for other dataframes. All the int64 columns are within range (i.e., range from 0-1000).

Any idea on how to fix the issue?