parquet Pyspark
df.write.parquet("AA_DWF_All.parquet",mode="overwrite")
df_new = spark.read.parquet("AA_DWF_All.parquet")
print(df_new.count())
Sore Stork
df.write.parquet("AA_DWF_All.parquet",mode="overwrite")
df_new = spark.read.parquet("AA_DWF_All.parquet")
print(df_new.count())