Dec 12, 2023
The calculation is done based on the way spark code generates the partitions. BUt there can be around 5% difference in number as AQE sometimes takes in other factors such as partition coalesce
The calculation is done based on the way spark code generates the partitions. BUt there can be around 5% difference in number as AQE sometimes takes in other factors such as partition coalesce
Resident Solutions Architect at Databricks | YouTube: https://www.youtube.com/@easewithdata | TopMate : https://topmate.io/subham_khandelwal