Aug 24 in Technical
Data Skew in Spark notebook
I’ve been encountering data skew issues in my Spark notebook. I’ve tried implementing salting and repartitioning with different columns, but regardless of the combinations or the number of partitions (up to 200), the result is still skewed data. I’m running out of ideas on how to resolve this. Has anyone else faced similar issues or have any suggestions?
0
2 comments
Data Skew in Spark notebook
skool.com/microsoft-fabric
A community for passionate analysts, data engineers, data scientists (& more!) looking to learn Microsoft Fabric - the end-to-end analytics platform.
Leaderboard (30-day)
1
+24
2
+21
3
+20
4
+17
5
+14
powered by