Skip to content

HivePartitionWriter: Cast values to appropriate types

Ebernhardson requested to merge work/ebernhardson/partition-writer-cast-values into main

In spark 2 we were able to provide a literal string to a partitioning column that was an int and it would cast behind the scenes, but in spark 3 this fails and says it cannot safely cast. Add a step that does the casting ahead of time, and a little sanity checking that the value we were provided matches the expected type.

Bug: T329876

Merge request reports