Skip to content

Commit

Permalink
don't unwrap copy, we need to original so that the hadoopconf gets mo…
Browse files Browse the repository at this point in the history
…dified by build
  • Loading branch information
Jonathan Vexler committed May 30, 2024
1 parent f126475 commit 63737ca
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ class HoodieFileGroupReaderBasedParquetFileFormat(tableState: HoodieTableState,
val requestedAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(requiredSchema, sanitizedTableName)
val dataAvroSchema = AvroConversionUtils.convertStructTypeToAvroSchema(dataSchema, sanitizedTableName)
val parquetFileReader = spark.sparkContext.broadcast(sparkAdapter.createParquetFileReader(supportBatchResult,
spark.sessionState.conf, options, augmentedStorageConf.unwrapCopy()))
spark.sessionState.conf, options, augmentedStorageConf.unwrap()))
val broadcastedStorageConf = spark.sparkContext.broadcast(augmentedStorageConf)
val broadcastedDataSchema = spark.sparkContext.broadcast(dataAvroSchema)
val broadcastedRequestedSchema = spark.sparkContext.broadcast(requestedAvroSchema)
Expand Down

0 comments on commit 63737ca

Please sign in to comment.