Is it possible to use setSchemaUpdateOptions(ALLOW_FIELD_ADDITION) from BigQuery load config in Dataflow with the built-in BigQueryIO.Write?

I would like to use the experimental option that allows me to update a BigQuery schema when performing a load job.

I’m using Dataflow and the built-in BigQueryIO.write from the SDK.

I saw that with a JobConfigurationLoad.setSchemaUpdateOptions(ALLOW_FIELD_ADDITION) from the BigQuery API it’s possible, but I can’t find the equivalent with the BigQueryIO.

Does it exist somewhere or can I override some part in the BigQueryIO to do that ?

Thank you very much,

AFAIK, that experimental option is not yet exposed via the Dataflow/Beam APIs in BigQueryIO, and it would not be a trivial task to override something in in that class – I wouldn’t recommended going down that route.

One workaround I can think of would be to redirect your sink to GCS instead of BigQuery, and then perform a normal BigQuery load job(s) at the end of your pipeline. That way you can use the option.