Skip to content

Latest commit

 

History

History
72 lines (53 loc) · 4.44 KB

CHANGES.md

File metadata and controls

72 lines (53 loc) · 4.44 KB

Current version (not yet released; still in development)

Highlights

  • New highly anticipated feature X added to Python SDK (BEAM-X).
  • New highly anticipated feature Y added to JavaSDK (BEAM-Y).

I/Os

  • Java SDK: Adds support for Thrift encoded data via ThriftIO. (BEAM-8561)
  • Java SDK: KafkaIO supports schema resolution using Confluent Schema Registry. (BEAM-7310)
  • Python SDK: Support for Google Cloud Spanner. This is an experimental module for reading and writing data from Google Cloud Spanner (BEAM-7246).
  • Python SDK: Adds support for standard HDFS URLs (with server name). (#10223).
  • Support for X source added (Java/Python) (BEAM-X).

New Features / Improvements

  • X feature added (Java/Python) (BEAM-X).
  • New AnnotateVideo & AnnotateVideoWithContext PTransform's that integrates GCP Video Intelligence functionality. (Python) (BEAM-9146)
  • New AnnotateImage & AnnotateImageWithContext PTransform's for element-wise & batch image annotation using Google Cloud Vision API. (Python) (BEAM-9247)
  • Added a PTransform for inspection and deidentification of text using Google Cloud DLP. (Python) (BEAM-9258)
  • Added ability to write to BigQuery via Avro file loads (Python)(BEAM-8841)
  • Direct runner for FnApi supports further parallelism (Python)(BEAM-9228)

Breaking Changes

  • ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. (BEAM-3453).
  • SpannerAccessor in Java is now package-private to reduce API surface. SpannerConfig.connectToSpanner has been moved to SpannerAccessor.create. (BEAM-9310).
  • ParquetIO hadoop dependency should be now provided by the users (BEAM-8616).
  • Docker images will be deployed to apache/beam repositories from 2.20. They used to be deployed to apachebeam repository. (BEAM-9063)

Deprecations

  • X behavior is deprecated and will be removed in X versions (BEAM-X).

Bugfixes

  • Fixed X (Java/Python) (BEAM-X).
  • Fixed exception when running in IPython notebook (Python) (BEAM-X9277).
  • Fixed 1833 (Python) (BEAM-1833)

Known Issues

[2.19.0] - 2020-01-31

  • For versions 2.19.0 and older release notes are available on Apache Beam Blog.