Google Cloud Storage Adapter

The Google Cloud Storage Adapter provides access to Google Cloud Storage buckets and objects in the Google Cloud Platform.

The adapter supports the following features:
  • The Google Cloud Storage adapter provides access to Google Cloud Storage buckets and objects in the Google Cloud Platform.
  • Discovering available buckets and objects with the option to filter objects based on the provided delimiter and prefix values.
  • Authenticating the user using the specified service account credentials file, or automatically by the environment.
  • Reading the specified object in a single chunk or in multiple chunks of the specified size.
  • Keeping or deleting the specified object after reading and processing it successfully, as part of committing the map transaction.
  • Reading the list of objects in the specified bucket, with optional filtering, for further processing the list in a map.
  • Writing an object, with the option to fail or overwrite the target object if it exists already.
  • Writing an object in a single chunk or in multiple chunks by configuring the map inputs to run in burst mode.
  • Writing data to an intermediate (temporary) object and moving it to the target object as part of committing the map transaction.