Skip to main content
Version: v4.30

load_gcs

Loads bytes from a Google Cloud Storage object.

load_gcs uri:string, [anonymous=bool]

Description​

The load_gcs operator connects to a GCS bucket to acquire raw bytes from a GCS object.

The connector tries to retrieve the appropriate credentials using Google's Application Default Credentials.

uri: string​

The path to the GCS object.

The syntax is gs://<bucket-name>/<full-path-to-object>(?<options>). The <options> are query parameters. Per the Arrow documentation, the following options exist:

For GCS, the supported parameters are scheme, endpoint_override, and retry_limit_seconds.

anonymous = bool (optional)​

Ignore any predefined credentials and try to use anonymous credentials.

Examples​

Read JSON from an object log.json in the folder logs in bucket.

load_gcs "gs://bucket/logs/log.json"
read_json