save_gcs

Saves bytes to a Google Cloud Storage object.

save_gcs uri:string, [anonymous=bool]

Description

The save_gcs operator connects to a GCS bucket to save raw bytes to a GCS object.

The connector tries to retrieve the appropriate credentials using Google’s Application Default Credentials.

uri: string

The path to the GCS object.

The syntax is gs://<bucket-name>/<full-path-to-object>(?<options>). The <options> are query parameters. Per the Arrow documentation, the following options exist:

For GCS, the supported parameters are scheme, endpoint_override, and retry_limit_seconds.

anonymous = bool (optional)

Ignore any predefined credentials and try to use anonymous credentials.

Examples

Write JSON to an object test.json in bucket, but using a different GCS-compatible endpoint:

write_json
save_gcs "gs://bucket/test.json?endpoint_override=gcs.mycloudservice.com"

See Also

load_gcs