Secrets

Secrets are accepted by operators for parameters that may be sensitive, such as authentication tokens, passwords or even URLs.

Usage in TQL

There are two ways to pass an argument to a operator that expects a secret:

• By providing a plain string (Ad-hod Secret):

  to_splunk "https://localhost:8088", hec_token="my-plaintext-token"

• By using the secret function (Manged Secret):

  to_splunk "https://localhost:8088", hec_token=secret("splunk-hec-token")

  This will fetch the value of the secret named splunk-hec-token.

Operators generally do not document that they accept a secret, but will accept secrets where appropriate.

The secret type

Secrets are a special type in Tenzir's type system. The secret type only supports a limited set of operations, all of which yield secret.

Encoding & Decoding

Secrets can be Base64-decoded using the decode_base64 function and Base64-encoded using the encode_base64 function.

secret("my-secret").decode_base64()

Concatenation

Secrets can be concatenated with other secrets as well as strings using the + operator.

auth = "Bearer " + secret("my-secret")
url = secret("user-name") + ":" + secret("password")

Limited Nesting

The result of a concatenation can no longer be encoded or decoded.

Ad-hoc Secrets

Ad-hoc secrets are secrets that are created from a string directly within TQL. This happens when you provide a string to an operator that expects a secret.

Providing plain string values can be useful when developing pipelines, if you do not want to add the secret to the configuration or a secret store.

It is also useful for arguments that are not a secret for all users, such as URLs.

It is important to understand that secrets created from plain strings do not enjoy the same security as managed secrets. Their value is directly available in the TQL pipeline definition, as well as the compiled and executed representation. As such, it may be persisted on the node.

Managed Secrets

The secret function retrieves a managed secret.

Managed secrets are looked up in the following order:

1. The environment of the Tenzir Node
2. The configuration of the Tenzir Node
3. The Tenzir Platform secret store for the workspace the Tenzir Node is running in

A secret's actual value is only looked up when it is required by the operator accepting the secret. If the value is looked up over any network connection, it is additionally encrypted using ECIES with a one-time, per secret key. The value stays encrypted through the entire transfer until the final usage site.

A tenzir client process can use the secret function only if it has a Tenzir Node to connect to.

Tenzir Configuration Secrets

Secrets can be specified in the tenzir.yaml config file, under the path tenzir.secrets.

tenzir:
  secrets:
    # Add your secrets there.
    geheim: 1528F9F3-FAFA-45B4-BC3C-B755D0E0D9C2

Since Tenzir's configuration options can also be set as environment variables, this means that secrets can also be defined in the environment. The above secret could also be defined via the environment variable TENZIR_SECRETS__GEHEIM. An environment variable takes precedence over an equivalent key in the configuration file.

See the configuration reference for more details.

Be aware that the tenzir.secrets section is hidden from the config() function.

Platform Secrets

The Tenzir Platform stores a separate set of secrets for every workspace. These secrets are accessible to all Tenzir Nodes connected to that workspace.

Secrets can be added, updated or deleted from a workspace using either the tenzir-platform CLI or the web interface. Read more details in the CLI reference.

For example, to add a new secret geheim, use the following command:

tenzir-platform secret add geheim --value=1528F9F3-FAFA-45B4-BC3C-B755D0E0D9C2

To manage secrets from the web interface, go to the Workspace Settings screen by clicking on the gear icon in the workspace selector.

External Secret Stores

Sovereign Edition Recommended

The ability to use an external secret store is most useful for Sovereign Edition customers running a self-hosted instance of the platform.

Instead of using its internal secret store, the Tenzir Platform can be configured to provide access to the secrets stored in an external secrets store. This access is read-only, writing to or deleting from an external secrets store is currently not supported.

External secret stores can currently only be configured using the tenzir-platform CLI. To configure an external secret store, use the secret store add subcommand.

tenzir-platform secret store add aws --region='eu-west-1' --assumed-role-arn='arn:aws:iam::1234567890:role/tenzir-platform-secrets-access' --prefix=tenzir/

At the moment, only AWS Secrets Manager is supported as external secrets store.

In order to provide access the external secrets to the nodes, the Tenzir Platform must be given the necessary permissions to read secret values from the external store. In the example above, this means that the Tenzir Platform must be able to assume the specified role, and the role must have permissions to read secrets under the prefix tenzir/ from the Secrets Manager instance in the account of the assumed role.

See the CLI reference for more details.

Legacy Model

The configuration option tenzir.legacy-secret-model can be used to change the behavior of the secret function to return a string instead of a secret.

When using the legacy model, only secrets from the Tenzir Node's configuration can be used, no secrets from the Tenzir Platform's secret store will be available.

We do not recommend enabling this option.