Azure Log Analytics

Azure Monitor is Microsoft's cloud solution for collecting and analyzing logs and system events. Azure Log Analytics is a part of Monitor and comes with an Logs Ingestion API for sending data to tables within a Log Analytics workspace that which is a unique environment for log data, such as from Microsoft Sentinel and Microsoft Defender for Cloud. Log Anlaytics tables are either pre-defined standard tables that follow a given schema, or user-defined custom tables.

The diagram below illustrates the key components involved when sending data to a Log Analytics table:

The Data Collection Endpoint (DCE) is an authenticated HTTPS API endpoint that accepts HTTP POST requests with events encoded as JSON arrays in the request body. The Data Collection Rule (DCR) offers optional transformation of arriving data and routes the data to a Log Analytics table.

Azure Monitor Setup Tutorial

The following use cases assume that you have already set up the Azure Monitor side, for example, by following the official tutorial that walks through for setting up a sample Entra application to authenticate against the API, to create a DCE to receive data, to create a custom table in a Log Analytics workspace and DCR to forward data to that table, and to give the applciation the proper permissions to access the created DCE and DCR.

Send logs to custom table

Let's assume that you have the following CSV file that you want to send to a custom table:

users.csv

user,age
Alice,42
Bob,43
Charlie,44

Assuming you have already created a custom table called Custom-Users, you can send this file to the table using the to_azure_log_analytics operator:

load_file "users.csv"
read_csv
to_azure_log_analytics
  tenant_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  client_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  client_secret="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  dce="https://my-dce.westeurope-1.ingest.monitor.azure.com",
  dcr="dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  table="Custom-Users"