Shape lists

Lists (arrays) contain ordered sequences of values. This guide shows you how to work with lists — accessing elements, sorting and slicing, transforming values, and combining data structures.

Access list elements

Get values from specific positions:

from {items: ["first", "second", "third", "fourth"]}
first_item = items[0]
last_item = items[-1]
length = items.length()

{
  items: ["first", "second", "third", "fourth"],
  first_item: "first",
  last_item: "fourth",
  length: 4
}

Index notation:

• [0] - First element (0-based indexing)
• [-1] - Last element (negative indices count from end)
• .length() - Get the number of elements

Add elements to lists

Use append() and prepend():

from {colors: ["red", "green"]}
with_blue = colors.append("blue")
with_yellow = with_blue.prepend("yellow")
multi_append = colors.append("blue").append("purple")

{
  colors: ["red", "green"],
  with_blue: ["red", "green", "blue"],
  with_yellow: ["yellow", "red", "green", "blue"],
  multi_append: ["red", "green", "blue", "purple"]
}

Combine lists

Join multiple lists with concatenate() or spread syntax:

from {
  list1: [1, 2, 3],
  list2: [4, 5, 6],
  list3: [7, 8, 9]
}
combined = concatenate(concatenate(list1, list2), list3)
spread = [...list1, ...list2, ...list3]
with_value = [...list1, 10, ...list2]

{
  list1: [1, 2, 3],
  list2: [4, 5, 6],
  list3: [7, 8, 9],
  combined: [1, 2, 3, 4, 5, 6, 7, 8, 9],
  spread: [1, 2, 3, 4, 5, 6, 7, 8, 9],
  with_value: [1, 2, 3, 10, 4, 5, 6]
}

Transform list elements

Apply functions to each element with map():

from {
  prices: [10, 20, 30],
  names: ["alice", "bob", "charlie"]
}
with_tax = prices.map(p => p * 1.1)
uppercase = names.map(n => n.to_upper())
squared = prices.map(x => x * x)

{
  prices: [10, 20, 30],
  names: ["alice", "bob", "charlie"],
  with_tax: [11.0, 22.0, 33.0],
  uppercase: ["ALICE", "BOB", "CHARLIE"],
  squared: [100, 400, 900]
}

Filter list elements

Keep only elements that match a condition with where():

from {
  numbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  users: ["alice", "bob", "anna", "alex"]
}
// Note: The modulo operator (%) is not currently supported in TQL
// Here's an alternative approach for filtering:
big_nums = numbers.where(n => n > 5)
small_nums = numbers.where(n => n <= 5)
a_names = users.where(u => u.starts_with("a"))

{
  numbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  users: ["alice", "bob", "anna", "alex"],
  big_nums: [6, 7, 8, 9, 10],
  small_nums: [1, 2, 3, 4, 5],
  a_names: ["alice", "anna", "alex"]
}

Sort lists

Order elements with sort():

from {
  numbers: [3, 1, 4, 1, 5, 9],
  words: ["zebra", "apple", "banana"]
}
sorted_nums = numbers.sort()
sorted_words = words.sort()

{
  numbers: [3, 1, 4, 1, 5, 9],
  words: ["zebra", "apple", "banana"],
  sorted_nums: [1, 1, 3, 4, 5, 9],
  sorted_words: ["apple", "banana", "zebra"]
}

Sort in descending order with desc:

from {numbers: [3, 1, 4, 1, 5, 9]}
descending = numbers.sort(desc=true)

{numbers: [3, 1, 4, 1, 5, 9], descending: [9, 5, 4, 3, 1, 1]}

Use a custom comparator to sort records by a specific field:

from {
  users: [
    {name: "Charlie", age: 35},
    {name: "Alice", age: 30},
    {name: "Bob", age: 25}
  ]
}
by_age = users.sort(cmp=(a, b) => a.age < b.age)
by_name = users.sort(cmp=(a, b) => a.name < b.name)

{
  users: [
    {name: "Charlie", age: 35},
    {name: "Alice", age: 30},
    {name: "Bob", age: 25}
  ],
  by_age: [
    {name: "Bob", age: 25},
    {name: "Alice", age: 30},
    {name: "Charlie", age: 35}
  ],
  by_name: [
    {name: "Alice", age: 30},
    {name: "Bob", age: 25},
    {name: "Charlie", age: 35}
  ]
}

Slice lists

Extract portions of a list with slice():

Get the first N elements:

from {xs: [1, 2, 3, 4, 5]}
first_three = xs.slice(end=3)

{xs: [1, 2, 3, 4, 5], first_three: [1, 2, 3]}

Get the last N elements:

from {xs: [1, 2, 3, 4, 5]}
last_three = xs.slice(begin=-3)

{xs: [1, 2, 3, 4, 5], last_three: [3, 4, 5]}

Extract a range:

from {xs: [10, 20, 30, 40, 50, 60, 70]}
middle = xs.slice(begin=2, end=5)

{xs: [10, 20, 30, 40, 50, 60, 70], middle: [30, 40, 50]}

Reverse a list:

from {xs: [1, 2, 3, 4, 5]}
reversed = xs.slice(stride=-1)

{xs: [1, 2, 3, 4, 5], reversed: [5, 4, 3, 2, 1]}

Select every Nth element:

from {xs: [1, 2, 3, 4, 5, 6, 7, 8]}
every_other = xs.slice(stride=2)

{xs: [1, 2, 3, 4, 5, 6, 7, 8], every_other: [1, 3, 5, 7]}

Top-k and bottom-k

Chain sort() and slice() to implement a top-k query over list elements:

from {scores: [72, 95, 88, 61, 83, 97, 56]}
top_3 = scores.sort(desc=true).slice(end=3)
bottom_3 = scores.sort().slice(end=3)

{
  scores: [72, 95, 88, 61, 83, 97, 56],
  top_3: [97, 95, 88],
  bottom_3: [56, 61, 72]
}

Get unique values

Remove duplicates with distinct():

from {
  items: ["a", "b", "a", "c", "b", "d"],
  numbers: [1, 2, 2, 3, 3, 3, 4]
}
unique_items = distinct(items).sort()
unique_nums = distinct(numbers).sort()

{
  items: ["a", "b", "a", "c", "b", "d"],
  numbers: [1, 2, 2, 3, 3, 3, 4],
  unique_items: ["a", "b", "c", "d"],
  unique_nums: [1, 2, 3, 4]
}

Use lists as sets

Use add() for set-insertion (append only if absent) and remove() to delete all occurrences:

from {tags: ["info", "warning", "info"]}
with_error = tags.add("error")
still_same = tags.add("info")
without_info = tags.remove("info")

{
  tags: ["info", "warning", "info"],
  with_error: ["info", "warning", "info", "error"],
  still_same: ["info", "warning", "info"],
  without_info: ["warning"]
}

Combine add() with distinct() when you need a deduplicated collection:

from {xs: [1, 2, 2, 3]}
unique = distinct(xs).sort()
with_4 = unique.add(4)
with_2 = unique.add(2)

{
  xs: [1, 2, 2, 3],
  unique: [1, 2, 3],
  with_4: [1, 2, 3, 4],
  with_2: [1, 2, 3]
}

Flatten nested lists

Note: Direct list flattening is not currently supported in TQL. The flatten() function is designed for flattening records, not lists. To work with nested lists, you would need to process them element by element.

Combine lists and records

Work with collections of records:

from {
  users: [
    {name: "Alice", age: 30, city: "NYC"},
    {name: "Bob", age: 25, city: "SF"},
    {name: "Charlie", age: 35, city: "NYC"}
  ]
}
names = users.map(u => u.name)
nyc_users = users.where(u => u.city == "NYC")
avg_age = users.map(u => u.age).sum() / users.length()

{
  users: [
    {name: "Alice", age: 30, city: "NYC"},
    {name: "Bob", age: 25, city: "SF"},
    {name: "Charlie", age: 35, city: "NYC"}
  ],
  names: ["Alice", "Bob", "Charlie"],
  nyc_users: [
    {name: "Alice", age: 30, city: "NYC"},
    {name: "Charlie", age: 35, city: "NYC"}
  ],
  avg_age: 30.0
}

Advanced transformations

Zip lists together

Combine parallel lists with zip():

from {
  names: ["Alice", "Bob", "Charlie"],
  ages: [30, 25, 35],
  cities: ["NYC", "SF", "LA"]
}
// Note: zip only takes 2 arguments, returns records with left/right fields
name_age = zip(names, ages)
zipped = zip(name_age, cities)
users = zipped.map(z => {
  name: z.left.left,
  age: z.left.right,
  city: z.right
})

{
  names: ["Alice", "Bob", "Charlie"],
  ages: [30, 25, 35],
  cities: ["NYC", "SF", "LA"],
  name_age: [
    {left: "Alice", right: 30},
    {left: "Bob", right: 25},
    {left: "Charlie", right: 35}
  ],
  zipped: [
    {left: {left: "Alice", right: 30}, right: "NYC"},
    {left: {left: "Bob", right: 25}, right: "SF"},
    {left: {left: "Charlie", right: 35}, right: "LA"}
  ],
  users: [
    {name: "Alice", age: 30, city: "NYC"},
    {name: "Bob", age: 25, city: "SF"},
    {name: "Charlie", age: 35, city: "LA"}
  ]
}

Practical example: Pairing related data

Combine parallel arrays with transformations for data normalization:

from {
  // DNS query data with parallel arrays
  answers: ["192.168.1.1", "10.0.0.1", "172.16.0.1"],
  ttls: [300s, 600s, 900s],
  // Certificate SAN names
  domains: ["example.com", "www.example.com", "api.example.com"]
}

// Pair DNS answers with their TTLs and transform
dns_records = zip(answers, ttls).map(x => {
  rdata: x.left,
  ttl_seconds: x.right.count_seconds(),
  cached_until: now() + x.right
})

// Transform simple arrays to structured data
san_entries = domains.map(name => {
  name: name,
  type: "DNSName",
  verified: name.ends_with(".example.com")
})

{
  answers: ["192.168.1.1", "10.0.0.1", "172.16.0.1"],
  ttls: [300s, 600s, 900s],
  domains: ["example.com", "www.example.com", "api.example.com"],
  dns_records: [
    {
      rdata: "192.168.1.1",
      ttl_seconds: 300,
      cached_until: 2025-08-14T12:36:45.123456Z
    },
    {
      rdata: "10.0.0.1",
      ttl_seconds: 600,
      cached_until: 2025-08-14T12:41:45.123456Z
    },
    {
      rdata: "172.16.0.1",
      ttl_seconds: 900,
      cached_until: 2025-08-14T12:46:45.123456Z
    }
  ],
  san_entries: [
    {
      name: "example.com",
      type: "DNSName",
      verified: false
    },
    {
      name: "www.example.com",
      type: "DNSName",
      verified: true
    },
    {
      name: "api.example.com",
      type: "DNSName",
      verified: true
    }
  ]
}

This pattern is particularly useful when:

• Converting parallel arrays from APIs or logs into structured records
• Normalizing data for standard formats (like OCSF)
• Adding computed fields during the transformation

Enumerate with indices

Add row numbers to your data using the enumerate operator:

from {item: "apple"}, {item: "banana"}, {item: "cherry"}
enumerate row

{row: 0, item: "apple"}
{row: 1, item: "banana"}
{row: 2, item: "cherry"}

This is useful for tracking position in sequences or creating unique identifiers for each event.

Practical examples

Extract and transform nested data

from {
  response: {
    status: 200,
    data: {
      users: [
        {id: 1, name: "Alice", scores: [85, 92, 88]},
        {id: 2, name: "Bob", scores: [78, 81, 85]}
      ]
    }
  }
}
users = response.data.users
summaries = users.map(u => {
  name: u.name,
  avg_score: u.scores.sum() / u.scores.length(),
  max_score: u.scores.max()
})

{
  response: {...},
  users: [
    {id: 1, name: "Alice", scores: [85, 92, 88]},
    {id: 2, name: "Bob", scores: [78, 81, 85]}
  ],
  summaries: [
    {name: "Alice", avg_score: 88.33333333333333, max_score: 92},
    {name: "Bob", avg_score: 81.33333333333333, max_score: 85}
  ]
}

Work with indexed data

Create lookups by extracting specific fields:

from {
  items: [
    {id: "A001", name: "Widget", price: 10},
    {id: "B002", name: "Gadget", price: 20},
    {id: "C003", name: "Tool", price: 15}
  ]
}
first_item = items.first()
ids = items.map(item => item.id)
names = items.map(item => item.name)
expensive = items.where(item => item.price > 15)

{
  items: [...],
  first_item: {id: "A001", name: "Widget", price: 10},
  ids: ["A001", "B002", "C003"],
  names: ["Widget", "Gadget", "Tool"],
  expensive: [
    {id: "B002", name: "Gadget", price: 20}
  ]
}

Best practices

1. Choose the right structure: Use lists for ordered data, records for named fields
2. Avoid deep nesting: Flatten structures when possible for easier access
3. Use functional methods: Prefer map(), filter(), etc. over manual loops
4. Handle empty collections: Check length before accessing elements
5. Preserve immutability: Collection functions return new values, not modify existing

See Also

• fnsort
• fnslice
• Shape records
• Filter and select data
• Transform values
• Reshape complex data