v9.0

The ‘Observable Extraction Rules’ are responsible for handling incoming alert data from the payload and extract key observables and it’s relations, which are essential data points like IP addresses, URLs, usernames, file hashes, and hostnames. These observables are critical for understanding the context and impact of an alert, enabling subsequent enrichment, triage, and response actions.

​
How to Create an Observable Extraction Rule

1

Navigate to Case Management Settings

Navigate to Case Management Settings and click on the ‘Observable Extraction Rules’ tab under the General Settings section.

2

Click 'New Rule' button located in the top-right corner of the page.

3

Fill out the following parameters:

If the required vendor, case type, or observable type is not available in the predefined list, you can create a custom one in the Case Management Settings.
ParameterDescription
Rule NameThe name of the “Extract Observable Rule”
DescriptionProvide a brief description of the observable extraction rule
VendorsThe vendor source of the alert. You can select one or multiple vendors. If no vendors are selected, all vendors from the list will be included by default.
Alert NameThe name of the alert name helps identify which rule will be applied
Use RegexUse regex to indicate matching criteria in the alert name.
Case Type:The case type defines the classification or category assigned to the case.
Specify the case type based on the selected vendor and alert name.

Extract Observable Mapping

ParameterDescription
Alert KeyThe ‘Alert Key’ is the location within the alert payload where the observable can be found. The path follows a dotted notation, starting from the root of the JSON object and navigating to the value you want to extract.
The Observable TypeField of interest extracted from incoming alert’s payload. Examples include IP address, file hash, DNS, Users or Email.
Optional- The Observable RelationObservable relations define the contextual relationship and significance of an observable (e.g., an email address, IP address, file hash) within a case or alert.

​
Observable Extraction Logic

The observable extraction mechanism in Blink supports parsing complex alert payloads to automatically extract relevant data points (observables) based on a defined key path. This enables observables such as email addresses, file names, and URLs to be dynamically identified from deeply nested structures.

The system supports two main extraction behaviors:

​
1. Extracting Observables from a List at the Final Path Segment

When the last field in the key path is a list, the system will extract each item in that list as a separate observable.

Focusing on this section within the alert structure:

"threatDetectionMethods": [
  "UserReported",
  "Phishing",
  "SpoofedSender"
]

Using the key path:

alerts.entities.threatDetectionMethods
  • alerts - is a list of alerts.
  • entities- is a list within each alert.
  • threatDetectionMethods - is a list of strings inside an entity.

Extracted observables:

  • "UserReported"
  • "Phishing"
  • "SpoofedSender"

Each string in the list is treated as an independent observable.

Important: All extracted observables are automatically converted to strings, even if their original value is a number, boolean, or another data type. This standardization ensures consistent formatting and compatibility when the observables are used in downstream steps such as filters, conditions, or lookups. The only exception to this rule is lists, which retain their original data type structure.

​
2. Dot-Walking Through Lists in the Key Path

When any intermediate field in the key path is a list, the system will walk through each item in the list and check whether the next field in the path exists, continuing recursively. If the key exists, its value is extracted and if not, that item is skipped.

For Example:

"alerts": [
  {
    "entities": [
      {
        "attachments": [
          { "fileName": "invoice.html", "fileType": "html" }
        ]
      },
      {
        "attachments": [
          { "fileName": "report.docx", "fileType": "docx" }
        ]
      }
    ]
  }
]

Using the key path:

alerts.entities.attachments.fileName

The system will:

  • Loop over each alert in alerts
  • Then over each entity in entities
  • Then over each attachment in attachments
  • Extract the fileName field from each attachment

Extracted observables:

  • "invoice.html"
  • "report.docx"

Each value is extracted independently and returned as a separate observable.

​
Observable Types and Relations

An observable refers to a discrete piece of content—such as an email address, IP address, or file hash—that can be analyzed, enriched, and tracked across alerts. Each observable can only be assigned one observable type. For example, an email address like johndoe@gmail.com will always have the observable type Email Address.

While the observable type is fixed, the role or context in which the observable appears can vary across alerts. These contextual roles are expressed using relations.

​
Alert Name Syntax Rules

  1. If the ‘Alert Name’ input is left blank, the ‘Extract Observable Rules’ will be applied to all alerts.

  2. When matching alert names, the system prioritizes the longest matching string. Exact matches take precedence over regular expression (regex) patterns.

  3. The Alert Name field in ‘Extract Observable Rule’ supports Regular Expressions (Regex), allowing you to define flexible matching patterns. This enables dynamic grouping and processing of similar alerts — even when alert names contain varying formats, dynamic elements like timestamps or IDs, or share common prefixes or suffixes.

​
Understand the Alert Payload Structure

Before creating an Extraction Observable Rule, you need to understand the alert payload. The payload contains important information, like the sender’s email address or device details, which can be extracted as observables.

Here’s an example of an alert payload:

{
  "@odata.etag": "W/\"CQAAABYAAABOkmv/qhPtT7Ka/WdrM4/xAAHdWH1R\"",
  "id": "awdawdawdawda-qhPtT7Ka-WdrM4-xAAAAAAEMAABOkmv-qhPtT7Ka-WdrM4-xAAHd62AUAAA=",
  "receivedDateTime": "2022-04-19T18:33:43Z",
  "sender": {
    "emailAddress": {
      "address": "frodo@middleearth.com",
      "name": "Frodo Baggins"
    }
  },
  "subject": "Mordor-47 CS Sensor installation"
}
1

Define the Observable Type

First, decide what type of observable you are extracting. In this example, we want to extract an email address, so the observable type will be User Email.

2

Identify the Observable and its Relation (if applicable)

Next, find the specific piece of information to extract and check if there’s a relation. In this case, we are extracting the email address (frodo@middleearth.com) and storing it as an observable of type Email Address.

  • The relation here could be that the email is associated with the sender, so the observable relation would be Sender.
    If there’s no relation, you can skip this step. But if there is, make sure to define the relation type (e.g., Sender)
3

Identify the Path

The path is the location of the observable in the payload, written in dotted notation. The path to the email address is:

"sender.emailAddress.address"
4

Create the Extraction Rule

Now that you know the observable, its type, and its path, you can create the rule to extract it.

5

Apply the Rule for Other Observables

You can repeat this process for other observables in the payload. For example, if you wanted to extract the sender’s name, the path would be "sender.emailAddress.name", and the type could be User Name. If the name has a relation, like being associated with the sender, the relation could be Sender.

6

Test the Template

Once your template is ready, test it using the Extract Observables Action to make sure the system correctly extracts the observables based on your defined paths, types, and relations.

​
Unmapped Alerts

In the “Alert Processed” widget within Case Management, if you receive a notification about “unmapped alerts,” it indicates that certain alerts need to be manually mapped. To resolve this, follow the steps outlined above to create custom templates based on the alert payloads.

To ensure all alerts are properly processed, we recommend using the Handle Unprocessed Alerts recovery workflow. This workflow automatically retries unmapped alerts that failed to process—typically due to missing or incorrect templates—and should be run after fixing template issues or when alert processing has previously failed.