Skip to main content
v9.0
The ‘Observable Extraction Rules’ are responsible for handling incoming alert data from the payload and extract key observables and it’s relations, which are essential data points like IP addresses, URLs, usernames, file hashes, and hostnames. These observables are critical for understanding the context and impact of an alert, enabling subsequent enrichment, triage, and response actions.
When reviewing the sample alert payload and the vendor associated with, consider the following questions:
  1. Are the field names and structures consistent across different alert types from this vendor?
  2. Does the context or content of the alerts vary significantly between types?
  3. Can you reasonably estimate how many alerts will be generated by the rules you are creating?
Your answers will help determine the best rule configuration strategy. In some cases, it may be appropriate to create specific rules tailored to each alert type. In others, a single generic rule may suffice. You can also combine both approaches, define specific rules for well, understood alert types and use a generic rule as a fallback.In this setup, the system will prioritize specific rules. The generic rule will only be applied when no specific rule matches the alert, effectively serving as a catch-all fallback.

​
How to Create an Observable Extraction Rule

1

Navigate to Case Management Settings

Navigate to Case Management Settings and click on the ‘Observable Extraction Rules’ tab under the General Settings section.
2

Click 'New Rule' button located in the top-right corner of the page.

3

Fill out the following parameters:

If the required vendor, case type, or observable type or observable relation is not available in the predefined list, you can create a custom one in the Case Management Settings.
ParameterDescription
Rule NameThe name of the “Extract Observable Rule”
DescriptionProvide a brief description of the observable extraction rule
VendorsThe vendor source of the alert. You can select one or multiple vendors. If no vendors are selected, all vendors from the list will be included by default.
Alert NameThe name of the alert name helps identify which rule will be applied
Use RegexUse regex to indicate matching criteria in the alert name.
Case TypeThe case type defines the classification or category assigned to the case.
Specify the case type based on the selected vendor and alert name.
Extract Observable Mapping
ParameterDescription
Alert KeyThe ‘Alert Key’ is the location within the alert payload where the observable can be found. The path follows a dotted notation, starting from the root of the JSON object and navigating to the value you want to extract.
The Observable TypeField of interest extracted from incoming alert’s payload. Examples include IP address, file hash, DNS, Users or Email.
Optional- The Observable RelationObservable relations define the contextual relationship and significance of an observable (e.g., an email address, IP address, file hash) within a case or alert.

​
Observable Extraction Logic

The observable extraction mechanism in Blink supports parsing complex alert payloads to automatically extract relevant data points (observables) based on a defined key path. This enables observables such as email addresses, file names, and URLs to be dynamically identified from deeply nested structures. The system supports two main extraction behaviors:

​
1. Extracting Observables from a List at the Final Path Segment

When the last field in the key path is a list, the system will extract each item in that list as a separate observable. 
{
    "alerts": [
        {
            "alertId": "alert_987654",
            "category": "Phishing",
            "description": "A user reported a phishing email using the Report Message add-in.",
            "detectorId": "UserReportedPhish",
            "endTime": "2025-06-09T10:00:00Z",
            "entities": [
                {
                    "entityType": "Mailbox",
                    "mailboxPrimaryAddress": "jane.doe@contoso.com"
                },
                {
                    "attachments": [
                        {
                            "fileName": "invoice.html",
                            "fileType": "html",
                            "malwareFamily": null
                        }
                    ],
                    "deliveryAction": "Delivered",
                    "entityType": "EmailMessage",
                    "internetMessageId": "<abc123xyz@spoofed.com>",
                    "recipient": {
                        "emailAddress": "jane.doe@contoso.com"
                    },
                    "sender": {
                        "displayName": "Microsoft Support",
                        "emailAddress": "spoofed-support@microsoft.com"
                    },
                    "subject": "Urgent: Action required on your account",
                    "threatDetectionMethods": [
                        "UserReported",
                        "Phishing",
                        "SpoofedSender"
                    ],
                    "urls": [
                        {
                            "clickAction": "Clicked",
                            "threatType": "Phishing",
                            "url": "http://malicious-site.com/login"
                        }
                    ]
                }
            ],
            "severity": "Medium",
            "startTime": "2025-06-09T09:58:00Z",
            "title": "User-reported phishing email"
        }
    ],
    "assignedTo": "SecOps Team",
    "comments": [
        {
            "author": "jane.doe@contoso.com",
            "comment": "This email looked suspicious and requested account login."
        }
    ],
    "createdTime": "2025-06-09T10:24:00Z",
    "id": "incident_123457",
    "incidentName": "Phishing email reported by user",
    "incidentSeverity": "Medium",
    "lastUpdateTime": "2025-06-09T10:45:12Z",
    "status": "Active",
    "tags": [
        "UserReported",
        "Phishing",
        "Email"
    ]
}
Focusing on this section within the alert structure: 
"threatDetectionMethods": [
  "UserReported",
  "Phishing",
  "SpoofedSender"
]
Using the key path: 
alerts.entities.threatDetectionMethods
  • alerts - is a list of alerts.
  • entities- is a list within each alert.
  • threatDetectionMethods - is a list of strings inside an entity.
Extracted observables:
  • "UserReported"
  • "Phishing"
  • "SpoofedSender"
Each string in the list is treated as an independent observable.
Important: All extracted observables are automatically converted to strings, even if their original value is a number, boolean, or another data type. This standardization ensures consistent formatting and compatibility when the observables are used in downstream steps such as filters, conditions, or lookups. The only exception to this rule is lists, which retain their original data type structure.

​
2. Dot-Walking Through Lists in the Key Path

When any intermediate field in the key path is a list, the system will walk through each item in the list and check whether the next field in the path exists, continuing recursively. If the key exists, its value is extracted and if not, that item is skipped. For Example: 
"alerts": [
  {
    "entities": [
      {
        "attachments": [
          { "fileName": "invoice.html", "fileType": "html" }
        ]
      },
      {
        "attachments": [
          { "fileName": "report.docx", "fileType": "docx" }
        ]
      }
    ]
  }
]
Using the key path: 
alerts.entities.attachments.fileName
The system will:
  • Loop over each alert in alerts
  • Then over each entity in entities
  • Then over each attachment in attachments
  • Extract the fileName field from each attachment
Extracted observables:
  • "invoice.html"
  • "report.docx"
Each value is extracted independently and returned as a separate observable.

​
Observable Types and Relations

An observable refers to a discrete piece of content, such as an email address, IP address, or file hash, that can be analyzed, enriched, and tracked across alerts. Each observable can only be assigned one observable type. For example, an email address like johndoe@gmail.com will always have the observable type Email Address.
Reminder: If a required observable type is not available in the predefined list, you can create a custom one in the Case Management Settings.
  1. Unknown
  2. Hostname
  3. IP Address
  4. MAC Address
  5. URL String
  6. Username
  7. Email Address
  8. URL String
  9. File Name
  10. Hash
  11. Process Name
  12. Resource UID
  13. Port
  14. Subnet
  15. Command Line
  16. Country
  17. Process ID
  18. HTTP User Agent
  19. CWE Object : uid
  20. CVE Object: uid
  21. User Credential ID
  22. Endpoint
  23. User
  24. Email
  25. Uniform Resource Locator
  26. File
  27. Process
  28. Geo Location
  29. Container
  30. Registry Key
  31. Registry Value
  32. Fingerprint
  33. Other
An observable relation defines the role or context in which an observable is associated with an alert or case. While the observable’s type (e.g., IP, domain, file hash) remains fixed, its relation describes how it is connected to the event—such as being a source IP, destination IP, malicious payload, or parent process.
Reminder: If a required observable relation is not available in the predefined list, you can create a custom one in the Case Management Settings.
  1. Actor
  2. Attacker IP Address
  3. Device
  4. Parent Process Hash
  5. Receiver
  6. Recipient
  7. Sender
  8. Target
  9. Target Device
  10. Target Host
  11. Target IP Address
  12. Target User
  13. sha256
Consider the observable johndoe@gmail.com, typed as Email Address. This observable might appear in different roles depending on the alert:
  • In one alert, it is the recipient of a phishing email.
  • In another alert, it is the sender of a suspicious message.
Rather than creating new observable types such as “Sender Email” or “Recipient Email,” the observable retains its type (Email Address) and the roles are captured using relations. This approach preserves consistency while supporting flexible contextual representation.
Similarly, take the observable 192.168.1.100 with the type IP Address. In different alerts, the same IP address may appear in distinct roles:
  • As the source IP in a port scanning attempt.
  • As the destination IP in an unauthorized outbound connection.
Instead of defining separate observable types like “Source IP” or “Destination IP,” these roles are modeled as relations associated with the same observable.
  • Consistent enrichment: Observables are enriched once, regardless of the roles they take on.
  • Improved correlation: All related activity tied to a specific observable can be traced across alerts, independent of its role.
  • Flexible investigations: Analysts can filter, pivot, and investigate based on relations without duplicating observables.
This provides a clean separation between what the observable is (its type) and how it is involved in each alert (its relation), enabling more powerful and efficient threat investigation workflows.

​
Alert Name Syntax Rules

  1. If the ‘Alert Name’ input is left blank, the ‘Extract Observable Rules’ will be applied to all alerts.
  2. When matching alert names, the system prioritizes the longest matching string. Exact matches take precedence over regular expression (regex) patterns.
  3. The Alert Name field in ‘Extract Observable Rule’ supports Regular Expressions (Regex), allowing you to define flexible matching patterns. This enables dynamic grouping and processing of similar alerts — even when alert names contain varying formats, dynamic elements like timestamps or IDs, or share common prefixes or suffixes.
If we want to match all alert names that start with “Critical Alert” followed by a series of numbers.Regex Use Case Example
^Critical Alert [0-9]+$
Explanation:
  • ^ matches the beginning of the alert name.
  • Critical Alert matches the literal text.
  • [0-9]+ matches one or more digits.
  • $ matches the end of the alert name. This would match:
  • Critical Alert 123
  • Critical Alert 4567
But not:
  • Critical Alert (no numbers)
  • Info Alert 123 (doesn’t start with “Critical Alert”)

​
Understand the Alert Payload Structure

Before creating an Extraction Observable Rule, you need to understand the alert payload. The payload contains important information, like the sender’s email address or device details, which can be extracted as observables. Here’s an example of an alert payload: 
{
  "@odata.etag": "W/\"CQAAABYAAABOkmv/qhPtT7Ka/WdrM4/xAAHdWH1R\"",
  "id": "awdawdawdawda-qhPtT7Ka-WdrM4-xAAAAAAEMAABOkmv-qhPtT7Ka-WdrM4-xAAHd62AUAAA=",
  "receivedDateTime": "2022-04-19T18:33:43Z",
  "sender": {
    "emailAddress": {
      "address": "frodo@middleearth.com",
      "name": "Frodo Baggins"
    }
  },
  "subject": "Mordor-47 CS Sensor installation"
}
1

Define the Observable Type

First, decide what type of observable you are extracting. In this example, we want to extract an email address, so the observable type will be User Email.
2

Identify the Observable and its Relation (if applicable)

Next, find the specific piece of information to extract and check if there’s a relation. In this case, we are extracting the email address (frodo@middleearth.com) and storing it as an observable of type Email Address.
  • The relation here could be that the email is associated with the sender, so the observable relation would be Sender.
    If there’s no relation, you can skip this step. But if there is, make sure to define the relation type (e.g., Sender)
3

Identify the Path

The path is the location of the observable in the payload, written in dotted notation. The path to the email address is:
"sender.emailAddress.address"
4

Create the Extraction Rule

Now that you know the observable, its type, and its path, you can create the rule to extract it.
5

Apply the Rule for Other Observables

You can repeat this process for other observables in the payload. For example, if you wanted to extract the sender’s name, the path would be "sender.emailAddress.name", and the type could be User Name. If the name has a relation, like being associated with the sender, the relation could be Sender.
6

Test the Template

Once your template is ready, test it using the Extract Observables Action to make sure the system correctly extracts the observables based on your defined paths, types, and relations.

​
Unmapped Alerts

In the “Alert Processed” widget within Case Management, if you receive a notification about “unmapped alerts,” it indicates that certain alerts need to be manually mapped. To resolve this, follow the steps outlined above to create custom templates based on the alert payloads.
To ensure all alerts are properly processed, we recommend using the Handle Unprocessed Alerts recovery workflow. This workflow automatically retries unmapped alerts that failed to process—typically due to missing or incorrect templates—and should be run after fixing template issues or when alert processing has previously failed.