When you're talking about Security Validation and the terms passed and failed, you must consider the context before you know what those terms actually mean. Specific context includes:
- Jobs / Actions Passing / Failing
- Special case 1: Email Actions
- Special case 2: Malicious DNS Query Actions
- Special case 3: Host CLI Actions
- Special case 4: Cloud Validation Actions
- Monitors Passing / Failing
- Operational Readiness tests Passing / Failing
Pass / Fail: Jobs and Actions
When you run an Action, which we call a Job Action, the Validation Platform identifies whether the Action is blocked and/or detected. The platform uses that information to determine if the Job Action passed or failed. By default, if the Job Action is blocked or detected, it receives a "Pass" score. However, the platform is configurable, allowing you to update the pass/fail definitions to better reflect your organization's needs. There are three types of pass/fail rules you can create:
- VID Rules: Defining requirements for a specific Action
- Dimension Rules: Defining requirements that are applied to all Actions that are assigned the selected dimensions (Attack Vector, Attacker Location, Behavior Type, Covert, OS/Platform, & Stage of Attack)
- Default Rule: The rule that is applied to Actions that are not impacted by a VID or Dimension rule
For each of these rules, you have the following options to choose from to define the pass criteria:
- Either
- Blocked
- Detected
- Both
A Job Action will only have one pass/fail rule applied to it. The platform applies the rules from most specific (VID rules) to most general (the Default rule). For example, if there is a VID and Dimension Rule that impacts a Job Action, the VID rule is used to determine the pass/fail results for the Job. Here are details and explanations for each type of Rule.
TIP: If you need to maintain historic pass/fail information and have AEDA, configuring Monitors is another way to define expected results for Actions.
Pass/Fail Settings page
To Review, Add, or Update the Pass/Fail Rules
- Launch the Director.
- Go to Settings > Director Settings.
- Select Pass/Fail.
Review the existing rules, create or update rules, or set the desired event status.
IMPORTANT: Any updates you make to the pass/fail rules impacts all Jobs, not just future Jobs. For example, if you update the default rule so a Job Action must be Detected to pass, all jobs that have been run will be reviewed and updated based on their detection status.
- (Optional) Change the Show event status as No when detection window is expired? setting to modify the event status that is attached to an Action that is detected as expired.
Email Rules / Actions
When you run an Email Action, you're testing the response of your email security controls. For Security Validation to understand what the responses mean, and thus determine if the Job Action passes or fails, you must setup Email Rules. These tell the platform what responses mean the Action was blocked, and thus if the Job Action passes or fails.
Malicious DNS Query Actions
Security Validation doesn't know how to interpret blocked results for Malicious DNS Query Actions without user input. Creating Rules for Malicious DNS Query Actions is how you configure the system to identify what Actions should be blocked, leading to the final pass/fail determination.
Host CLI Actions and Cloud Validation Actions
These Actions includes scripts to have certain actions performed on systems. As such, you must build in what your expected blocked conditions are. Otherwise, Security Validation won't know if the Job Action passes or fails.