Skip to content

Commit 648daef

Browse files
authored
[docs] feat(expectation): improv documentation
1 parent e13fc07 commit 648daef

13 files changed

Lines changed: 408 additions & 132 deletions
-63 KB
Binary file not shown.
Binary file not shown.
-10.6 KB
Binary file not shown.

docs/usage/expectations.md

Lines changed: 25 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -1,149 +1,43 @@
1-
# Expectations
21

3-
Expectations define what is expected from an [Asset (endpoint)](assets.md) or
4-
a [Players](people.md#players) when facing an [Inject](inject-overview.md) in terms of
5-
security posture. Each expectation has a score representing how well it has been met by the target.
62

7-
## Expectation types
3+
#### Why use it?
84

9-
Expectations can be categorized as either Manual or Automatic, depending on how they are validated.
5+
Use this mode when **every member must succeed**, such as:
106

11-
### Manual expectations
7+
* mandatory training
8+
* compliance requirements
9+
* baseline security checks
1210

13-
Manual expectations require validation by the **exercise organizer** (animation team).
14-
They are simple, customizable, and **user-controlled**.
11+
#### How does it work?
1512

16-
Examples:
13+
* All targets succeed → **100**
14+
* At least one target fails → **0**
1715

18-
- **Team response validation**
19-
Check whether the **incident response team** correctly acknowledges and escalates a phishing inject during the exercise.
16+
#### Example
2017

21-
- **Player task validation**
22-
Ensure that an **analyst player** correctly follows the playbook and reports an incident in the ticketing system.
18+
Group of 4 players:
2319

24-
### Automatic expectations
20+
* 3 succeed, 1 fails → **0 (Failed)**
21+
* 4 succeed → **100 (Success)**
2522

26-
Automatic expectations are validated automatically under specific conditions.
23+
### At least one target must validate
2724

28-
- `Prevention`: automatically validated by security integrations (e.g., quarantine event).
29-
- `Detection`: automatically validated by security integrations (e.g., incident alert).
30-
- `Vulnerability`: automatically validated based on the presence of CVEs.
31-
- `Expect targets to read the article(s)`: automatically validated once targets have read the injected article.
25+
#### Why use it?
3226

33-
## Validation Mode
27+
Use this mode when **a single successful response is enough**, such as:
3428

35-
There are two modes for validating an expectation :
29+
* SOC detection
30+
* on-call escalation
31+
* redundancy testing
3632

37-
### All targets (per group) must validate the expectation
33+
#### How does it work?
3834

39-
- The result depends on **every member's performance**.
40-
- If one target fails, the entire group fails.
41-
- Final score = **100** if all succeed, otherwise **0**.
35+
* ≥ 1 success → **100**
36+
* No success → **0**
4237

43-
Example: 3 players succeed, 1 fails → Group score = **0 (Failed)**.
44-
Example: 4 players succeed → Group score = **100 (Success)**.
38+
#### Example
4539

46-
### At least one target (per group) must validate the expectation
40+
Group of 4 players:
4741

48-
- Success depends on **at least one target succeeding**.
49-
- The group is considered successful if one target validates.
50-
- Final score = **100** if ≥1 succeeds, otherwise **0**.
51-
52-
Example: 3 players fail, 1 succeeds → Group score = **100 (Success)**.
53-
Example: 4 players fail → Group score = **0 (Failed)**.
54-
55-
![Validation mode](assets/validation_mode.png)
56-
57-
## Expectation manipulation
58-
59-
### Add an expectation to an Inject
60-
61-
1. Navigate to the inject's content and click **Add expectations**.
62-
2. Select the type of expectation you want to add.
63-
3. Define its score and validation mode.
64-
65-
You can add multiple expectations to a single inject.
66-
67-
![Add expectations to an inject](assets/inject_expectations_list.png)
68-
69-
### Validate a manual expectation
70-
71-
- During a simulation, go to the **Animation tab → Validation screen** to manually validate expectations.
72-
73-
![Validate a manual expectation from the animation tab](assets/manual_expectation_validation_animation_tab.png)
74-
75-
- For atomic testing with manual injects, validation can be done directly in the **Overview tab**.
76-
77-
![Add the validation of a manual expectation in atomic testing](assets/add_manual_validation_atomic_testing.png)
78-
79-
![Add the validation of a manual expectation in atomic testing](assets/manual_expectation_validation_atomic_testing.png)
80-
81-
### Validate technical injects
82-
83-
- Users can manually add detection/prevention results for their custom security platforms.
84-
- Automatic validation will also occur when connected to compatible collectors.
85-
86-
![Add the validation of a technical expectation in atomic testing](assets/add_technical_expectation_validation.png)
87-
88-
![Add the validation of a technical expectation in atomic testing](assets/technical_expectation_validation.png)
89-
90-
### Update Rules
91-
92-
Expectation statuses propagate between entities as follows:
93-
94-
| Action | Behavior |
95-
|--------------------|------------------------------------------------------------------------|
96-
| Update Agent | Update its status (0 = fail, 100 = success). |
97-
| Update Asset | Asset becomes **valid (100)** only if **all its agents are valid**. |
98-
| Update Asset Group | Compute status depending on validation mode: |
99-
| | - **All assets must validate** → 100 only if all assets succeed. |
100-
| | - **At least one asset must validate** → 100 if at least one succeeds. |
101-
| Update Player | Compute status for the team depending on validation mode. |
102-
| Update Team | Propagate status to players. |
103-
104-
### Customize expectations
105-
106-
#### Default score
107-
108-
Each expectation has a **default score value** at creation, configurable via environment variables:
109-
110-
| Parameter | Environment variable | Default value | Description |
111-
|:-----------------------------------------------|:-----------------------------------------------|:--------------|:-------------------------------------------|
112-
| openaev.expectation.manual.default-score-value | OPENAEV_EXPECTATION_MANUAL_DEFAULT-SCORE-VALUE | 50 | Default score value for manual expectation |
113-
114-
#### Expiration time
115-
116-
Expectations must be validated within a time limit. Defaults are set in the system but can be overridden:
117-
118-
- Manual Response: By default if not manual expectations are set the expiration will done after 24 hours
119-
- Detection/Prevention: By default if no expectations are set the expiration will done after 6 hour
120-
121-
| Parameter | Environment variable | Default value | Description |
122-
|:-----------------------------------------------|:-----------------------------------------------|:--------------|:--------------------------------------------------------------------|
123-
| openaev.expectation.technical.expiration-time | OPENAEV_EXPECTATION_TECHNICAL_EXPIRATION-TIME | 21600 | Expiration time for Technical expectation (detection & prevention) |
124-
| openaev.expectation.detection.expiration-time | OPENAEV_EXPECTATION_DETECTION_EXPIRATION-TIME | 21600 | Expiration time for detection expectation |
125-
| openaev.expectation.prevention.expiration-time | OPENAEV_EXPECTATION_PREVENTION_EXPIRATION-TIME | 21600 | Expiration time for prevention expectation |
126-
| openaev.expectation.human.expiration-time | OPENAEV_EXPECTATION_HUMAN_EXPIRATION-TIME | 86400 | Expiration time for human expectation (manual, challenge & article) |
127-
| openaev.expectation.challenge.expiration-time | OPENAEV_EXPECTATION_CHALLENGE_EXPIRATION-TIME | 86400 | Expiration time for challenge expectation |
128-
| openaev.expectation.article.expiration-time | OPENAEV_EXPECTATION_ARTICLE_EXPIRATION-TIME | 86400 | Expiration time for article expectation |
129-
| openaev.expectation.manual.expiration-time | OPENAEV_EXPECTATION_MANUAL_EXPIRATION-TIME | 86400 | Expiration time for manual expectation |
130-
131-
A default expiration time is set for technical and human expectations. Users can override them for each type of
132-
expectations.
133-
134-
- In the UI
135-
136-
![Customize expiration time in UI](assets/expectation_customize_expiration_time.png)
137-
138-
When creating an expectation, users can set an expiration time. The system's default times are set on the form and users
139-
decide to override it.
140-
141-
Once the expiration time will be reached, the Expectation Expiration Manager will fail the expectation and will show the
142-
result as `Not Detected` or `Not Prevented` depending on the expectation's type.
143-
144-
## When is a Simulation Finished?
145-
146-
A simulation is considered **finished** once all defined expectations are resolved:
147-
148-
- **Technical expectations** → validated through **Detection, Prevention, Vulnerability checks**.
149-
- **Human expectations** → validated manually by the organizer.
42+
* 1 succeeds, 3 fail → **100 (Success)**
43+
* 4 fail → **0 (Failed)**

docs/usage/assets/add_technical_expectation_validation.png renamed to docs/usage/expectations/assets/add_technical_expectation_validation.png

File renamed without changes.

docs/usage/assets/expectation_customize_expiration_time.png renamed to docs/usage/expectations/assets/create_manual_expectation.png

File renamed without changes.
File renamed without changes.

docs/usage/assets/manual_expectation_validation_animation_tab.png renamed to docs/usage/expectations/assets/manual_expectation_validation_animation_tab.png

File renamed without changes.
File renamed without changes.
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
## Expectations — Scoring, Expiration & UI
2+
3+
This page explains how to **configure, validate and operate expectations**.
4+
5+
## Add an Expectation to an Inject
6+
7+
1. Open the Inject
8+
2. Click **Add expectations**
9+
3. Select the expectation type
10+
4. Configure: score, validation mode, expiration time (optional)
11+
12+
💡 You can attach **multiple expectations** to a single Inject.
13+
14+
![Create manual expectation](assets/create_manual_expectation.png)
15+
16+
## Validate Expectations
17+
18+
### Manual expectations
19+
20+
* During a simulation: **Animation → Validation**
21+
* During atomic testing: **Overview** tab
22+
23+
![Validate a manual expectation from the animation tab](assets/manual_expectation_validation_animation_tab.png)
24+
25+
### Technical expectations
26+
27+
* Automatically validated via collectors
28+
* Or validated manually by adding detection or prevention results
29+
30+
![Add the validation of a technical expectation in atomic testing](assets/add_technical_expectation_validation.png)
31+
32+
## Scoring
33+
34+
### Default score
35+
36+
Each expectation type has a **default score**, applied at creation.
37+
38+
| Type | Default score |
39+
| ------ | ------------- |
40+
| Manual | 50 |
41+
42+
Configured via environment variables.
43+
44+
| Parameter | Environment variable | Default value | Description |
45+
|:-----------------------------------------------|:-----------------------------------------------|:--------------|:-------------------------------------------|
46+
| openaev.expectation.manual.default-score-value | OPENAEV_EXPECTATION_MANUAL_DEFAULT-SCORE-VALUE | 50 | Default score value for manual expectation |
47+
48+
## Expiration Time
49+
50+
Expectations must validate **within a defined time window**.
51+
52+
If the time expires:
53+
54+
* the expectation **fails automatically**
55+
* the result is marked as:
56+
57+
* `Not Detected`
58+
* `Not Prevented`
59+
* or equivalent
60+
61+
### Default expiration values
62+
63+
| Expectation type | Default |
64+
| ---------------------------------- | -------- |
65+
| Detection / Prevention | 6 hours |
66+
| Human (manual, article, challenge) | 24 hours |
67+
68+
You can override expiration times:
69+
70+
* globally (environment variables)
71+
* per expectation (UI)
72+
73+
| Parameter | Environment variable | Default value | Description |
74+
|:-----------------------------------------------|:-----------------------------------------------|:--------------|:--------------------------------------------------------------------|
75+
| openaev.expectation.technical.expiration-time | OPENAEV_EXPECTATION_TECHNICAL_EXPIRATION-TIME | 21600 | Expiration time for Technical expectation (detection & prevention) |
76+
| openaev.expectation.detection.expiration-time | OPENAEV_EXPECTATION_DETECTION_EXPIRATION-TIME | 21600 | Expiration time for detection expectation |
77+
| openaev.expectation.prevention.expiration-time | OPENAEV_EXPECTATION_PREVENTION_EXPIRATION-TIME | 21600 | Expiration time for prevention expectation |
78+
| openaev.expectation.human.expiration-time | OPENAEV_EXPECTATION_HUMAN_EXPIRATION-TIME | 86400 | Expiration time for human expectation (manual, challenge & article) |
79+
| openaev.expectation.challenge.expiration-time | OPENAEV_EXPECTATION_CHALLENGE_EXPIRATION-TIME | 86400 | Expiration time for challenge expectation |
80+
| openaev.expectation.article.expiration-time | OPENAEV_EXPECTATION_ARTICLE_EXPIRATION-TIME | 86400 | Expiration time for article expectation |
81+
| openaev.expectation.manual.expiration-time | OPENAEV_EXPECTATION_MANUAL_EXPIRATION-TIME | 86400 | Expiration time for manual expectation |

0 commit comments

Comments
 (0)