|
1 | | -# Expectations |
2 | 1 |
|
3 | | -Expectations define what is expected from an [Asset (endpoint)](assets.md) or |
4 | | -a [Players](people.md#players) when facing an [Inject](inject-overview.md) in terms of |
5 | | -security posture. Each expectation has a score representing how well it has been met by the target. |
6 | 2 |
|
7 | | -## Expectation types |
| 3 | +#### Why use it? |
8 | 4 |
|
9 | | -Expectations can be categorized as either Manual or Automatic, depending on how they are validated. |
| 5 | +Use this mode when **every member must succeed**, such as: |
10 | 6 |
|
11 | | -### Manual expectations |
| 7 | +* mandatory training |
| 8 | +* compliance requirements |
| 9 | +* baseline security checks |
12 | 10 |
|
13 | | -Manual expectations require validation by the **exercise organizer** (animation team). |
14 | | -They are simple, customizable, and **user-controlled**. |
| 11 | +#### How does it work? |
15 | 12 |
|
16 | | -Examples: |
| 13 | +* All targets succeed → **100** |
| 14 | +* At least one target fails → **0** |
17 | 15 |
|
18 | | -- **Team response validation** |
19 | | - Check whether the **incident response team** correctly acknowledges and escalates a phishing inject during the exercise. |
| 16 | +#### Example |
20 | 17 |
|
21 | | -- **Player task validation** |
22 | | - Ensure that an **analyst player** correctly follows the playbook and reports an incident in the ticketing system. |
| 18 | +Group of 4 players: |
23 | 19 |
|
24 | | -### Automatic expectations |
| 20 | +* 3 succeed, 1 fails → **0 (Failed)** |
| 21 | +* 4 succeed → **100 (Success)** |
25 | 22 |
|
26 | | -Automatic expectations are validated automatically under specific conditions. |
| 23 | +### At least one target must validate |
27 | 24 |
|
28 | | -- `Prevention`: automatically validated by security integrations (e.g., quarantine event). |
29 | | -- `Detection`: automatically validated by security integrations (e.g., incident alert). |
30 | | -- `Vulnerability`: automatically validated based on the presence of CVEs. |
31 | | -- `Expect targets to read the article(s)`: automatically validated once targets have read the injected article. |
| 25 | +#### Why use it? |
32 | 26 |
|
33 | | -## Validation Mode |
| 27 | +Use this mode when **a single successful response is enough**, such as: |
34 | 28 |
|
35 | | -There are two modes for validating an expectation : |
| 29 | +* SOC detection |
| 30 | +* on-call escalation |
| 31 | +* redundancy testing |
36 | 32 |
|
37 | | -### All targets (per group) must validate the expectation |
| 33 | +#### How does it work? |
38 | 34 |
|
39 | | -- The result depends on **every member's performance**. |
40 | | -- If one target fails, the entire group fails. |
41 | | -- Final score = **100** if all succeed, otherwise **0**. |
| 35 | +* ≥ 1 success → **100** |
| 36 | +* No success → **0** |
42 | 37 |
|
43 | | -Example: 3 players succeed, 1 fails → Group score = **0 (Failed)**. |
44 | | -Example: 4 players succeed → Group score = **100 (Success)**. |
| 38 | +#### Example |
45 | 39 |
|
46 | | -### At least one target (per group) must validate the expectation |
| 40 | +Group of 4 players: |
47 | 41 |
|
48 | | -- Success depends on **at least one target succeeding**. |
49 | | -- The group is considered successful if one target validates. |
50 | | -- Final score = **100** if ≥1 succeeds, otherwise **0**. |
51 | | - |
52 | | -Example: 3 players fail, 1 succeeds → Group score = **100 (Success)**. |
53 | | -Example: 4 players fail → Group score = **0 (Failed)**. |
54 | | - |
55 | | - |
56 | | - |
57 | | -## Expectation manipulation |
58 | | - |
59 | | -### Add an expectation to an Inject |
60 | | - |
61 | | -1. Navigate to the inject's content and click **Add expectations**. |
62 | | -2. Select the type of expectation you want to add. |
63 | | -3. Define its score and validation mode. |
64 | | - |
65 | | -You can add multiple expectations to a single inject. |
66 | | - |
67 | | - |
68 | | - |
69 | | -### Validate a manual expectation |
70 | | - |
71 | | -- During a simulation, go to the **Animation tab → Validation screen** to manually validate expectations. |
72 | | - |
73 | | - |
74 | | - |
75 | | -- For atomic testing with manual injects, validation can be done directly in the **Overview tab**. |
76 | | - |
77 | | - |
78 | | - |
79 | | - |
80 | | - |
81 | | -### Validate technical injects |
82 | | - |
83 | | -- Users can manually add detection/prevention results for their custom security platforms. |
84 | | -- Automatic validation will also occur when connected to compatible collectors. |
85 | | - |
86 | | - |
87 | | - |
88 | | - |
89 | | - |
90 | | -### Update Rules |
91 | | - |
92 | | -Expectation statuses propagate between entities as follows: |
93 | | - |
94 | | -| Action | Behavior | |
95 | | -|--------------------|------------------------------------------------------------------------| |
96 | | -| Update Agent | Update its status (0 = fail, 100 = success). | |
97 | | -| Update Asset | Asset becomes **valid (100)** only if **all its agents are valid**. | |
98 | | -| Update Asset Group | Compute status depending on validation mode: | |
99 | | -| | - **All assets must validate** → 100 only if all assets succeed. | |
100 | | -| | - **At least one asset must validate** → 100 if at least one succeeds. | |
101 | | -| Update Player | Compute status for the team depending on validation mode. | |
102 | | -| Update Team | Propagate status to players. | |
103 | | - |
104 | | -### Customize expectations |
105 | | - |
106 | | -#### Default score |
107 | | - |
108 | | -Each expectation has a **default score value** at creation, configurable via environment variables: |
109 | | - |
110 | | -| Parameter | Environment variable | Default value | Description | |
111 | | -|:-----------------------------------------------|:-----------------------------------------------|:--------------|:-------------------------------------------| |
112 | | -| openaev.expectation.manual.default-score-value | OPENAEV_EXPECTATION_MANUAL_DEFAULT-SCORE-VALUE | 50 | Default score value for manual expectation | |
113 | | - |
114 | | -#### Expiration time |
115 | | - |
116 | | -Expectations must be validated within a time limit. Defaults are set in the system but can be overridden: |
117 | | - |
118 | | -- Manual Response: By default if not manual expectations are set the expiration will done after 24 hours |
119 | | -- Detection/Prevention: By default if no expectations are set the expiration will done after 6 hour |
120 | | - |
121 | | -| Parameter | Environment variable | Default value | Description | |
122 | | -|:-----------------------------------------------|:-----------------------------------------------|:--------------|:--------------------------------------------------------------------| |
123 | | -| openaev.expectation.technical.expiration-time | OPENAEV_EXPECTATION_TECHNICAL_EXPIRATION-TIME | 21600 | Expiration time for Technical expectation (detection & prevention) | |
124 | | -| openaev.expectation.detection.expiration-time | OPENAEV_EXPECTATION_DETECTION_EXPIRATION-TIME | 21600 | Expiration time for detection expectation | |
125 | | -| openaev.expectation.prevention.expiration-time | OPENAEV_EXPECTATION_PREVENTION_EXPIRATION-TIME | 21600 | Expiration time for prevention expectation | |
126 | | -| openaev.expectation.human.expiration-time | OPENAEV_EXPECTATION_HUMAN_EXPIRATION-TIME | 86400 | Expiration time for human expectation (manual, challenge & article) | |
127 | | -| openaev.expectation.challenge.expiration-time | OPENAEV_EXPECTATION_CHALLENGE_EXPIRATION-TIME | 86400 | Expiration time for challenge expectation | |
128 | | -| openaev.expectation.article.expiration-time | OPENAEV_EXPECTATION_ARTICLE_EXPIRATION-TIME | 86400 | Expiration time for article expectation | |
129 | | -| openaev.expectation.manual.expiration-time | OPENAEV_EXPECTATION_MANUAL_EXPIRATION-TIME | 86400 | Expiration time for manual expectation | |
130 | | - |
131 | | -A default expiration time is set for technical and human expectations. Users can override them for each type of |
132 | | -expectations. |
133 | | - |
134 | | -- In the UI |
135 | | - |
136 | | - |
137 | | - |
138 | | -When creating an expectation, users can set an expiration time. The system's default times are set on the form and users |
139 | | -decide to override it. |
140 | | - |
141 | | -Once the expiration time will be reached, the Expectation Expiration Manager will fail the expectation and will show the |
142 | | -result as `Not Detected` or `Not Prevented` depending on the expectation's type. |
143 | | - |
144 | | -## When is a Simulation Finished? |
145 | | - |
146 | | -A simulation is considered **finished** once all defined expectations are resolved: |
147 | | - |
148 | | -- **Technical expectations** → validated through **Detection, Prevention, Vulnerability checks**. |
149 | | -- **Human expectations** → validated manually by the organizer. |
| 42 | +* 1 succeeds, 3 fail → **100 (Success)** |
| 43 | +* 4 fail → **0 (Failed)** |
0 commit comments