Skip to content

Commit 09c84f8

Browse files
authored
Merge pull request #543 from JensGrote/feat/mhc-anchor
feat: add Meaningful Human Control (MHC) anchor
2 parents 3f3cb0e + cce1891 commit 09c84f8

2 files changed

Lines changed: 154 additions & 0 deletions

File tree

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
= Meaningful Human Control (MHC)
2+
:categories: strategic-planning
3+
:roles: software-architect, product-owner, consultant, educator, team-lead, data-scientist, systems-thinker
4+
:related: regulated-environment, cynefin-framework, luhmann-system-theory, simon-constructivism, systemic-consulting
5+
:proponents: Article 36, Noel Sharkey, ICRC, UN CCW GGE, IEEE Global Initiative
6+
:tags: ethics, human-control, autonomy, weapons, ai-governance, accountability, human-rights
7+
:tier: 2
8+
9+
[%collapsible]
10+
====
11+
Full Name:: Meaningful Human Control
12+
13+
Also known as:: MHC, Meaningful Human Control over Individual Attacks
14+
15+
[discrete]
16+
== *Core Concepts*:
17+
18+
Substantive human control:: Humans must retain genuine, substantive control over autonomous systems making high-stakes decisions — not merely formulaic "human-in-the-loop" oversight
19+
20+
Situational awareness:: Human operators require adequate information about the target, context, weapon/system behavior, and foreseeable effects to make informed judgments
21+
22+
Accountability chain:: Those responsible for assessing information and executing actions must be clearly identifiable and accountable for outcomes; accountability cannot be transferred to machines
23+
24+
Sharkey's five levels:: Operational framework for human supervisory control: (L1) human deliberates before attack → (L2) program suggests targets, human chooses → (L3) program selects, human approves → (L4) program acts, human has restricted veto → (L5) fully autonomous without human involvement
25+
26+
Positive human action:: Initiating critical actions (especially use of force) requires affirmative human authorization, not merely passive monitoring
27+
28+
Predictable, reliable, transparent technology:: Autonomous systems must be designed for predictability, graceful degradation, and transparency to enable meaningful control
29+
30+
Timely intervention:: Human operators must retain the capability for timely intervention, override, or abort — not just pre-programmed constraints
31+
32+
Key Proponents:: Article 36 (coined the term, 2013), Noel Sharkey (five-level framework, 2014), ICRC (endorsed MHC as legal/ethical requirement, 2018), UN CCW GGE (Guiding Principles, 2019), IEEE Global Initiative (Ethically Aligned Design, 2019)
33+
34+
[discrete]
35+
== *When to Use*:
36+
37+
* Designing or evaluating autonomous systems in high-stakes domains (weapons systems, medical AI, autonomous driving, critical infrastructure)
38+
* Assessing whether human oversight of an AI system is genuinely effective or merely procedural
39+
* Defining accountability chains for autonomous decision-making
40+
* Requirements engineering for systems with autonomy in critical functions
41+
* AI governance and ethics discussions in product management and policy
42+
* Legal and compliance review of AI systems under emerging regulations (EU AI Act)
43+
44+
[discrete]
45+
== *When NOT to Use*:
46+
47+
* Low-stakes automation decisions (content recommendations, spam filtering, routine data processing)
48+
* As a substitute for domain-specific legal requirements (IHL, medical device regulation, data protection law)
49+
50+
[discrete]
51+
== *Related Anchors*:
52+
53+
* <<regulated-environment,Regulated Environment>>
54+
* <<cynefin-framework,Cynefin Framework>>
55+
* <<luhmann-system-theory,Luhmann System Theory>>
56+
* <<simon-constructivism,Simon Constructivism>>
57+
* <<systemic-consulting,Systemic Consulting>>
58+
59+
[discrete]
60+
== *Quality Criteria Checklist* (Tier-2 Justification)
61+
62+
This anchor is classified as *Tier 2 — Needs qualification*. It is not self-standing; it requires domain context and explicit verification criteria to be meaningfully applied. The following checklist documents the qualification requirements mapped to Issue #540.
63+
64+
. **Acceptance Criteria** (measurable)
65+
* High-stakes domain identified: yes/no — specify domain (weapons, medical, autonomous driving, critical infrastructure, algorithmic sentencing)
66+
* Autonomy level classified: yes/no — specify Sharkey level (L1–L5) for each critical function
67+
* Human operator identified: yes/no — specify role, training level, and decision authority
68+
* Accountability chain documented: yes/no — specify who decides, who approves, who audits
69+
* Legal/regulatory framework identified: yes/no — specify applicable law (IHL, EU AI Act, GDPR, etc.)
70+
71+
. **Evidence Types Required**
72+
* System specification documenting autonomy boundaries and human-machine interaction points
73+
* Operator training and qualification records
74+
* Audit trail design demonstrating human accountability at each decision point
75+
* Risk assessment showing residual risks after human oversight measures
76+
* Legal review confirming compliance with applicable regulatory framework
77+
78+
. **Minimum Documentation / Artifacts**
79+
* Human-Machine Interface (HMI) specification with override and abort mechanisms
80+
* Decision authority matrix (who decides what, under what conditions)
81+
* Operator situational awareness requirements (information feed, latency, decision time)
82+
* Graceful degradation protocol for communication loss or system failure
83+
* Test protocol validating human intervention capability under operational conditions
84+
85+
. **Validation Methods**
86+
* Simulated scenario testing with time-pressure and information-degradation conditions
87+
* Red-team evaluation of human-override effectiveness
88+
* Independent audit of accountability chain completeness
89+
* Legal review against applicable IHL or regulatory requirements
90+
* Post-deployment monitoring plan for measuring MHC erosion over time
91+
92+
. **Tier-2 Justification Summary**
93+
* *Why not Tier 3*: MHC cannot be evaluated without domain-specific context (weapons vs. medical vs. automotive). The same system may satisfy MHC in one domain and fail in another. Sharkey level L3 may be sufficient for cargo ships but not for lethal targeting.
94+
* *Why not Tier 1*: MHC is a well-established, multi-source concept with clear definition, consistent usage across domains, attributable origin, and rich conceptual activation.
95+
* *Qualification path*: To apply MHC, the user must answer: "Who controls what, with what information, under what constraints, and who is accountable?" The five criteria above operationalize this question.
96+
====
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
= Meaningful Human Control (MHC)
2+
:categories: strategic-planning
3+
:roles: software-architect, product-owner, consultant, educator, team-lead, data-scientist, systems-thinker
4+
:related: regulated-environment, cynefin-framework, luhmann-system-theory, simon-constructivism, systemic-consulting
5+
:proponents: Article 36, Noel Sharkey, ICRC, UN CCW GGE, IEEE Global Initiative
6+
:tags: ethik, menschliche-kontrolle, autonomie, waffen, ki-governance, verantwortlichkeit, menschenrechte
7+
:tier: 2
8+
9+
[%collapsible]
10+
====
11+
Full Name:: Meaningful Human Control (sinnerhaltende menschliche Kontrolle)
12+
13+
Also known as:: MHC, sinnerhaltende menschliche Kontrolle über einzelne Angriffe
14+
15+
[discrete]
16+
== *Kernkonzepte*:
17+
18+
Substantielle menschliche Kontrolle:: Menschen müssen echte, substanzielle Kontrolle über autonome Systeme behalten, die folgenreiche Entscheidungen treffen — nicht nur formelhafte "Human-in-the-Loop"-Aufsicht
19+
20+
Situationsbewusstsein:: Menschliche Bediener benötigen ausreichende Informationen über Ziel, Kontext, Systemverhalten und vorhersehbare Auswirkungen, um fundierte Urteile zu fällen
21+
22+
Verantwortungskette:: Die für die Informationsbewertung und Handlungsausführung Verantwortlichen müssen klar identifizierbar und rechenschaftspflichtig sein; Verantwortung kann nicht auf Maschinen übertragen werden
23+
24+
Sharkeys fünf Stufen:: Operativer Rahmen für menschliche überwachende Kontrolle: (S1) Mensch deliberiert vor Angriff → (S2) Programm schlägt Ziele vor, Mensch wählt → (S3) Programm wählt aus, Mensch genehmigt → (S4) Programm handelt, Mensch hat eingeschränktes Veto → (S5) vollautonom ohne menschliches Eingreifen
25+
26+
Positive menschliche Handlung:: Die Einleitung kritischer Aktionen (insbesondere Gewaltanwendung) erfordert eine aktive menschliche Autorisierung, nicht nur passive Überwachung
27+
28+
Vorhersagbare, zuverlässige, transparente Technologie:: Autonome Systeme müssen auf Vorhersagbarkeit, elegante Degradation und Transparenz ausgelegt sein, um sinnvolle Kontrolle zu ermöglichen
29+
30+
Rechtzeitiges Eingreifen:: Menschliche Bediener müssen die Fähigkeit zu rechtzeitigem Eingreifen, Übersteuern oder Abbruch behalten — nicht nur vorprogrammierte Einschränkungen
31+
32+
Key Proponents:: Article 36 (prägte den Begriff, 2013), Noel Sharkey (Fünf-Stufen-Rahmen, 2014), IKRK (befürwortete MHC als rechtliche/ethische Anforderung, 2018), UN CCW GGE (Leitprinzipien, 2019), IEEE Global Initiative (Ethically Aligned Design, 2019)
33+
34+
[discrete]
35+
== *Verwendung*:
36+
37+
* Entwurf oder Bewertung autonomer Systeme in kritischen Bereichen (Waffensysteme, medizinische KI, autonomes Fahren, kritische Infrastruktur)
38+
* Prüfung, ob die menschliche Aufsicht über ein KI-System wirklich wirksam oder nur prozessual ist
39+
* Definition von Verantwortungsketten für autonome Entscheidungsfindung
40+
* Anforderungsentwicklung für Systeme mit Autonomie in kritischen Funktionen
41+
* KI-Governance- und Ethikdiskussionen in Produktmanagement und Politik
42+
* Rechtliche und Compliance-Prüfung von KI-Systemen nach neuen Regulierungen (EU AI Act)
43+
44+
[discrete]
45+
== *Nicht Verwenden*:
46+
47+
* Bei risikoarmen Automatisierungsentscheidungen (Inhaltsempfehlungen, Spam-Filter, routinemäßige Datenverarbeitung)
48+
* Als Ersatz für domänenspezifische rechtliche Anforderungen (IHL, Medizinprodukteregulierung, Datenschutzrecht)
49+
50+
[discrete]
51+
== *Verwandte Anker*:
52+
53+
* <<regulated-environment,Regulierte Umgebung>>
54+
* <<cynefin-framework,Cynefin-Framework>>
55+
* <<luhmann-system-theory,Luhmann-Systemtheorie>>
56+
* <<simon-constructivism,Simon-Konstruktivismus>>
57+
* <<systemic-consulting,Systemische Beratung>>
58+
====

0 commit comments

Comments
 (0)