|
| 1 | += Meaningful Human Control (MHC) |
| 2 | +:categories: strategic-planning |
| 3 | +:roles: software-architect, product-owner, consultant, educator, team-lead, data-scientist, systems-thinker |
| 4 | +:related: regulated-environment, cynefin-framework, luhmann-system-theory, simon-constructivism, systemic-consulting |
| 5 | +:proponents: Article 36, Noel Sharkey, ICRC, UN CCW GGE, IEEE Global Initiative |
| 6 | +:tags: ethics, human-control, autonomy, weapons, ai-governance, accountability, human-rights |
| 7 | +:tier: 2 |
| 8 | + |
| 9 | +[%collapsible] |
| 10 | +==== |
| 11 | +Full Name:: Meaningful Human Control |
| 12 | + |
| 13 | +Also known as:: MHC, Meaningful Human Control over Individual Attacks |
| 14 | + |
| 15 | +[discrete] |
| 16 | +== *Core Concepts*: |
| 17 | + |
| 18 | +Substantive human control:: Humans must retain genuine, substantive control over autonomous systems making high-stakes decisions — not merely formulaic "human-in-the-loop" oversight |
| 19 | + |
| 20 | +Situational awareness:: Human operators require adequate information about the target, context, weapon/system behavior, and foreseeable effects to make informed judgments |
| 21 | + |
| 22 | +Accountability chain:: Those responsible for assessing information and executing actions must be clearly identifiable and accountable for outcomes; accountability cannot be transferred to machines |
| 23 | + |
| 24 | +Sharkey's five levels:: Operational framework for human supervisory control: (L1) human deliberates before attack → (L2) program suggests targets, human chooses → (L3) program selects, human approves → (L4) program acts, human has restricted veto → (L5) fully autonomous without human involvement |
| 25 | + |
| 26 | +Positive human action:: Initiating critical actions (especially use of force) requires affirmative human authorization, not merely passive monitoring |
| 27 | + |
| 28 | +Predictable, reliable, transparent technology:: Autonomous systems must be designed for predictability, graceful degradation, and transparency to enable meaningful control |
| 29 | + |
| 30 | +Timely intervention:: Human operators must retain the capability for timely intervention, override, or abort — not just pre-programmed constraints |
| 31 | + |
| 32 | +Key Proponents:: Article 36 (coined the term, 2013), Noel Sharkey (five-level framework, 2014), ICRC (endorsed MHC as legal/ethical requirement, 2018), UN CCW GGE (Guiding Principles, 2019), IEEE Global Initiative (Ethically Aligned Design, 2019) |
| 33 | + |
| 34 | +[discrete] |
| 35 | +== *When to Use*: |
| 36 | + |
| 37 | +* Designing or evaluating autonomous systems in high-stakes domains (weapons systems, medical AI, autonomous driving, critical infrastructure) |
| 38 | +* Assessing whether human oversight of an AI system is genuinely effective or merely procedural |
| 39 | +* Defining accountability chains for autonomous decision-making |
| 40 | +* Requirements engineering for systems with autonomy in critical functions |
| 41 | +* AI governance and ethics discussions in product management and policy |
| 42 | +* Legal and compliance review of AI systems under emerging regulations (EU AI Act) |
| 43 | + |
| 44 | +[discrete] |
| 45 | +== *When NOT to Use*: |
| 46 | + |
| 47 | +* Low-stakes automation decisions (content recommendations, spam filtering, routine data processing) |
| 48 | +* As a substitute for domain-specific legal requirements (IHL, medical device regulation, data protection law) |
| 49 | + |
| 50 | +[discrete] |
| 51 | +== *Related Anchors*: |
| 52 | + |
| 53 | +* <<regulated-environment,Regulated Environment>> |
| 54 | +* <<cynefin-framework,Cynefin Framework>> |
| 55 | +* <<luhmann-system-theory,Luhmann System Theory>> |
| 56 | +* <<simon-constructivism,Simon Constructivism>> |
| 57 | +* <<systemic-consulting,Systemic Consulting>> |
| 58 | + |
| 59 | +[discrete] |
| 60 | +== *Quality Criteria Checklist* (Tier-2 Justification) |
| 61 | + |
| 62 | +This anchor is classified as *Tier 2 — Needs qualification*. It is not self-standing; it requires domain context and explicit verification criteria to be meaningfully applied. The following checklist documents the qualification requirements mapped to Issue #540. |
| 63 | + |
| 64 | +. **Acceptance Criteria** (measurable) |
| 65 | +* High-stakes domain identified: yes/no — specify domain (weapons, medical, autonomous driving, critical infrastructure, algorithmic sentencing) |
| 66 | +* Autonomy level classified: yes/no — specify Sharkey level (L1–L5) for each critical function |
| 67 | +* Human operator identified: yes/no — specify role, training level, and decision authority |
| 68 | +* Accountability chain documented: yes/no — specify who decides, who approves, who audits |
| 69 | +* Legal/regulatory framework identified: yes/no — specify applicable law (IHL, EU AI Act, GDPR, etc.) |
| 70 | + |
| 71 | +. **Evidence Types Required** |
| 72 | +* System specification documenting autonomy boundaries and human-machine interaction points |
| 73 | +* Operator training and qualification records |
| 74 | +* Audit trail design demonstrating human accountability at each decision point |
| 75 | +* Risk assessment showing residual risks after human oversight measures |
| 76 | +* Legal review confirming compliance with applicable regulatory framework |
| 77 | + |
| 78 | +. **Minimum Documentation / Artifacts** |
| 79 | +* Human-Machine Interface (HMI) specification with override and abort mechanisms |
| 80 | +* Decision authority matrix (who decides what, under what conditions) |
| 81 | +* Operator situational awareness requirements (information feed, latency, decision time) |
| 82 | +* Graceful degradation protocol for communication loss or system failure |
| 83 | +* Test protocol validating human intervention capability under operational conditions |
| 84 | + |
| 85 | +. **Validation Methods** |
| 86 | +* Simulated scenario testing with time-pressure and information-degradation conditions |
| 87 | +* Red-team evaluation of human-override effectiveness |
| 88 | +* Independent audit of accountability chain completeness |
| 89 | +* Legal review against applicable IHL or regulatory requirements |
| 90 | +* Post-deployment monitoring plan for measuring MHC erosion over time |
| 91 | + |
| 92 | +. **Tier-2 Justification Summary** |
| 93 | +* *Why not Tier 3*: MHC cannot be evaluated without domain-specific context (weapons vs. medical vs. automotive). The same system may satisfy MHC in one domain and fail in another. Sharkey level L3 may be sufficient for cargo ships but not for lethal targeting. |
| 94 | +* *Why not Tier 1*: MHC is a well-established, multi-source concept with clear definition, consistent usage across domains, attributable origin, and rich conceptual activation. |
| 95 | +* *Qualification path*: To apply MHC, the user must answer: "Who controls what, with what information, under what constraints, and who is accountable?" The five criteria above operationalize this question. |
| 96 | +==== |
0 commit comments