msp json rev2

[xznhj8129]

User description

Final-ish revision to structure and docgen for now, will focus on other stuff, correct various type errors in the spec and removes redundant fields

---

PR Type

Enhancement, Documentation

---

Description

• Restructured array field definitions in MSP JSON schema

  • Removed redundant array_ctype field from all array definitions
  • Split array type info: ctype now contains base element type
  • Added array_size_define field for symbolic size references

• Enhanced documentation generation tools

  • Added format_ctype() function for proper array type formatting
  • Added describe_array_bytes() helper for size calculations
  • Improved sizeof_entry() logic for array size handling

• Generated JSON enum export and improved build tooling

  • Added JSON output generation for enumerations (inav_enums.json)
  • Made INAV source path configurable via command-line argument
  • Updated build script with checksum tracking and hash display

• Corrected enum references and field descriptions throughout spec

  • Fixed sensor hardware enum names (e.g., accHardware_e → accelerationSensor_e)
  • Added missing enum definitions and corrected type references
  • Updated documentation with proper array size notations

---

Diagram Walkthrough

flowchart LR
  A["Array Field Schema"] -->|Remove array_ctype| B["Simplified Structure"]
  B -->|Add array_size_define| C["Enhanced Metadata"]
  D["Gen Tools"] -->|format_ctype| E["Better Formatting"]
  D -->|describe_array_bytes| E
  F["Build Process"] -->|JSON Export| G["inav_enums.json"]
  F -->|Configurable Paths| G
  H["Enum References"] -->|Corrections| I["Updated Spec"]

Loading

File Walkthrough

	Relevant files
Enhancement	4 files gen_msp_md.py Enhanced array formatting and size calculation logic          +67/-12  gen_enum_md.py Added JSON enumeration export functionality                            +7/-0      get_all_inav_enums_h.py Made INAV source path configurable via arguments                  +17/-2    gen_docs.sh Updated build script with path configuration and checksums +6/-3
Bug fix	1 files msp_messages.json Restructured array definitions and corrected enum references +138/-159
Documentation	5 files msp_ref.md Updated documentation format and array size notations        +96/-87  inav_enums_ref.md Added new enums and corrected existing enum definitions    +90/-42  format.md Updated schema documentation with new array field structure +25/-17  docs_v2_header.md Simplified revision header format                                                +1/-1      original_msp_ref.md Added deprecation warning to obsolete reference                    +2/-1
Configuration changes	2 files msp_messages.checksum Updated checksum for revised JSON schema                                  +1/-1      rev Incremented revision number to version 2                                  +1/-1

---

[qodo-code-review]

PR Compliance Guide 🔍

(Compliance updated until commit e3aaf9d)

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Unsafe file write Description: The function now writes inav_enums.json to the current working directory without validation or user control, which can overwrite existing files and enables potential path confusion or unintended distribution of data if run in untrusted directories. gen_enum_md.py [259-279] Referred Code out.append(f"- [{e.name}](#enum-{e.name.lower()})") out.append("") for e in sorted(enums, key=lambda x: x.name.lower()): jsonfile[e.name] = {} out.append("---") out.append(f"## <a id=\"enum-{e.name.lower()}\"></a>`{e.name}`\n") if e.source_note: out.append(f"> Source: {e.source_note}\n") jsonfile[e.name]['_source'] = e.source_note out.append("| Enumerator | Value | Condition |") out.append("|---|---:|---|") for it in e.items: name_md = f"`{it.name}`" val = it.value_display cond = it.cond out.append(f"| {name_md} | {val} | {cond} |") jsonfile[e.name][name_md.strip('`')] = [val, cond] if len(cond)>0 else val out.append("") # While we're at it, chuck this all into a JSON file Path("inav_enums.json").write_text(json.dumps(jsonfile,indent=4), encoding="utf-8") return "\n".join(out)
	Integrity check bypass Description: The script echoes the checksum hash and updates msp_messages.checksum automatically on mismatch without verification or user confirmation, which could allow unnoticed tampering of msp_messages.json to persist by updating the trusted checksum. gen_docs.sh [1-17] Referred Code INAV_MAIN_PATH="../../../src/main" echo "###########" echo get_all_inav_enums_h.py python get_all_inav_enums_h.py --inav-root "$INAV_MAIN_PATH" echo "###########" echo "msp_messages.json checksum" actual="$(md5sum msp_messages.json | awk '{print $1}')" expected="$(awk '{print $1}' msp_messages.checksum)" echo "Hash:" $actual if [[ "$actual" != "$expected" ]]; then n="$(cat rev)" echo $((n + 1)) > rev echo "File changed, incrementing revision" echo $actual > msp_messages.checksum fi
	CI pipeline blockage Description: Adding an interactive read -n 1 -s -r -p "Press any key to continue" blocks non-interactive or CI executions and can cause stuck pipelines, enabling denial-of-service in automated environments. gen_docs.sh [24-27] Referred Code echo gen_enum_md.py python gen_enum_md.py rm all_enums.h read -n 1 -s -r -p "Press any key to continue"
	Unvalidated path input Description: The --inav-root argument is used to construct a path without validation or sandboxing, which could read arbitrary files if untrusted input is supplied; at minimum, it lacks checks to ensure it points inside the expected repository tree. get_all_inav_enums_h.py [94-107] Referred Code parser = argparse.ArgumentParser(description='Collect all enums from INAV sources.') parser.add_argument( '--inav-root', default='../inav/src/main', help="Path to the INAV 'src/main' directory (default: %(default)s)", ) return parser.parse_args() args = parse_args() base_dir = Path(args.inav_root).expanduser() for sd in SUBDIRS: root = base_dir / sd if not root.is_dir():
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Missing audit log: The new generation of an external JSON file inav_enums.json lacks any logging of the action (who/when/what), making auditability of this critical generation step unclear. Referred Code out.append(f"- [{e.name}](#enum-{e.name.lower()})") out.append("") for e in sorted(enums, key=lambda x: x.name.lower()): jsonfile[e.name] = {} out.append("---") out.append(f"## <a id=\"enum-{e.name.lower()}\"></a>`{e.name}`\n") if e.source_note: out.append(f"> Source: {e.source_note}\n") jsonfile[e.name]['_source'] = e.source_note out.append("| Enumerator | Value | Condition |") out.append("|---|---:|---|") for it in e.items: name_md = f"`{it.name}`" val = it.value_display cond = it.cond out.append(f"| {name_md} | {val} | {cond} |") jsonfile[e.name][name_md.strip('`')] = [val, cond] if len(cond)>0 else val out.append("") # While we're at it, chuck this all into a JSON file Path("inav_enums.json").write_text(json.dumps(jsonfile,indent=4), encoding="utf-8") return "\n".join(out) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: No write error handling: Writing inav_enums.json via Path(...).write_text(...) occurs without try/except or validation, so file I/O errors or serialization issues would fail without context. Referred Code # While we're at it, chuck this all into a JSON file Path("inav_enums.json").write_text(json.dumps(jsonfile,indent=4), encoding="utf-8") return "\n".join(out) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Unstructured logging: The script echoes raw hashes and messages without structured format, which may hinder automated auditing and could inadvertently expose internal paths or metadata depending on environment. Referred Code INAV_MAIN_PATH="../../../src/main" echo "###########" echo get_all_inav_enums_h.py python get_all_inav_enums_h.py --inav-root "$INAV_MAIN_PATH" echo "###########" echo "msp_messages.json checksum" actual="$(md5sum msp_messages.json | awk '{print $1}')" expected="$(awk '{print $1}' msp_messages.checksum)" echo "Hash:" $actual if [[ "$actual" != "$expected" ]]; then n="$(cat rev)" echo $((n + 1)) > rev echo "File changed, incrementing revision" echo $actual > msp_messages.checksum fi Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Input not validated: The new --inav-root argument is used to construct paths without validation or safeguards (e.g., existence checks beyond per-subdir, traversal constraints), which could lead to unexpected file scanning. Referred Code parser = argparse.ArgumentParser(description='Collect all enums from INAV sources.') parser.add_argument( '--inav-root', default='../inav/src/main', help="Path to the INAV 'src/main' directory (default: %(default)s)", ) return parser.parse_args() args = parse_args() base_dir = Path(args.inav_root).expanduser() for sd in SUBDIRS: root = base_dir / sd if not root.is_dir(): Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

---

Previous compliance checks

Compliance check up to commit e3aaf9d

Security Compliance
⚪	Uncontrolled file write Description: The function now writes a JSON file inav_enums.json to the current working directory without validation or user control, which could overwrite existing files or leak data depending on where the script is run; this uncontrolled file output presents a possible path for data exposure or unintended file modification. gen_enum_md.py [253-279] Referred Code jsonfile = {} out = [] out.append("# Enumerations\n") out.append("**Auto-generated reference for MSP, refer to source for development, not this file, due to variations with #ifdefs which needs verification.**\n") out.append("## Table of contents\n") for e in sorted(enums, key=lambda x: x.name.lower()): out.append(f"- [{e.name}](#enum-{e.name.lower()})") out.append("") for e in sorted(enums, key=lambda x: x.name.lower()): jsonfile[e.name] = {} out.append("---") out.append(f"## <a id=\"enum-{e.name.lower()}\"></a>`{e.name}`\n") if e.source_note: out.append(f"> Source: {e.source_note}\n") jsonfile[e.name]['_source'] = e.source_note out.append("| Enumerator | Value | Condition |") out.append("|---|---:|---|") for it in e.items: name_md = f"`{it.name}`" val = it.value_display cond = it.cond ... (clipped 6 lines)
	Insecure build script usage Description: The script derives expected by awk '{print $1}' msp_messages.checksum and updates the checksum file with only the raw hash, altering the previously two-field format and echoing the hash; while primarily integrity logic, invoking external tools and modifying files based on computed hashes could be abused if run in an untrusted directory (e.g., path injection in tools or manipulated files), representing a possible integrity risk in automation contexts. gen_docs.sh [1-17] Referred Code INAV_MAIN_PATH="../../../src/main" echo "###########" echo get_all_inav_enums_h.py python get_all_inav_enums_h.py --inav-root "$INAV_MAIN_PATH" echo "###########" echo "msp_messages.json checksum" actual="$(md5sum msp_messages.json | awk '{print $1}')" expected="$(awk '{print $1}' msp_messages.checksum)" echo "Hash:" $actual if [[ "$actual" != "$expected" ]]; then n="$(cat rev)" echo $((n + 1)) > rev echo "File changed, incrementing revision" echo $actual > msp_messages.checksum fi
	Unvalidated path input Description: The script accepts a user-provided --inav-root path and recursively scans it without sandboxing or validation, which could enable unintended traversal and processing of large or sensitive directory trees if misused, posing a possible information exposure or DoS risk when run with broad permissions. get_all_inav_enums_h.py [94-107] Referred Code parser = argparse.ArgumentParser(description='Collect all enums from INAV sources.') parser.add_argument( '--inav-root', default='../inav/src/main', help="Path to the INAV 'src/main' directory (default: %(default)s)", ) return parser.parse_args() args = parse_args() base_dir = Path(args.inav_root).expanduser() for sd in SUBDIRS: root = base_dir / sd if not root.is_dir():
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Unlogged Action: The function now writes a JSON file (inav_enums.json) to disk without any accompanying audit/log entry indicating who triggered it, when, and the outcome. Referred Code # While we're at it, chuck this all into a JSON file Path("inav_enums.json").write_text(json.dumps(jsonfile,indent=4), encoding="utf-8") Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Missing Error Handling: New file write of inav_enums.json does not handle potential I/O errors (exceptions, permissions, disk full) or report actionable context. Referred Code # While we're at it, chuck this all into a JSON file Path("inav_enums.json").write_text(json.dumps(jsonfile,indent=4), encoding="utf-8") return "\n".join(out) Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Unstructured Logging: The script echoes raw checksum values and messages without structured logging, which may hinder automated auditing though no sensitive data is exposed. Referred Code echo "Hash:" $actual if [[ "$actual" != "$expected" ]]; then n="$(cat rev)" echo $((n + 1)) > rev echo "File changed, incrementing revision" echo $actual > msp_messages.checksum fi Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Input Validation: The new --inav-root argument is accepted and used to traverse the filesystem without validation or safeguards (e.g., path existence already checked per subdir, but no normalization or restriction), which may pose risks depending on execution context. Referred Code parser = argparse.ArgumentParser(description='Collect all enums from INAV sources.') parser.add_argument( '--inav-root', default='../inav/src/main', help="Path to the INAV 'src/main' directory (default: %(default)s)", ) return parser.parse_args() args = parse_args() base_dir = Path(args.inav_root).expanduser() for sd in SUBDIRS: root = base_dir / sd if not root.is_dir(): Learn more about managing compliance generic rules or creating your own custom rules

[qodo-code-review]

PR Code Suggestions ✨

No code suggestions found for the PR.