Replace fragile per-type import bookkeeping with ffigen symbol-file dedup#76
Replace fragile per-type import bookkeeping with ffigen symbol-file dedup#76JSUYA wants to merge 8 commits into
Conversation
configs/<version>/modules.yaml now describes every module (library, headers, type-map imports, enum renames, ...) and the generation scripts derive symgen.yaml, entrypoints.h, entrypoints_*.h, and ffigen_*.yaml from it: - scripts/build_configs.py: render derived configs from modules.yaml - scripts/import_configs.py: bootstrap modules.yaml from existing configs (one-time migration helper) - scripts/verify_configs.py: semantic diff between two config dirs - scripts/generate_tizen.py: derive UnnamedUnion/UnnamedStruct hide clauses from the binding files instead of hardcoding the time module Verified: build_configs.py output is semantically identical to the committed configs for all six versions, and regenerating the 6.0 bindings (symgen + ffigen + convert_description + generate_tizen) reproduces the committed files byte-for-byte.
configs/<version>/ now contains only the hand-maintained modules.yaml. symgen.yaml, entrypoints*.h, and ffigen_*.yaml are rendered into build/configs/<version>/ by scripts/build_configs.py when running generate_bindings.sh or generate_callbacks.sh, so the ~780 derived files no longer need to be committed and kept in sync by hand. - generate_bindings.sh now also runs convert_description.dart and generate_tizen.py, covering the whole per-version generation flow. - Add symgen_copyright_year to modules.yaml so rendered symgen.yaml preserves each version's original copyright year. - Add scripts/check_consistency.py and a Verify workflow that render all configs, check modules.yaml against lib/, and analyze lib. - Add .pubignore to keep build-time-only files out of the package.
Cross-module shared types were deduplicated by hand-enumerating every typedef/struct/union in per-module `imports` lists in modules.yaml, which build_configs.py rendered into library-imports + type-map blocks. Each new duplicate required finding the owning module and appending the exact type name, and the lists (48-50 edges per version) had to be kept in sync with the C headers. Replace the enumerated lists with a plain `deps` list of module names and use ffigen's native symbol-file mechanism instead: a module named in another module's deps exports a symbol file (output.symbol-file), and the depending module imports it (import.symbol-files), so ffigen references the provider's declarations for every shared type automatically. The dependency edges are unchanged; only the per-type bookkeeping is gone (-1,288 lines across the six manifests). - build_configs.py renders provider/consumer blocks from deps, keeps type-map only for dart:ffi primitive_typedefs, and emits ffigen_order.txt (topological order, providers first) since a consumer needs its providers' symbol files to exist. - generate_bindings.sh runs ffigen in that order and then runs the new rename_unnamed.py, which prefixes ffigen's per-file UnnamedStructN/ UnnamedUnionN names with the module stem so they are globally unique and no longer need hide clauses. - generate_tizen.py now hides any duplicated top-level export (classes, enums, typedefs, and const macros), recognizing symbol-file aliases (typedef X = impN.X;) so the real declaration keeps ownership. - resolve_type_dups.py derives a single deps edge per duplicate pair instead of growing enumerated type lists, and regenerates changed modules in dependency order. - check_consistency.py additionally validates that deps name existing modules and form a DAG. The 6.0 dependency edges and rendered configs are identical to the ones validated on jsuya/ffigen-6.0-pushable (dart analyze clean, integration test 56/57 on RPI4, the one failure being an unrelated NOT_SUPPORTED).
Follow-up cleanup to the symbol-file dedup pipeline, addressing coupling and script sprawl without changing any generated output. - Unify the snake_case -> UpperCamelCase convention into a single modules_yaml.to_upper_camel(). It was implemented three times (build_configs, generate_tizen, rename_unnamed) with two different algorithms (slice-upper vs str.title()) that silently diverge on name parts containing digits or capitals, so the class name and the anonymous-type prefix could disagree. - Make build_configs' include groups data-driven: any group present in modules.yaml is now emitted, instead of being silently dropped unless its key was hardcoded in the renderer. - generate_bindings.sh wipes build/configs/<version> before rendering, so a removed or renamed module leaves no stale ffigen_*.yaml or .symbols/*.yaml behind. - Remove dead code: the unused to_camel_case(), and the unreachable entrypoint-insertion block in update_modules (an early return made it and its `inserted` bookkeeping unreachable). - Delete migration-only scripts now that all six modules.yaml are committed: import_configs.py (bootstrap done), verify_configs.py (round-trip proof done), scan_rootstrap.py (its rootstrap_manifest.yaml baseline was never committed or used). Verified: Tizen 6.0 regeneration is byte-identical to the pre-refactor output (118 files), dart analyze is clean, and check_consistency passes for all six versions.
Continues the script-sprawl cleanup by dropping the "rootstrap module update" cluster that only the per-module ffi/dart split ever needed. - Delete update_modules.py: an undocumented, over-engineered bootstrapper for a new version's modules.yaml (token-vote category prediction plus multi-tier header synthesis). The documented workflow copies the previous version's modules.yaml by hand; this parallel path was never referenced by the guide and produced entries that had to be reviewed anyway. - Slim rootstrap_index.py (258 -> 186 lines) down to what resolve_type_dups actually uses -- pkg-config Requires direction and include-graph reachability. Drop the manifest persistence (to_manifest / from_manifest / open) and the update-only queries (header_exists / expand_glob / includedir_owners); the rootstrap_manifest.yaml baseline they served was never committed. - resolve_type_dups.py now builds the ownership index only from a live rootstrap (the manifest fallback had no producer left); it still degrades cleanly to dependency in-degree / file order when no rootstrap is present. - Remove modules_yaml naming helpers (lib_to_module_name, soname_of) that only update_modules / import_configs used. Verified: rootstrap_index's kept queries and resolve_type_dups' Ownership.decide run correctly against the Tizen 6.0 rootstrap; the generate path is unaffected -- 6.0 regeneration is byte-identical, dart analyze is clean, and check_consistency passes for all six versions.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a .pubignore file to exclude development, configuration, and build-time-only files (such as .github/, .gemini/, configs/, and scripts/) from being published to the pub package registry, while also replicating relevant .gitignore rules. There are no review comments, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
The Verify workflow regenerates lib/<version>/tizen.dart with generate_tizen.py and fails if the result differs from what is committed. The symbol-file dedup change reworked hide computation (compute_hide_map) to sort the hidden names alphabetically, whereas the previous code emitted union names before struct names. The committed files still had the old ordering (`hide UnnamedUnion1, UnnamedStruct1`), so the check failed. Regenerate the six tizen.dart files so they match the current generator (`hide UnnamedStruct1, UnnamedUnion1`). The hidden sets are unchanged, so this is ordering-only; dart analyze stays clean.
6d039fd to
04eb550
Compare
|
Have you tested this plugin in tizen-fs? |
This PR removes the intermediate output, the ffi file. I have checked that the APIs in integration_test are working correctly. I have not yet tested tizen home, as there are parts that require additional editing starting from the rootstrap level. Are there any anticipated issues? |
If this plugin works correctly with tizen-fs, there shouldn't be any issues. |
I am currently merging this PR with the internal tizenfs branch. I will share the results after testing. |
Since tizen-fs uses the 9.0 API, I have create 9.0 bindings and rebased the commits for tizen-fs to conduct tests. |
2 participants
Shared types between modules were deduplicated by hand in configs//modules.yaml.
Every type shared across two modules had to be enumerated one by one:
48–50 such entries per version, kept in sync with the C headers by hand.
Miss one and the type re-duplicates, causing an ambiguous-export error.
Anonymous structs/unions collided on ffigen's coincidental UnnamedUnion1 names and were hidden from tizen.dart, making them unreachable.
So in this PR,
Replaces the manual type lists with a plain deps list, backed by ffigen's native symbol-file dedup:
declarations for all shared types — no per-type list to maintain.
It also removes now-obsolete generator scripts: the one-time migration/verification helpers (import_configs.py, verify_configs.py) and the
orphaned version-bootstrap tooling (update_modules.py, scan_rootstrap.py), trimming rootstrap_index.py to what resolve_type_dups.py still
needs.
Structure and scripts only — the generated bindings under lib/ are unchanged and will be regenerated in per-version follow-up PRs.