diff --git a/README.md b/README.md index 7e28f34..002db0b 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ A Sphinx extension for discovering, linking, and documenting source code across ## Features - **Source Discovery**: Automatically discover source files in your project -- **Virtual Documentation**: Generate documentation from code without modifying source files +- **Analyse**: Parse source codes and extract specified markers with their metadata - **Code Linking**: Create intelligent links between code elements - **Sphinx Integration**: Seamless integration with existing Sphinx documentation @@ -19,6 +19,7 @@ Add to your `conf.py`: ```python extensions = ['sphinx_needs', 'sphinx_codelinks'] +src_trace_config_from_toml = "codelinks.toml" ``` ## Documentation @@ -28,7 +29,7 @@ Full documentation: https://codelinks.useblocks.com ## Components - **Source Discovery** ([`src/sphinx_codelinks/source_discover`](src/sphinx_codelinks/source_discover)): Code analysis and discovery -- **Virtual Docs** ([`src/sphinx_codelinks/virtual_docs`](src/sphinx_codelinks/virtual_docs)): Documentation generation +- **Analyse** ([`src/sphinx_codelinks/analyse`](src/sphinx_codelinks/analyse)): Documentation generation - **Sphinx Extension** ([`src/sphinx_codelinks/sphinx_extension`](src/sphinx_codelinks/sphinx_extension)): Sphinx integration - **Command Line** ([`src/sphinx_codelinks/cmd.py`](src/sphinx_codelinks/cmd.py)): CLI interface diff --git a/docs/source/components/configuration.rst b/docs/source/components/configuration.rst index b078bf1..09b7713 100644 --- a/docs/source/components/configuration.rst +++ b/docs/source/components/configuration.rst @@ -28,7 +28,7 @@ Specifies the path to a `TOML file `__ containing **Sphinx-Code .. code-block:: python # In conf.py - src_trace_config_from_toml = "src_trace.toml" + src_trace_config_from_toml = "codelinks.toml" When using a TOML configuration file: diff --git a/docs/source/components/directive.rst b/docs/source/components/directive.rst index 4012aa5..64f6395 100644 --- a/docs/source/components/directive.rst +++ b/docs/source/components/directive.rst @@ -3,7 +3,9 @@ Directive ========= -``CodeLinks`` provides the ``src-trace`` directive and it can be used in the following ways: +.. attention:: ``src-trace`` directive currently only supports :ref:`one-line need definition `. + +``CodeLinks`` provides ``src-trace`` directive and it can be used in the following ways: .. code-block:: rst diff --git a/docs/source/components/oneline.rst b/docs/source/components/oneline.rst index 63ee49b..8b0c470 100644 --- a/docs/source/components/oneline.rst +++ b/docs/source/components/oneline.rst @@ -91,14 +91,25 @@ For example, with the following **needs_fields** configuration: .. _`fields_config`: -.. code-block:: python +.. tabs:: + + .. code-tab:: python + + needs_fields = [ + {"name": "title"}, + {"name": "id"}, + {"name": "type", "default": "impl"}, + {"name": "links", "type": "list[str]", "default": []}, + ], + + .. code-tab:: toml - needs_fields=[ - {"name": "title"}, - {"name": "id"}, - {"name": "type", "default": "impl"}, - {"name": "links", "type": "list[str]", "default": []}, - ], + needs_fields = [ + { name = "title" }, + { name = "id" }, + { name = "type", default = "impl" }, + { name = "links", type = "list[str]", default = [] }, + ] the one-line comment shall be defined as follows: @@ -122,17 +133,26 @@ when it is not given in the need definition. For example, with the following needs_fields definition, -.. code-block:: python +.. tabs:: + + .. code-tab:: python + + needs_fields = [ + { + "name": "title" + }, + { + "name": "type", + "default": "implementation" + }, + ] + + .. code-tab:: toml - needs_fields = [ - { - "name": "title" - }, - { - "name": "type", - "default": "implementation" - }, - ] + needs_fields = [ + { name = "title" }, + { name = "type", default = "implementation" } + ] the following need definition in source code is equivalent to RST shown below: diff --git a/docs/source/components/write.rst b/docs/source/components/write.rst new file mode 100644 index 0000000..78d9ea3 --- /dev/null +++ b/docs/source/components/write.rst @@ -0,0 +1,49 @@ +.. _write: + +Write +===== + +The ``write`` command is used to generate file formats such as reStructuredText from the markers extracted by the ``analyse`` command. +This allows you to create documentation that includes links to source code based on the need ids specified in your code comments. + +Example usage +------------- + +With the following extracted markers fron the ``analyse`` command, + +.. code-block:: json + + { + "my_project": [ + { + "filepath": "/home/demo/git_repo/ub/sphinx-codelinks/tests/data/need_id_refs/dummy_1.cpp", + "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/need_id_refs/dummy_1.cpp#L3", + "source_map": { + "start": {"row": 2, "column": 13}, + "end": {"row": 2, "column": 51} + }, + "tagged_scope": "void dummy_func1(){\n //...\n }", + "need_ids": ["NEED_001", "NEED_002", "NEED_003", "NEED_004"], + "marker": "@need-ids:", + "type": "need-id-refs" + }, + ], + } + +The following RST file with :external+needs:ref:`needextend ` directive can be generated by the ``write rst`` command: + +.. code-block:: rst + + .. needextend:: NEED_001 + :remote-url: https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/need_id_refs/dummy_1.cpp#L3 + + .. needextend:: NEED_002 + :remote-url: https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/need_id_refs/dummy_1.cpp#L3 + + .. needextend:: NEED_003 + :remote-url: https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/need_id_refs/dummy_1.cpp#L3 + + .. needextend:: NEED_004 + :remote-url: https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/need_id_refs/dummy_1.cpp#L3 + +More examples can be found in `test cases `__ diff --git a/docs/source/development/change_log.rst b/docs/source/development/change_log.rst index 6d38699..a943dc7 100644 --- a/docs/source/development/change_log.rst +++ b/docs/source/development/change_log.rst @@ -3,7 +3,47 @@ Changelog ========= -.. _release:0.1.2: +.. _`release:1.0.0`: + +1.0.0 +----- + +:Released: 22.08.2025 + +New and Improved +................ + +- ✨ Added a new ``analyse`` CLI command and corresponding API. + + The ``analyse`` command parses source files (Python, C/C++) and extracts markers from comments. + It can extract three types of markers, as documented in the :ref:`analyse ` section: + + - One-line need definitions + - Need ID references + - Marked RST blocks + + The extracted markers and their metadata are saved to a JSON file for further processing. + +- ✨ Added a new ``write rst`` CLI command. + + The ``write rst`` command write a reStructuredText file with :external+needs:ref:`needextend ` directive from the extracted markers generated by ``analyse``. + The generated RST can be included in the Sphinx documentation to create the source code links in the existing needs + +- 🔨 Replaced ``virtual_docs`` with the new ``analyse`` module. + + The ``virtual_docs`` feature, which handled one-line need definitions (:ref:`OneLineCommentStyle `), + has been migrated into the new ``analyse`` module and removed from the core. + The caching feature of ``virtual_docs`` is temporarily removed and may be reintroduced later. + +- 🔨 Updated the ``src-trace`` Sphinx directive. + + The ``src-trace`` directive now uses the new ``analyse`` API instead of the old ``virtual_docs`` one. + +- 🔨 Unified configuration in TOML + + The configuration for ``src-trace`` directive defined in TOML is now compatible with the new ``analyse`` module. + +.. _`release:0.1.2`: 0.1.2 ----- @@ -24,7 +64,7 @@ Fixes Local links between docs and one-line need definitions work correctly, when :ref:`src_dir ` in multiple project configurations point at different locations. -.. _release:0.1.1: +.. _`release:0.1.1`: 0.1.1 ----- diff --git a/docs/source/development/roadmap.rst b/docs/source/development/roadmap.rst index b21529a..b193139 100644 --- a/docs/source/development/roadmap.rst +++ b/docs/source/development/roadmap.rst @@ -3,31 +3,31 @@ Roadmap ======= -Other Comment styles --------------------- +Command-Line Interface +---------------------- -Currently, only ``C/C++`` comment style is supported. -The other comment styles for different programming languages are planed, such as: +- Introduce configurable ``--verbose`` and ``--quiet`` logging options. -- Python -- Rust -- YAML -- SyML +Configuration Files +------------------- -Nested .gitignore ------------------ +- Support automatic discovery of TOML configuration files (e.g., ``pyproject.toml``). +- Improve integration with the ``ubCode`` and ``Sphinx-Needs`` ecosystems. -``CodeLinks`` respects ``.gitignore`` file, but if the .gitignore files are nested, it's not supported. -Respecting nested ``.gitignore`` in the context of the git repositories is planned. +Source Code Parsing +------------------- -Flexible way to define Sphinx-Needs need items in source code -------------------------------------------------------------- +- Introduce a configurable option to strip leading characters (e.g., ``*``) from commented RST blocks. +- Enrich tagged scopes with additional metadata. +- Extend language support by adding parsers for more comment styles, including but not limited to: -The only way to define ``Sphinx-Needs`` need items is through ``one-line comment style``. -Raw RST text and multi-lines comments style are planned to support + - Rust + - YAML -Export needs.json ------------------ +- Enhance ``.gitignore`` handling to support nested ``.gitignore`` files. -To facilitate CI workflow and enhance the portability of ``need items`` defined in source code, -we plan to have the feature to export the needs defined in source code to a JSON file. +Defining Needs in Source Code +----------------------------- + +- Introduce flexible ways to define ``Sphinx-Needs`` items in source code, such as using raw RST text and multi-line comments. +- Implement a feature to export needs defined in source code to a ``needs.json`` file, improving CI workflows and portability. diff --git a/docs/source/index.rst b/docs/source/index.rst index f1bcc25..94ff269 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -12,9 +12,11 @@ .. div:: sd-fs-5 sd-font-italic - ``Sphinx-CodeLinks`` is designed for Engineering-as-Code workflows to facilitate Application Lifecycle Management (ALM). - It enables users to define ``Sphinx-Needs`` need items within source code using a single line and automatically extract them - into the documentation during the Sphinx build process. + ``Sphinx-CodeLinks`` is a lightweight solution for enabling traceability between source code and documentation. + With a single line in your code, you can: + + 1. Defining new ``Sphinx-Needs`` - extracted and integrated into the Sphinx build + 2. Embedding existing need IDs - extracted for cross-referencing with need items in the documentation .. grid:: 1 1 2 2 :gutter: 2 2 3 3 @@ -63,6 +65,7 @@ Contents components/oneline components/analyse components/discover + components/write .. toctree:: :maxdepth: 1 diff --git a/pyproject.toml b/pyproject.toml index 67040ff..86018a4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "sphinx-codelinks" -version = "0.1.2" +version = "1.0.0" description = "Fast Source Code Traceability for Sphinx-Needs" authors = [{ name = "team useblocks", email = "info@useblocks.com" }] maintainers = [ diff --git a/src/sphinx_codelinks/README.md b/src/sphinx_codelinks/README.md deleted file mode 100644 index de5c307..0000000 --- a/src/sphinx_codelinks/README.md +++ /dev/null @@ -1,46 +0,0 @@ -# CodeLinks - -## Overview - -This is a Sphinx extension to extract Sphinx-Needs items from source files -such as C, C++ and others. - -The need items are defined in the source files as comments and can be used to define -test case specifications or implementation markers. - -Various definition styles are supported, such as one-line, multi-line or raw RST. - -The project consists of the following three components: - -- Source Discovery: determines list of source files from a given directory -- Virtual Docs: extract need annotations while keeping the source map -- Source Tracing: Sphinx extension to represent the collected the needs in the documentation - -`Source Discovery` and `Virtual Docs` can be used as `APIs` or `CLI tools`. -The detail usages can be found in the [test cases](./tests). - -The library is built to be - -- ⚡ fast for large code bases and -- 📃 support a multitude of languages. - -## Source Discovery - -Recursively collect the file paths from a given directory. -It can be configured to respect `.gitignore`. - -## Virtual Docs - -Virtual Docs parses the discoverd files and - -- extracts the need items from the comments in the source files. -- extracts additional metadata such as extra options and links. -- generates virtual documents containing the above-mentioned information into `json` files. -- caches virtual docs for incremental builds. -- keeps the source map to the path and line number of the original source files. - -## CodeLinks - -CodeLinks is a Sphinx Extension based on Sphinx-Needs. It provides the directive `src-tracing` -to collect the needs defined in source files by using `Source Discovery` and `Virtual Docs` -under the hood.