Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 28 additions & 5 deletions docs/source/basics/introduction.rst
Original file line number Diff line number Diff line change
@@ -1,14 +1,37 @@
Introduction
============

``CodeLinks`` is a Sphinx extension that provides the ``src-trace`` directive to establish traceability between source code and :external+needs:doc:`Sphinx-Needs <index>` items.
``CodeLinks`` is a versatile utility that enables fast source code tracing and connects it to
the :external+needs:doc:`Sphinx-Needs <index>` ecosystem.

At its core, ``CodeLinks`` uses a powerful ``Analyse`` to parse source code comments and extract valuable information. The ``Analyse`` can identify and extract three distinct types of content:
It has multiple components:

- **One-line need definitions**: Create new Sphinx-Needs directly from a single, :ref:`customized comment line <oneline>` in your source code.
- source code analyzer for multiple programming languages and comment styles
- generator for various output formats that contain the extracted markers or needs
- Sphinx extension that integrates ``CodeLinks`` with Sphinx-Needs
- CLI application to drive the analysis and generation process

The configuration for ``CodeLinks`` is done via a TOML file, which can be used
for both the :ref:`Sphinx extension <configuration>` and the :ref:`CLI application <cli>`.

The configuration determines how markers and languages are ingested and how the Sphinx extension should behave.

At its core, ``CodeLinks`` parses the source code structure and extracts source markers.
Source markers can be special comments or language-specific constructs like docstrings for Python.

``CodeLinks`` supports 3 distinct marker types:

- :ref:`One-line need definitions <oneline>`: Create new Sphinx-Needs directly from a single customized comment line
in your source code.
- **Need ID references**: Link code to existing need items without creating new ones, perfect for tracing implementations to requirements.
- **Marked RST text**: Extract blocks of reStructuredText embedded within comments, allowing you to include rich documentation with associated metadata right next to your code.

``src-trace`` directive then consumes ``One-line need definitions`` to generate traceability between source code and your documentation.
When used in a Sphinx context, a new :ref:`directive` creates items at the location where it is placed (for a subset
of the analyzed files/folders).

For use cases where ``CodeLinks`` should not integrate with Sphinx, but rather generate output files, the
:ref:`cli` can be used. Currently it can write out ``needextend`` directives for the need ID reference comment style.
Other output files are planned such as full need items as RST or needs.json.

The ``Analyse``, along with the ``SourceDiscovery`` module, also provides both a **Python API** for extensibility and a **CLI** for integration into CI/CD pipelines.
The availability of most commands as :ref:`cli` also helps integrate ``CodeLinks`` into build systems and CI/CD pipelines.
Focus is put on performance, portability and caching of processing steps.
2 changes: 1 addition & 1 deletion docs/source/basics/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Sphinx Config
.. literalinclude:: ./../../src_trace.toml
:caption: src_trace.toml
:language: toml
:lines: 27-29


One-line comment
----------------
Expand Down
208 changes: 140 additions & 68 deletions docs/source/components/analyse.rst
Original file line number Diff line number Diff line change
@@ -1,80 +1,125 @@
Analyse
=======
.. _analyse:

The ``Analyse`` is a :ref:`CLI tool <cli>` that also provides an API for programmatic use. Its primary function is to extract specific, marked content from comments within source code files.
Source Analyse
==============

It can extract three types of content:
The **Source Analyse** module is a powerful component of **Sphinx-CodeLinks** that extracts documentation-related content from source code comments. It provides both CLI and API interfaces for flexible integration into documentation workflows.

- Sphinx-Needs ID References
- Oneline needs (see :ref:`OneLineCommentStyle <oneline>`)
- Marked reStructuredText (RST) blocks
**Key Capabilities:**

Configuration
-------------
- Extract **Sphinx-Needs** ID references from source code comments
- Process custom one-line comment patterns for rapid documentation
- Extract marked reStructuredText (RST) blocks embedded in comments
- Generate structured JSON output for further processing
- Support for multiple programming language comment styles

The ``Analyse`` is configured using a ``toml`` file. The examples throughout this document are based on the following configuration:
Overview
--------

.. literalinclude:: ./../../../tests/data/analyse/default_config.toml
:caption: default_config.toml
:language: toml
Source Analyse works by parsing source code files and identifying specially marked comments that contain documentation information. This enables developers to embed documentation directly in their source code while maintaining clean separation between code and documentation.

This configuration instructs the analyse to extract ``Sphinx-Needs ID Refs`` and ``Marked rst text`` using the defined markers.
The module supports three primary extraction modes:

Sphinx-Needs ID Refs
--------------------
1. **Sphinx-Needs ID References** - Links between code and requirements/specifications
2. **One-line Needs** - Simplified syntax for creating documentation needs
3. **Marked RST Blocks** - Full reStructuredText content embedded in comments

Supported Content Types
-----------------------

Sphinx-Needs ID References
~~~~~~~~~~~~~~~~~~~~~~~~~~

Extract references to **Sphinx-Needs** items directly from source code comments, enabling traceability between code implementations and requirements.

One-line Needs
~~~~~~~~~~~~~~

Use simplified comment patterns to define **Sphinx-Needs** items without complex RST syntax. See :ref:`OneLineCommentStyle <oneline>` for detailed information.

Marked RST Blocks
~~~~~~~~~~~~~~~~~

Embed complete reStructuredText content within source code comments for rich documentation that can be extracted and processed.

Limitations
-----------

**Current Limitations:**

- **Language Support**: Only C/C++ (``//``, ``/* */``) and Python (``#``) comment styles are supported
- **Single Comment Style**: Each analysis run processes only one comment style at a time

Extraction Examples
-------------------

The following examples are configured with :ref:`the analyse configuration <analyse_config>`,

Sphinx-Needs ID References
~~~~~~~~~~~~~~~~~~~~~~~~~~

Below is an example of a C++ source file containing need ID references and the corresponding JSON output from the analyse.

.. tabs::

.. code-tab:: cpp

#include <iostream>
#include <iostream>

// @need-ids: need_001, need_002, need_003, need_004
void dummy_func1(){
//...
}
// @need-ids: need_001, need_002, need_003, need_004
void dummy_func1(){
//...
}

// @need-ids: need_003
int main() {
std::cout << "Starting demo_1..." << std::endl;
dummy_func1();
std::cout << "Demo_1 finished." << std::endl;
return 0;
}
// @need-ids: need_003
int main() {
std::cout << "Starting demo_1..." << std::endl;
dummy_func1();
std::cout << "Demo_1 finished." << std::endl;
return 0;
}

.. code-tab:: json

[
{
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L3",
"source_map": {
"start": { "row": 2, "column": 13 },
"end": { "row": 2, "column": 51 }
},
"tagged_scope": "void dummy_func1(){\n //...\n }",
"need_ids": ["need_001", "need_002", "need_003", "need_004"],
"marker": "@need-ids:",
"type": "need-id-refs"
},
{
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L8",
"source_map": {
"start": { "row": 7, "column": 13 },
"end": { "row": 7, "column": 21 }
},
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
"need_ids": ["need_003"],
"marker": "@need-ids:",
"type": "need-id-refs"
}
]

Marked RST
----------
[
{
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L3",
"source_map": {
"start": { "row": 2, "column": 13 },
"end": { "row": 2, "column": 51 }
},
"tagged_scope": "void dummy_func1(){\n //...\n }",
"need_ids": ["need_001", "need_002", "need_003", "need_004"],
"marker": "@need-ids:",
"type": "need-id-refs"
},
{
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L8",
"source_map": {
"start": { "row": 7, "column": 13 },
"end": { "row": 7, "column": 21 }
},
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
"need_ids": ["need_003"],
"marker": "@need-ids:",
"type": "need-id-refs"
}
]

**Output Structure:**

- ``filepath`` - Path to the source file containing the reference
- ``remote_url`` - URL to the source code in the remote repository
- ``source_map`` - Location information (row/column) of the marker
- ``tagged_scope`` - The code scope associated with the marker
- ``need_ids`` - List of referenced need IDs
- ``marker`` - The marker string used for identification
- ``type`` - Type of extraction ("need-id-refs")

Marked RST Blocks
~~~~~~~~~~~~~~~~~

This example demonstrates how the analyse extracts RST blocks from comments.

Expand Down Expand Up @@ -102,16 +147,15 @@ This example demonstrates how the analyse extracts RST blocks from comments.
return 0;
}


.. code-tab:: json

[
{
"filepath": "marked_rst/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/26b301138eef25c5130518d96eaa7a29a9c6c9fe/marked_rst/dummy_1.cpp#L4",
"source_map": {
"start": { "row": 3, "column": 8 },
"end": { "row": 3, "column": 61 }
"start": { "row": 3, "column": 8 },
"end": { "row": 3, "column": 61 }
},
"tagged_scope": "void dummy_func1(){\n //...\n }",
"rst": ".. impl:: implement dummy function 1\n :id: IMPL_71\n",
Expand All @@ -121,20 +165,48 @@ This example demonstrates how the analyse extracts RST blocks from comments.
"filepath": "marked_rst/dummy_1.cpp",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/26b301138eef25c5130518d96eaa7a29a9c6c9fe/marked_rst/dummy_1.cpp#L14",
"source_map": {
"start": { "row": 13, "column": 7 },
"end": { "row": 13, "column": 40 }
"start": { "row": 13, "column": 7 },
"end": { "row": 13, "column": 40 }
},
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
"rst": "..impl:: implement main function ",
"type": "rst"
}
]

Limitations
-----------
**Output Structure:**

- ``filepath`` - Path to the source file containing the RST block
- ``remote_url`` - URL to the source code in the remote repository
- ``source_map`` - Location information of the RST markers
- ``tagged_scope`` - The code scope associated with the RST block
- ``rst`` - The extracted reStructuredText content
- ``type`` - Type of extraction ("rst")

**RST Block Formats:**

The module supports both multi-line and single-line RST blocks:

- **Multi-line blocks**: Use ``@rst`` and ``@endrst`` on separate lines
- **Single-line blocks**: Use ``@rst content @endrst`` on the same line

One-line Needs
--------------

**One-line Needs** provide a simplified syntax for creating **Sphinx-Needs** items directly in source code comments without requiring full RST syntax.

For comprehensive information about one-line needs configuration and usage, see :ref:`OneLineCommentStyle <oneline>`.

**Basic Example:**

.. code-block:: c

// @Function Implementation, IMPL_001, impl, [REQ_001, REQ_002]

This single comment line creates a complete **Sphinx-Needs** item equivalent to:

Please be aware of the following limitations:
.. code-block:: rst

- **Supported Languages**: The analyse only supports comment styles for C/C++ (``//``, ``/*...*/``) and Python (``#``).
- **Single Comment Style**: An analysis run can only process a single comment style at a time.
- **Configuration Incompatibility**: The TOML configuration file cannot be shared with the ``CodeLink`` Sphinx extensions.
.. impl:: Function Implementation
:id: IMPL_001
:links: REQ_001, REQ_002
4 changes: 2 additions & 2 deletions docs/source/components/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
Command Line Interface (CLI)
============================

``Sphinx-CodeLinks`` provides CLI for users to integrate documentation build into CI/CD pipeline
``Sphinx-CodeLinks`` provides a CLI for users to integrate documentation builds into CI/CD pipelines
and for local development.

It features help pages. add ``-h`` or ``--help`` to any command to see the available options.
It features help pages. Add ``-h`` or ``--help`` to any command to see the available options.

.. typer:: sphinx_codelinks.cmd.app
:prog: codelinks
Expand Down
Loading
Loading