Skip to content

Commit 28337c9

Browse files
juiwenchenjuiwenchen
authored
Merge pull request #22 from useblocks/compatible-config
🔨 Same TOML configuration for src-trace and analyse
2 parents 133cb8c + 69ca1dc commit 28337c9

38 files changed

Lines changed: 1587 additions & 1241 deletions

‎docs/source/basics/introduction.rst‎

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,37 @@
11
Introduction
22
============
33

4-
``CodeLinks`` is a Sphinx extension that provides the ``src-trace`` directive to establish traceability between source code and :external+needs:doc:`Sphinx-Needs <index>` items.
4+
``CodeLinks`` is a versatile utility that enables fast source code tracing and connects it to
5+
the :external+needs:doc:`Sphinx-Needs <index>` ecosystem.
56

6-
At its core, ``CodeLinks`` uses a powerful ``Analyse`` to parse source code comments and extract valuable information. The ``Analyse`` can identify and extract three distinct types of content:
7+
It has multiple components:
78

8-
- **One-line need definitions**: Create new Sphinx-Needs directly from a single, :ref:`customized comment line <oneline>` in your source code.
9+
- source code analyzer for multiple programming languages and comment styles
10+
- generator for various output formats that contain the extracted markers or needs
11+
- Sphinx extension that integrates ``CodeLinks`` with Sphinx-Needs
12+
- CLI application to drive the analysis and generation process
13+
14+
The configuration for ``CodeLinks`` is done via a TOML file, which can be used
15+
for both the :ref:`Sphinx extension <configuration>` and the :ref:`CLI application <cli>`.
16+
17+
The configuration determines how markers and languages are ingested and how the Sphinx extension should behave.
18+
19+
At its core, ``CodeLinks`` parses the source code structure and extracts source markers.
20+
Source markers can be special comments or language-specific constructs like docstrings for Python.
21+
22+
``CodeLinks`` supports 3 distinct marker types:
23+
24+
- :ref:`One-line need definitions <oneline>`: Create new Sphinx-Needs directly from a single customized comment line
25+
in your source code.
926
- **Need ID references**: Link code to existing need items without creating new ones, perfect for tracing implementations to requirements.
1027
- **Marked RST text**: Extract blocks of reStructuredText embedded within comments, allowing you to include rich documentation with associated metadata right next to your code.
1128

12-
``src-trace`` directive then consumes ``One-line need definitions`` to generate traceability between source code and your documentation.
29+
When used in a Sphinx context, a new :ref:`directive` creates items at the location where it is placed (for a subset
30+
of the analyzed files/folders).
31+
32+
For use cases where ``CodeLinks`` should not integrate with Sphinx, but rather generate output files, the
33+
:ref:`cli` can be used. Currently it can write out ``needextend`` directives for the need ID reference comment style.
34+
Other output files are planned such as full need items as RST or needs.json.
1335

14-
The ``Analyse``, along with the ``SourceDiscovery`` module, also provides both a **Python API** for extensibility and a **CLI** for integration into CI/CD pipelines.
36+
The availability of most commands as :ref:`cli` also helps integrate ``CodeLinks`` into build systems and CI/CD pipelines.
37+
Focus is put on performance, portability and caching of processing steps.

‎docs/source/basics/quickstart.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Sphinx Config
2626
.. literalinclude:: ./../../src_trace.toml
2727
:caption: src_trace.toml
2828
:language: toml
29-
:lines: 27-29
29+
3030

3131
One-line comment
3232
----------------

‎docs/source/components/analyse.rst‎

Lines changed: 140 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,125 @@
1-
Analyse
2-
=======
1+
.. _analyse:
32

4-
The ``Analyse`` is a :ref:`CLI tool <cli>` that also provides an API for programmatic use. Its primary function is to extract specific, marked content from comments within source code files.
3+
Source Analyse
4+
==============
55

6-
It can extract three types of content:
6+
The **Source Analyse** module is a powerful component of **Sphinx-CodeLinks** that extracts documentation-related content from source code comments. It provides both CLI and API interfaces for flexible integration into documentation workflows.
77

8-
- Sphinx-Needs ID References
9-
- Oneline needs (see :ref:`OneLineCommentStyle <oneline>`)
10-
- Marked reStructuredText (RST) blocks
8+
**Key Capabilities:**
119

12-
Configuration
13-
-------------
10+
- Extract **Sphinx-Needs** ID references from source code comments
11+
- Process custom one-line comment patterns for rapid documentation
12+
- Extract marked reStructuredText (RST) blocks embedded in comments
13+
- Generate structured JSON output for further processing
14+
- Support for multiple programming language comment styles
1415

15-
The ``Analyse`` is configured using a ``toml`` file. The examples throughout this document are based on the following configuration:
16+
Overview
17+
--------
1618

17-
.. literalinclude:: ./../../../tests/data/analyse/default_config.toml
18-
:caption: default_config.toml
19-
:language: toml
19+
Source Analyse works by parsing source code files and identifying specially marked comments that contain documentation information. This enables developers to embed documentation directly in their source code while maintaining clean separation between code and documentation.
2020

21-
This configuration instructs the analyse to extract ``Sphinx-Needs ID Refs`` and ``Marked rst text`` using the defined markers.
21+
The module supports three primary extraction modes:
2222

23-
Sphinx-Needs ID Refs
24-
--------------------
23+
1. **Sphinx-Needs ID References** - Links between code and requirements/specifications
24+
2. **One-line Needs** - Simplified syntax for creating documentation needs
25+
3. **Marked RST Blocks** - Full reStructuredText content embedded in comments
26+
27+
Supported Content Types
28+
-----------------------
29+
30+
Sphinx-Needs ID References
31+
~~~~~~~~~~~~~~~~~~~~~~~~~~
32+
33+
Extract references to **Sphinx-Needs** items directly from source code comments, enabling traceability between code implementations and requirements.
34+
35+
One-line Needs
36+
~~~~~~~~~~~~~~
37+
38+
Use simplified comment patterns to define **Sphinx-Needs** items without complex RST syntax. See :ref:`OneLineCommentStyle <oneline>` for detailed information.
39+
40+
Marked RST Blocks
41+
~~~~~~~~~~~~~~~~~
42+
43+
Embed complete reStructuredText content within source code comments for rich documentation that can be extracted and processed.
44+
45+
Limitations
46+
-----------
47+
48+
**Current Limitations:**
49+
50+
- **Language Support**: Only C/C++ (``//``, ``/* */``) and Python (``#``) comment styles are supported
51+
- **Single Comment Style**: Each analysis run processes only one comment style at a time
52+
53+
Extraction Examples
54+
-------------------
55+
56+
The following examples are configured with :ref:`the analyse configuration <analyse_config>`,
57+
58+
Sphinx-Needs ID References
59+
~~~~~~~~~~~~~~~~~~~~~~~~~~
2560

2661
Below is an example of a C++ source file containing need ID references and the corresponding JSON output from the analyse.
2762

2863
.. tabs::
2964

3065
.. code-tab:: cpp
3166

32-
#include <iostream>
67+
#include <iostream>
3368

34-
// @need-ids: need_001, need_002, need_003, need_004
35-
void dummy_func1(){
36-
//...
37-
}
69+
// @need-ids: need_001, need_002, need_003, need_004
70+
void dummy_func1(){
71+
//...
72+
}
3873

39-
// @need-ids: need_003
40-
int main() {
41-
std::cout << "Starting demo_1..." << std::endl;
42-
dummy_func1();
43-
std::cout << "Demo_1 finished." << std::endl;
44-
return 0;
45-
}
74+
// @need-ids: need_003
75+
int main() {
76+
std::cout << "Starting demo_1..." << std::endl;
77+
dummy_func1();
78+
std::cout << "Demo_1 finished." << std::endl;
79+
return 0;
80+
}
4681

4782
.. code-tab:: json
4883

49-
[
50-
{
51-
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
52-
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L3",
53-
"source_map": {
54-
"start": { "row": 2, "column": 13 },
55-
"end": { "row": 2, "column": 51 }
56-
},
57-
"tagged_scope": "void dummy_func1(){\n //...\n }",
58-
"need_ids": ["need_001", "need_002", "need_003", "need_004"],
59-
"marker": "@need-ids:",
60-
"type": "need-id-refs"
61-
},
62-
{
63-
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
64-
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L8",
65-
"source_map": {
66-
"start": { "row": 7, "column": 13 },
67-
"end": { "row": 7, "column": 21 }
68-
},
69-
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
70-
"need_ids": ["need_003"],
71-
"marker": "@need-ids:",
72-
"type": "need-id-refs"
73-
}
74-
]
75-
76-
Marked RST
77-
----------
84+
[
85+
{
86+
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
87+
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L3",
88+
"source_map": {
89+
"start": { "row": 2, "column": 13 },
90+
"end": { "row": 2, "column": 51 }
91+
},
92+
"tagged_scope": "void dummy_func1(){\n //...\n }",
93+
"need_ids": ["need_001", "need_002", "need_003", "need_004"],
94+
"marker": "@need-ids:",
95+
"type": "need-id-refs"
96+
},
97+
{
98+
"filepath": "tests/data/need_id_refs/dummy_1.cpp",
99+
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/fa5a9129d60203355ae9fe4a725246a88522c60c/tests/data/need_id_refs/dummy_1.cpp#L8",
100+
"source_map": {
101+
"start": { "row": 7, "column": 13 },
102+
"end": { "row": 7, "column": 21 }
103+
},
104+
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
105+
"need_ids": ["need_003"],
106+
"marker": "@need-ids:",
107+
"type": "need-id-refs"
108+
}
109+
]
110+
111+
**Output Structure:**
112+
113+
- ``filepath`` - Path to the source file containing the reference
114+
- ``remote_url`` - URL to the source code in the remote repository
115+
- ``source_map`` - Location information (row/column) of the marker
116+
- ``tagged_scope`` - The code scope associated with the marker
117+
- ``need_ids`` - List of referenced need IDs
118+
- ``marker`` - The marker string used for identification
119+
- ``type`` - Type of extraction ("need-id-refs")
120+
121+
Marked RST Blocks
122+
~~~~~~~~~~~~~~~~~
78123

79124
This example demonstrates how the analyse extracts RST blocks from comments.
80125

@@ -102,16 +147,15 @@ This example demonstrates how the analyse extracts RST blocks from comments.
102147
return 0;
103148
}
104149

105-
106150
.. code-tab:: json
107151

108152
[
109153
{
110154
"filepath": "marked_rst/dummy_1.cpp",
111155
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/26b301138eef25c5130518d96eaa7a29a9c6c9fe/marked_rst/dummy_1.cpp#L4",
112156
"source_map": {
113-
"start": { "row": 3, "column": 8 },
114-
"end": { "row": 3, "column": 61 }
157+
"start": { "row": 3, "column": 8 },
158+
"end": { "row": 3, "column": 61 }
115159
},
116160
"tagged_scope": "void dummy_func1(){\n //...\n }",
117161
"rst": ".. impl:: implement dummy function 1\n :id: IMPL_71\n",
@@ -121,20 +165,48 @@ This example demonstrates how the analyse extracts RST blocks from comments.
121165
"filepath": "marked_rst/dummy_1.cpp",
122166
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/26b301138eef25c5130518d96eaa7a29a9c6c9fe/marked_rst/dummy_1.cpp#L14",
123167
"source_map": {
124-
"start": { "row": 13, "column": 7 },
125-
"end": { "row": 13, "column": 40 }
168+
"start": { "row": 13, "column": 7 },
169+
"end": { "row": 13, "column": 40 }
126170
},
127171
"tagged_scope": "int main() {\n std::cout << \"Starting demo_1...\" << std::endl;\n dummy_func1();\n std::cout << \"Demo_1 finished.\" << std::endl;\n return 0;\n }",
128172
"rst": "..impl:: implement main function ",
129173
"type": "rst"
130174
}
131175
]
132176

133-
Limitations
134-
-----------
177+
**Output Structure:**
178+
179+
- ``filepath`` - Path to the source file containing the RST block
180+
- ``remote_url`` - URL to the source code in the remote repository
181+
- ``source_map`` - Location information of the RST markers
182+
- ``tagged_scope`` - The code scope associated with the RST block
183+
- ``rst`` - The extracted reStructuredText content
184+
- ``type`` - Type of extraction ("rst")
185+
186+
**RST Block Formats:**
187+
188+
The module supports both multi-line and single-line RST blocks:
189+
190+
- **Multi-line blocks**: Use ``@rst`` and ``@endrst`` on separate lines
191+
- **Single-line blocks**: Use ``@rst content @endrst`` on the same line
192+
193+
One-line Needs
194+
--------------
195+
196+
**One-line Needs** provide a simplified syntax for creating **Sphinx-Needs** items directly in source code comments without requiring full RST syntax.
197+
198+
For comprehensive information about one-line needs configuration and usage, see :ref:`OneLineCommentStyle <oneline>`.
199+
200+
**Basic Example:**
201+
202+
.. code-block:: c
203+
204+
// @Function Implementation, IMPL_001, impl, [REQ_001, REQ_002]
205+
206+
This single comment line creates a complete **Sphinx-Needs** item equivalent to:
135207

136-
Please be aware of the following limitations:
208+
.. code-block:: rst
137209
138-
- **Supported Languages**: The analyse only supports comment styles for C/C++ (``//``, ``/*...*/``) and Python (``#``).
139-
- **Single Comment Style**: An analysis run can only process a single comment style at a time.
140-
- **Configuration Incompatibility**: The TOML configuration file cannot be shared with the ``CodeLink`` Sphinx extensions.
210+
.. impl:: Function Implementation
211+
:id: IMPL_001
212+
:links: REQ_001, REQ_002

‎docs/source/components/cli.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@
33
Command Line Interface (CLI)
44
============================
55

6-
``Sphinx-CodeLinks`` provides CLI for users to integrate documentation build into CI/CD pipeline
6+
``Sphinx-CodeLinks`` provides a CLI for users to integrate documentation builds into CI/CD pipelines
77
and for local development.
88

9-
It features help pages. add ``-h`` or ``--help`` to any command to see the available options.
9+
It features help pages. Add ``-h`` or ``--help`` to any command to see the available options.
1010

1111
.. typer:: sphinx_codelinks.cmd.app
1212
:prog: codelinks

0 commit comments

Comments
 (0)