Skip to content

Commit 4c6c46b

Browse files
remove editor (#1746)
* remove editor * move use case to dataset * lint --------- Co-authored-by: RamilCDISC <113539111+RamilCDISC@users.noreply.github.com>
1 parent 70e4fa5 commit 4c6c46b

8 files changed

Lines changed: 13 additions & 153 deletions

File tree

docs/PYPI.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -339,7 +339,7 @@ reporting_services = reporting_factory.get_report_services()
339339

340340
**`column_prefix_map`** — maps the `--` variable prefix to the dataset domain (e.g. `{"--": "AE"}`), resolving placeholders like `--SEQ``AESEQ`.
341341

342-
**External dictionaries** — pass an `ExternalDictionariesContainer` to `RulesEngine` if validating rules that require MedDRA, WHODrug, LOINC, UNII, MedRT, or SNOMED. See the [External Dictionary Reference](https://cdisc-org.github.io/conformance-rules-editor/#/exdictionary).
342+
**External dictionaries** — pass an `ExternalDictionariesContainer` to `RulesEngine` if validating rules that require MedDRA, WHODrug, LOINC, UNII, MedRT, or SNOMED. See the [External Dictionary Reference](https://cdisc-org.github.io/cdisc-open-rules/#/exdictionary).
343343

344344
**Dask** — set `max_dataset_size=0` when initializing `DataServiceFactory` to force Dask processing for all datasets.
345345

docs/cli-reference.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,6 @@ python core.py validate --help
2222
| `-s, --standard TEXT` | CDISC standard to validate against (e.g. `sdtmig`, `tig`). Also via `PRODUCT` env var. |
2323
| `-v, --version TEXT` | Standard version (e.g. `3-4`). Also via `VERSION` env var. |
2424
| `-ss, --substandard TEXT` | **Required for TIG.** One of `SDTM`, `SEND`, `ADaM`, `CDASH`. Also via `SUBSTANDARD` env var. |
25-
| `-uc, --use-case TEXT` | Use Case for TIG Custom Domains. When performing a TIG validation with custom domain(s), this |
26-
| | must be given to identify the custom domains' use case One of `INDH`, `PROD`, `NONCLIN`, |
27-
| | `ANALYSIS`. Also via `USE_CASE` env var. |
2825

2926
### Dataset Input
3027

@@ -33,6 +30,9 @@ python core.py validate --help
3330
| `-d, --data TEXT` | Path to directory containing dataset files. Only the last value is used if specified multiple times. |
3431
| `-dp, --dataset-path TEXT` | Absolute path to a single dataset file. Can be specified multiple times. |
3532
| `-dxp, --define-xml-path TEXT` | Path to Define-XML. Also via `DEFINE_XML` env var. |
33+
| `-uc, --use-case TEXT` | Use Case for TIG Custom Domains. When performing a TIG validation with custom domain(s), this |
34+
| | must be given to identify the custom domains' use case One of `INDH`, `PROD`, `NONCLIN`, |
35+
| | `ANALYSIS`. Also via `USE_CASE` env var. |
3636
| `-ft, --filetype TEXT` | File extension filter applied to the `-d` directory (e.g. `xpt`). Has higher priority than --dataset-path parameter |
3737
| `-e, --encoding TEXT` | File encoding for reading datasets (default: `utf-8`). Common values: `cp1252`, `latin-1`, `utf-16`. |
3838
| `-vcp, --variables-csv-path` | Path to `_variables.csv` when using multiple `-dp` paths across different folders. |

docs/contributing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Conformance rules are maintained separately in [`cdisc-open-rules`](https://gith
1212
- Report an issue with an existing rule's logic
1313
- Contribute a rule implementation
1414

15-
Please open an issue or pull request in that repository. Rule authoring can also be done through the hosted [CORE Rule Editor](https://cdisc-org.github.io/conformance-rules-editor).
15+
Please open an issue or pull request in that repository.
1616

1717
For questions about rule contribution workflows, post in [GitHub Discussions](https://github.com/cdisc-org/cdisc-rules-engine/discussions).
1818

docs/development.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ These are derived from the OpenAPI specs in [`cdisc-org/DDF-RA`](https://github.
173173

174174
## Dataset Format Reference (JSON)
175175

176-
When validating a single rule with `--local-rules`, JSON datasets must match the Dataset-JSON format used by the rule editor:
176+
When validating a single rule with `--local-rules`, JSON datasets must match the json format used by the deprecated rule editor:
177177

178178
```json
179179
{

resources/schema/rule-merged/Operations.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@
117117
"properties": {
118118
"operator": {
119119
"const": "get_codelist_attributes",
120-
"markdownDescription": "\nFetches controlled terminology attribute values from CT packages based on row-specific CT package and version references. The operation constructs CT package names based on the standard being validated and the values in the `name` and `version` columns (e.g., SDTMIG \u2192 \"sdtmct-{version}\"). When the `name` column contains \"CDISC\" or \"CDISC CT\", it uses the validation run's standard to determine the package prefix and the version found in the cell of the specified column. The operation extracts all codes matching the specified ct_attribute from the package.\n\n**Required Parameters:**\n\n- `ct_attribute`: Attribute to extract - `\"Term CCODE\"`, `\"Codelist CCODE\"`, `\"Term Value\"`, `\"Codelist Value\"`, or `\"Term Preferred Term\"`\n- `name`: Column containing CT reference (e.g., \"TSVCDREF\") - identifies which terminology system is referenced\n- `version`: Column containing CT version (e.g., \"TSVCDVER\")\n\n```yaml\n- id: $VALID_TERM_CODES\n name: TSVCDREF\n operator: get_codelist_attributes\n ct_attribute: Term CCODE\n version: TSVCDVER\n```\n\n**Note:** due to editor not containing the cache, if using this operator in rule editor, you must put the ctpackage versions contained within your data in the library tab for it work properly in editor.\n"
120+
"markdownDescription": "\nFetches controlled terminology attribute values from CT packages based on row-specific CT package and version references. The operation constructs CT package names based on the standard being validated and the values in the `name` and `version` columns (e.g., SDTMIG \u2192 \"sdtmct-{version}\"). When the `name` column contains \"CDISC\" or \"CDISC CT\", it uses the validation run's standard to determine the package prefix and the version found in the cell of the specified column. The operation extracts all codes matching the specified ct_attribute from the package.\n\n**Required Parameters:**\n\n- `ct_attribute`: Attribute to extract - `\"Term CCODE\"`, `\"Codelist CCODE\"`, `\"Term Value\"`, `\"Codelist Value\"`, or `\"Term Preferred Term\"`\n- `name`: Column containing CT reference (e.g., \"TSVCDREF\") - identifies which terminology system is referenced\n- `version`: Column containing CT version (e.g., \"TSVCDVER\")\n\n```yaml\n- id: $VALID_TERM_CODES\n name: TSVCDREF\n operator: get_codelist_attributes\n ct_attribute: Term CCODE\n version: TSVCDVER\n```\n\n**Note:** if using this operator with excel data, you must put the ctpackage versions contained within your data in the library tab for it work properly.\n"
121121
}
122122
},
123123
"required": ["id", "operator", "name", "ct_attribute", "version"],

resources/schema/rule/ExDictionary.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# External Dictionaries
22

3-
External dictionaries provide standardized terminology and coding systems for medical terms, drugs, and laboratory tests. This guide covers how to integrate and validate external dictionaries in both the Command Line Interface (CLI) and Rule Editor.
3+
External dictionaries provide standardized terminology and coding systems for medical terms, drugs, and laboratory tests. This guide covers how to integrate and validate external dictionaries in both the Command Line Interface (CLI).
44

55
## Supported Dictionaries
66

@@ -123,7 +123,7 @@ Directory must contain the `Loinc.csv` with capital 'L'
123123
124124
```
125125

126-
## Operations & Rule Editor
126+
## Operations
127127

128128
### Dictionary Version Validation
129129

resources/schema/rule/Operations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@ Fetches controlled terminology attribute values from CT packages based on row-sp
168168
version: TSVCDVER
169169
```
170170

171-
**Note:** due to editor not containing the cache, if using this operator in rule editor, you must put the ctpackage versions contained within your data in the library tab for it work properly in editor.
171+
**Note:** if using this operator with excel data, you must put the ctpackage versions contained within your data in the library tab for it work properly.
172172

173173
### valid_codelist_dates
174174

resources/schema/rule/customrules.md

Lines changed: 3 additions & 143 deletions
Original file line numberDiff line numberDiff line change
@@ -1,119 +1,21 @@
1-
# Custom Editor Columns and Custom Rules
1+
# CDISC Custom Rule Extensions
22

3-
This guide explains how to extend CDISC rule definitions with custom attributes and how to access these attributes using the custom columns editor syntax.
3+
This guide explains how to extend CDISC rule definitions with your own custom attributes while maintaining compatibility with the core CDISC schema structure.
44

55
## Table of Contents
66

7-
- [Custom Columns in Editor Syntax](#custom-columns-in-editor-syntax)
8-
- [Overview and Basic Syntax](#overview-and-basic-syntax)
9-
- [Identifying Arrays in YAML](#identifying-arrays-in-yaml)
10-
- [Identifying Arrays in JSON](#identifying-arrays-in-json)
117
- [Custom Attributes Overview](#overview)
128
- [What Can and Cannot Be Changed](#what-can-and-cannot-be-changed)
139
- [Custom Schema Attributes](#custom-attributes)
14-
- [Custom Columns in Editor Syntax](#custom-columns-in-editor-syntax)
1510
- [Adding a Custom Organization](#adding-a-custom-organization)
1611
- [Example Rule with Custom Attributes](#example-rule-with-custom-attributes)
17-
- [Filtering Rules with Custom Columns](#filtering-rules-with-custom-columns)
1812
- [Validation](#validation)
1913
- [Best Practices](#best-practices)
2014
- [FAQ](#faq)
2115

22-
## Custom Columns in Editor Syntax
23-
24-
Custom columns allow you to query nested rule structures, including arrays, using a simple path syntax. This allows for better access and filtering for both standard and custom attributes.
25-
26-
## Overview and Basic Syntax
27-
28-
Custom columns allow you to query nested rule structures, including arrays, using a simple path syntax. All custom column paths use `.` to separate each nested object property of the rule and use `@` to denote array elements.
29-
30-
The general rules for constructing paths:
31-
32-
1. The path is **case-sensitive**
33-
2. The `@` symbol must come directly before the array name
34-
3. Array notation must be used to access array elements
35-
4. The path must reflect the exact structure of your data
36-
37-
### Identifying Arrays in YAML
38-
39-
When viewing a YAML document, you can identify arrays by looking for these indicators:
40-
41-
1. **Hyphen (-) at the Start**: Arrays elements are marked with a leading hyphen
42-
43-
```yaml
44-
Authorities: #This is an array
45-
- Organization: "Org1" # This is an element within the array
46-
Standards: #This is an array (within the Authorities Array)
47-
- Name: "Standard1" # This is also an element within the array
48-
- Name: "Standard2" # Another element within the array
49-
```
50-
51-
This means the Authorities array contains elements with Organization and Standards Properties on them. Similarly, Standards is an array with each element in the array having a Name property.
52-
53-
2. **No Hyphen**: Regular object properties don't have a hyphen
54-
55-
```yaml
56-
Core:
57-
Id: "123"
58-
Status: "Active"
59-
```
60-
61-
### Example YAML with Path Mapping
62-
63-
```yaml
64-
Authorities:
65-
- Organization: "Org1" # @Authorities.Organization
66-
Standards:
67-
- Name: "Standard1" # @Authorities.@Standards.Name
68-
Core:
69-
Id: "123" # Core.Id
70-
Status: "Active" # Core.Status
71-
```
72-
73-
The presence of hyphens (-) is your key indicator that you need to use @ notation in your custom path
74-
75-
### Identifying Arrays in JSON
76-
77-
You can also inspect the rule json looking for `[` square brackets to denote arrays.
78-
79-
### Single Array Example
80-
81-
For data structured like:
82-
83-
```json
84-
{
85-
"Check": {
86-
"all": [{ "operator": "equal_to" }, { "operator": "not_equal_to" }]
87-
}
88-
}
89-
```
90-
91-
Use: `Check.@all.operator` All is an array, containing objects with operator keys
92-
93-
### Nested Arrays Example
94-
95-
For data structured like:
96-
97-
```json
98-
{
99-
"Authorities": [
100-
{
101-
"Organization": "Org1",
102-
"Standards": [{ "Name": "Standard1" }, { "Name": "Standard2" }]
103-
}
104-
]
105-
}
106-
```
107-
108-
Use: `@Authorities.@Standards.Name`
109-
110-
# CDISC Custom Rule Extensions
111-
112-
This guide explains how to extend CDISC rule definitions with your own custom attributes while maintaining compatibility with the core CDISC schema structure, and how to access these attributes using the custom columns editor syntax.
113-
11416
## Overview
11517

116-
The CDISC Rules Engine schema supports custom extensions to help organizations better categorize, manage, and filter rules. These extensions maintain compatibility with standard CDISC rule definitions while adding organizational metadata tailored to your specific needs. The custom columns editor syntax allows you to query and filter these properties using a simple path notation.
18+
The CDISC Rules Engine schema supports custom extensions to help organizations better categorize, manage, and filter rules. These extensions maintain compatibility with standard CDISC rule definitions while adding organizational metadata tailored to your specific needs.
11719

11820
## What Can and Cannot Be Changed
11921

@@ -300,14 +202,6 @@ If you want to formally define custom properties in your schema (recommended for
300202
- Provide validation rules or constraints for each property
301203
- Share the documentation with all teams that will be using the rules
302204

303-
### Accessing Custom Properties
304-
305-
To access your custom properties in the editor using custom columns:
306-
307-
- Simple property: `Category.MyCustomProperty`
308-
- Nested property: `Category.ReviewInfo.LastReviewer`
309-
- Array property: `Category.@ValidationHistory.Result`
310-
311205
## Example Rule with Custom Attributes
312206

313207
Here's a complete example of a rule with custom attributes in the Category section:
@@ -374,39 +268,6 @@ Scope:
374268
Sensitivity: Record
375269
```
376270
377-
## Filtering Rules with Custom Columns
378-
379-
The custom columns editor syntax allows you to create powerful filters based on your custom attributes. Here are some common filtering scenarios:
380-
381-
### Setting Up Custom Columns
382-
383-
1. Add a custom column using the column selector
384-
2. Enter the path to the attribute you want to display/filter
385-
3. Set the column name
386-
387-
### Filter Examples
388-
389-
| To Filter By | Custom Column Path | Filter Value Example |
390-
| ---------------------- | -------------------------------------------------------- | --------------------- |
391-
| Sponsor | `@Authorities.Category.@Sponsors` | "Sponsor A" |
392-
| Specific vendor | `@Authorities.Category.@Vendors` | "CRO B" |
393-
| Therapeutic area | `@Authorities.Category.@TherapeuticAreas` | "Oncology" |
394-
| Trial | `@Authorities.Category.@Trials` | "ONC-2025-01" |
395-
| Purpose | `@Authorities.Category.Purpose` | "RAW data validation" |
396-
| Keyword | `@Authorities.Category.@Keywords` | "mykeyword" |
397-
| Custom property | `@Authorities.Category.Department` | "Clinical Data Mgmt" |
398-
| Authority organization | `@Authorities.Organization` | "MyCompany" |
399-
| Standard name | `@Authorities.@Standards.Name` | "Internal Standard" |
400-
| Rule ID | `@Authorities.@Standards.@References.Rule_Identifier.Id` | "INT001" |
401-
402-
### Complex Filtering
403-
404-
Combine multiple custom columns to create complex filters:
405-
406-
1. Add column for `@Authorities.Category.@TherapeuticAreas` and filter for "Oncology"
407-
2. Add column for `@Authorities.Category.Purpose` and filter for "RAW data validation"
408-
3. Add column for `@Authorities.Organization` and filter for "MyCompany"
409-
410271
## Best Practices
411272
412273
When adding custom attributes to rules:
@@ -416,7 +277,6 @@ When adding custom attributes to rules:
416277
3. **Use controlled vocabularies** - For fields like TherapeuticAreas, maintain a list of standard terms
417278
4. **Consider hierarchies** - Categorize using hierarchical structures when appropriate
418279
5. **Avoid redundancy** - Don't duplicate information already in core CDISC properties
419-
6. **Path naming** - When adding custom columns, use clear names that indicate the data being displayed
420280
421281
## Validation
422282

0 commit comments

Comments
 (0)