Skip to content

Commit 002713e

Browse files
JohTJohT
committed
Distinguish between dev and non dev NPM package dependencies
1 parent ebf2bb4 commit 002713e

10 files changed

Lines changed: 102 additions & 21 deletions

‎COMMANDS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,11 +76,11 @@ The [analyze.sh](./scripts/analysis/analyze.sh) command comes with these command
7676

7777
- `--profile Neo4jv5` uses the older long term support (march 2025) version v5.26.x of Neo4j and suitable compatible versions of plugins and JQAssistant. Without specifying a profile, the newest versions will be used. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
7878

79-
- `--profile Neo4j-latest-continue-on-scan-errors` is based on the default profile (`Neo4j-latest`) but uses the jQAssistant configuration template [template-neo4j-remote-jqassistant-continue-on-error.yaml](./scripts/configuration/template-neo4j-remote-jqassistant-continue-on-error.yaml) to continue on scan error instead of failing fast. This is temporarily useful when there is a known error that needs to be ignored. It is still recommended to use the default profile and fail fast if there is something wrong. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
79+
- `--profile Neo4j-latest-continue-on-scan-errors` is based on the default profile (`Neo4j-latest`) but uses the jQAssistant configuration template [template-neo4jv5-jqassistant-continue-on-error.yaml](./scripts/configuration/template-neo4jv5-jqassistant-continue-on-error.yaml) to continue on scan error instead of failing fast. This is temporarily useful when there is a known error that needs to be ignored. It is still recommended to use the default profile and fail fast if there is something wrong. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
8080

8181
- `--profile Neo4j-latest-low-memory` is based on the default profile (`Neo4j-latest`) but uses only half of the memory (RAM) as configured in [template-neo4j-low-memory.conf](./scripts/configuration/template-neo4j-low-memory.conf). This is useful for the analysis of smaller codebases with less resources. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
8282

83-
- `--profile Neo4j-latest-low-memory-continue-on-scan-errors` is based on the default profile (`Neo4j-latest`) but uses only half of the memory (RAM) as configured in [template-neo4j-low-memory.conf](./scripts/configuration/template-neo4j-low-memory.conf) and the jQAssistant configuration template [template-neo4j-remote-jqassistant-continue-on-error.yaml](./scripts/configuration/template-neo4j-remote-jqassistant-continue-on-error.yaml) to continue on scan error instead of failing fast. This is temporarily useful when there is a known error that needs to be ignored. It is still recommended to use the default profile and fail fast if there is something wrong. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
83+
- `--profile Neo4j-latest-low-memory-continue-on-scan-errors` is based on the default profile (`Neo4j-latest`) but uses only half of the memory (RAM) as configured in [template-neo4j-low-memory.conf](./scripts/configuration/template-neo4j-low-memory.conf) and the jQAssistant configuration template [template-neo4jv5-jqassistant-continue-on-error.yaml](./scripts/configuration/template-neo4jv5-jqassistant-continue-on-error.yaml) to continue on scan error instead of failing fast. This is temporarily useful when there is a known error that needs to be ignored. It is still recommended to use the default profile and fail fast if there is something wrong. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
8484

8585
- `--profile Neo4j-latest-high-memory` is based on the default profile (`Neo4j-latest`) but uses more memory (RAM) as configured in [template-neo4j-high-memory.conf](./scripts/configuration/template-neo4j-high-memory.conf). This is useful for the analysis of larger codebases with more resources. Other profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
8686

‎cypher/Internal_Dependencies/NPM_Package_build_levels_for_graphviz.cypher‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,10 @@
1616
WITH *, round((normalizedWeight * 5) + 1, 2) AS penWidth
1717
WITH *, "\\n(level " + coalesce(source.maxDistanceFromSource + "/" + maxLevel, "?") + ")" AS sourceLevelInfo
1818
WITH *, "\\n(level " + coalesce(target.maxDistanceFromSource + "/" + maxLevel, "?") + ")" AS targetLevelInfo
19-
WITH *, source.name + sourceLevelInfo AS fullSourceName
20-
WITH *, target.name + targetLevelInfo AS fullTargetName
19+
WITH *, CASE WHEN source:NpmDevPackage THEN "\\n[dev]" ELSE "" END AS sourceDevDependencyInfo
20+
WITH *, CASE WHEN target:NpmDevPackage THEN "\\n[dev]" ELSE "" END AS targetDevDependencyInfo
21+
WITH *, source.name + sourceLevelInfo + sourceDevDependencyInfo AS fullSourceName
22+
WITH *, target.name + targetLevelInfo + targetDevDependencyInfo AS fullTargetName
2123
WITH *, "\" -> \"" + fullTargetName
2224
+ "\" [label = " + dependency.weightByDependencyType + ";"
2325
+ " penwidth = " + penWidth + ";"

cypher/Path_Finding/Set_Parameters_NPM.cypher renamed to cypher/Path_Finding/Set_Parameters_NonDevNpmPackage.cypher

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
:params {
44
"dependencies_projection_language":"NPM",
5-
"dependencies_projection": "npm-package-path-finding",
6-
"dependencies_projection_node": "Package",
5+
"dependencies_projection": "npm-non-dev-package-path-finding",
6+
"dependencies_projection_node": "NpmNonDevPackage",
77
"dependencies_projection_weight_property": "weightByDependencyType",
88
}

‎cypher/Typescript_Enrichment/Label_npm_packages_by_dep_type.cypher‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Label NPM packages as NpmDevPackage or NpmNonDevPackage based on incoming DEPENDS_ON. Requires Link_npm_packages_with_depends_on_relationships.cypher
2+
// For each package node, calculate the maximum `weightByDependencyType` across incoming
3+
// `DEPENDS_ON` relationships. If the max is 1 -> label `NpmDevPackage`; if > 1 -> label
4+
// `NpmNonDevPackage`. If the property is missing or < 1 then no label is added.
5+
// weightByDependencyType= 1 is a dev dependency, 2 is a regular dependency, and 3 is a peer dependency.
6+
7+
MATCH (packageNode:NPM:Package)
8+
OPTIONAL MATCH (packageNode)<-[dependsOnRel:DEPENDS_ON]-()
9+
WITH packageNode, max(dependsOnRel.weightByDependencyType) AS maxWeight
10+
11+
CALL {
12+
WITH packageNode, maxWeight
13+
FOREACH (_ IN CASE WHEN maxWeight = 1 THEN [1] ELSE [] END |
14+
SET packageNode:NpmDevPackage
15+
)
16+
17+
FOREACH (_ IN CASE WHEN maxWeight > 1 THEN [1] ELSE [] END |
18+
SET packageNode:NpmNonDevPackage
19+
)
20+
} IN TRANSACTIONS OF 1000 ROWS
21+
22+
RETURN count(packageNode) AS evaluatedPackages,
23+
sum(CASE WHEN maxWeight = 1 THEN 1 ELSE 0 END) AS numberOfNpmDevPackages,
24+
sum(CASE WHEN maxWeight > 1 THEN 1 ELSE 0 END) AS numberOfNpmNonDevPackages,
25+
sum(CASE WHEN maxWeight IS NULL THEN 1 ELSE 0 END) AS numberOfPackagesWithMissingWeight,
26+
sum(CASE WHEN maxWeight < 1 AND maxWeight IS NOT NULL THEN 1 ELSE 0 END) AS numberOfPackagesWithWeightBelowOne

‎cypher/Typescript_Enrichment/Remove_npm_dependency_type_labels.cypher‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
// Remove NPM package type labels (idempotent).
2+
3+
MATCH (packageNode:NPM:Package)
4+
REMOVE packageNode:NpmDevPackage
5+
REMOVE packageNode:NpmNonDevPackage

‎scripts/prepareAnalysis.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,8 @@ execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Remove_duplicate_CONTAINS_relations_bet
7979
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Link_npm_dependencies_to_npm_packages.cypher"
8080
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Link_npm_packages_with_depends_on_relationships.cypher"
8181
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Enrich_npm_packages_with_dependency_counts.cypher"
82+
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Remove_npm_dependency_type_labels.cypher"
83+
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Label_npm_packages_by_dep_type.cypher"
8284
execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Link_projects_to_npm_packages.cypher"
8385
dataVerificationResult=$( execute_cypher "${TYPESCRIPT_CYPHER_DIR}/Verify_projects_linked_to_npm_packages.cypher" "${@}")
8486
if is_csv_column_greater_zero "${dataVerificationResult}" "unresolvedProjectsCount"; then

‎scripts/reports/CentralityCsv.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -439,12 +439,12 @@ if createUndirectedDependencyProjection "${MODULE_LANGUAGE}" "${MODULE_PROJECTIO
439439
runUndirectedCentralityAlgorithms "${MODULE_PROJECTION_UNDIRECTED}" "${MODULE_NODE}"
440440
fi
441441

442-
# -- NPM Package Centrality ---------------------------------------
442+
# -- Non Dev NPM Package Centrality ---------------------------------------
443443

444444
NPM_LANGUAGE="dependencies_projection_language=NPM"
445-
NPM_PROJECTION="dependencies_projection=npm-package-centrality"
446-
NPM_PROJECTION_UNDIRECTED="dependencies_projection=npm-package-centrality-undirected"
447-
NPM_NODE="dependencies_projection_node=Package"
445+
NPM_PROJECTION="dependencies_projection=npm-non-dev-package-centrality"
446+
NPM_PROJECTION_UNDIRECTED="dependencies_projection=npm-non-dev-package-centrality-undirected"
447+
NPM_NODE="dependencies_projection_node=NpmNonDevPackage"
448448
NPM_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
449449

450450
if createDirectedDependencyProjection "${NPM_LANGUAGE}" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"; then

‎scripts/reports/PathFindingCsv.sh‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -140,16 +140,28 @@ if createDirectedDependencyProjection "${MODULE_LANGUAGE}" "${MODULE_PROJECTION}
140140
runPathFindingAlgorithms "${MODULE_PROJECTION}" "${MODULE_NODE}" "${MODULE_WEIGHT}"
141141
fi
142142

143-
# -- NPM Package Path Finding -------------------------------
143+
# -- Non Dev NPM Package Path Finding -------------------------------
144144

145145
NPM_LANGUAGE="dependencies_projection_language=NPM"
146-
NPM_PROJECTION="dependencies_projection=npm-package-path-finding"
147-
NPM_NODE="dependencies_projection_node=Package"
146+
NPM_PROJECTION="dependencies_projection=npm-non-dev-package-path-finding"
147+
NPM_NODE="dependencies_projection_node=NpmNonDevPackage"
148148
NPM_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
149149

150150
if createDirectedDependencyProjection "${NPM_LANGUAGE}" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"; then
151151
runPathFindingAlgorithms "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"
152152
fi
153+
154+
# -- Dev NPM Package Path Finding -------------------------------
155+
156+
NPM_DEV_LANGUAGE="dependencies_projection_language=NPM"
157+
NPM_DEV_PROJECTION="dependencies_projection=npm-dev-package-path-finding"
158+
NPM_DEV_NODE="dependencies_projection_node=NpmDevPackage"
159+
NPM_DEV_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
160+
161+
if createDirectedDependencyProjection "${NPM_DEV_LANGUAGE}" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}"; then
162+
runPathFindingAlgorithms "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}"
163+
fi
164+
153165
# ---------------------------------------------------------------
154166

155167
# Clean-up after report generation. Empty reports will be deleted.

‎scripts/reports/PathFindingVisualization.sh‎

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -90,27 +90,50 @@ if createDirectedDependencyProjection "${MODULE_LANGUAGE}" "${MODULE_PROJECTION}
9090
source "${VISUALIZATION_SCRIPTS_DIR}/visualizeQueryResults.sh" "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
9191
fi
9292

93-
# NPM Packages: Longest Paths Visualization
93+
# Non Dev NPM Packages: Longest Paths Visualization
9494
NPM_LANGUAGE="dependencies_projection_language=NPM"
95-
NPM_PROJECTION="dependencies_projection=npm-package-path-finding"
96-
NPM_NODE="dependencies_projection_node=Package"
95+
NPM_PROJECTION="dependencies_projection=npm-non-dev-package-path-finding"
96+
NPM_NODE="dependencies_projection_node=NpmNonDevPackage"
9797
NPM_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
9898

9999
if createDirectedDependencyProjection "${NPM_LANGUAGE}" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"; then
100100
# Determines topological sort max distance from source if not already done for level info in visualization.
101101
execute_cypher_queries_until_results "${TOPOLOGICAL_SORT_CYPHER_DIR}/Topological_Sort_Exists.cypher" \
102102
"${TOPOLOGICAL_SORT_CYPHER_DIR}/Topological_Sort_Write.cypher" "${NPM_PROJECTION}" "${NPM_NODE}"
103103

104-
reportName="NpmPackageLongestPathsIsolated"
104+
reportName="NpmNonDevPackageLongestPathsIsolated"
105105
echo "${SCRIPT_NAME}: Creating visualization ${reportName}..."
106106
execute_cypher "${PATH_FINDINGS_CYPHER_DIR}/Path_Finding_6_Longest_paths_for_graphviz.cypher" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}" > "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
107107
source "${VISUALIZATION_SCRIPTS_DIR}/visualizeQueryResults.sh" "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
108108

109-
reportName="NpmPackageLongestPaths"
109+
reportName="NpmNonDevPackageLongestPaths"
110110
echo "${SCRIPT_NAME}: Creating visualization ${reportName}..."
111111
execute_cypher "${PATH_FINDINGS_CYPHER_DIR}/Path_Finding_6_Longest_paths_contributors_for_graphviz.cypher" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}" > "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
112112
source "${VISUALIZATION_SCRIPTS_DIR}/visualizeQueryResults.sh" "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
113113
fi
114114

115+
# Dev NPM Packages: Longest Paths Visualization
116+
117+
NPM_DEV_LANGUAGE="dependencies_projection_language=NPM"
118+
NPM_DEV_PROJECTION="dependencies_projection=npm-dev-package-path-finding"
119+
NPM_DEV_NODE="dependencies_projection_node=NpmDevPackage"
120+
NPM_DEV_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
121+
122+
if createDirectedDependencyProjection "${NPM_DEV_LANGUAGE}" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}"; then
123+
# Determines topological sort max distance from source if not already done for level info in visualization.
124+
execute_cypher_queries_until_results "${TOPOLOGICAL_SORT_CYPHER_DIR}/Topological_Sort_Exists.cypher" \
125+
"${TOPOLOGICAL_SORT_CYPHER_DIR}/Topological_Sort_Write.cypher" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}"
126+
127+
reportName="NpmDevPackageLongestPathsIsolated"
128+
echo "${SCRIPT_NAME}: Creating visualization ${reportName}..."
129+
execute_cypher "${PATH_FINDINGS_CYPHER_DIR}/Path_Finding_6_Longest_paths_for_graphviz.cypher" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}" > "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
130+
source "${VISUALIZATION_SCRIPTS_DIR}/visualizeQueryResults.sh" "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
131+
132+
reportName="NpmDevPackageLongestPaths"
133+
echo "${SCRIPT_NAME}: Creating visualization ${reportName}..."
134+
execute_cypher "${PATH_FINDINGS_CYPHER_DIR}/Path_Finding_6_Longest_paths_contributors_for_graphviz.cypher" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}" > "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
135+
source "${VISUALIZATION_SCRIPTS_DIR}/visualizeQueryResults.sh" "${FULL_REPORT_DIRECTORY}/${reportName}.csv"
136+
fi
137+
115138
# Clean-up after report generation. Empty reports will be deleted.
116139
source "${SCRIPTS_DIR}/cleanupAfterReportGeneration.sh" "${FULL_REPORT_DIRECTORY}"

‎scripts/reports/TopologicalSortCsv.sh‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -103,17 +103,28 @@ if createDirectedDependencyProjection "${MODULE_LANGUAGE}" "${MODULE_PROJECTION}
103103
time topologicalSort "${MODULE_PROJECTION}" "${MODULE_NODE}" "${MODULE_WEIGHT}"
104104
fi
105105

106-
# -- NPM Package Topology ---------------------------------------
106+
# -- Non Dev NPM Package Topology ---------------------------------------
107107

108108
NPM_LANGUAGE="dependencies_projection_language=NPM"
109-
NPM_PROJECTION="dependencies_projection=npm-package-topology"
110-
NPM_NODE="dependencies_projection_node=Package"
109+
NPM_PROJECTION="dependencies_projection=npm-non-dev-package-topology"
110+
NPM_NODE="dependencies_projection_node=NpmNonDevPackage"
111111
NPM_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
112112

113113
if createDirectedDependencyProjection "${NPM_LANGUAGE}" "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"; then
114114
time topologicalSort "${NPM_PROJECTION}" "${NPM_NODE}" "${NPM_WEIGHT}"
115115
fi
116116

117+
# -- Dev NPM Package Topology --------------------------------------------
118+
119+
NPM_DEV_PROJECTION="dependencies_projection=npm-dev-package-topology"
120+
NPM_DEV_NODE="dependencies_projection_node=NpmDevPackage"
121+
NPM_DEV_WEIGHT="dependencies_projection_weight_property=weightByDependencyType"
122+
123+
if createDirectedDependencyProjection "${NPM_LANGUAGE}" "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}"; then
124+
time topologicalSort "${NPM_DEV_PROJECTION}" "${NPM_DEV_NODE}" "${NPM_DEV_WEIGHT}"
125+
fi
126+
# ----------------------------------------------------------------------
127+
117128
# Clean-up after report generation. Empty reports will be deleted.
118129
source "${SCRIPTS_DIR}/cleanupAfterReportGeneration.sh" "${FULL_REPORT_DIRECTORY}"
119130

0 commit comments

Comments
 (0)