Move dem_rgba_tiles to Data/ and add bounding box support to ViewshedTool#948
Merged
tariqksoliman merged 235 commits intoNASA-AMMOS:developmentfrom Apr 23, 2026
Merged
Move dem_rgba_tiles to Data/ and add bounding box support to ViewshedTool#948tariqksoliman merged 235 commits intoNASA-AMMOS:developmentfrom
tariqksoliman merged 235 commits intoNASA-AMMOS:developmentfrom
Conversation
…tion - Configure detect-secrets with standard plugin set (AWS, Azure, GitHub, JWT, etc.) - Exclude src/external/, node_modules/, build/, .git/ from scanning - Matches pattern used by atlas repo's .secrets.baseline - Enables CI secret detection workflows to skip known false positives Co-Authored-By: tariq.k.soliman <tariq.k.soliman@jpl.nasa.gov>
- Mirrors atlas repo's secrets-detection.yaml workflow - Uses NASA-AMMOS/slim-detect-secrets for scanning - Triggers on push/PR to development branch - Excludes src/external/, node_modules/, build/, .git/ paths - Compares scan results against .secrets.baseline to detect new secrets - Fails CI if new secrets are detected, with remediation instructions Co-Authored-By: tariq.k.soliman <tariq.k.soliman@jpl.nasa.gov>
Run detect-secrets scan against the repo with the same exclude patterns as the CI workflow. All 7 findings are placeholder/sample values marked as is_secret=false: - sample.env: SECRET=aSecretKey, DB_PASS=password (sample config) - API/logger.js: placeholder secret keyword - sds/unity/terraform/terraform.tfvars: sample credentials - tests/unit/*.spec.js: test fixture secrets This ensures the CI workflow's diff check passes — only genuinely new secrets introduced in future commits will fail the build. Co-Authored-By: tariq.k.soliman <tariq.k.soliman@jpl.nasa.gov>
Co-Authored-By: tariq.k.soliman <tariq.k.soliman@jpl.nasa.gov>
…aseline chore(security): Add .secrets.baseline and CI secrets detection workflow
…ent injection - Fix geometry.type filter: use Sequelize replacement parameters instead of string interpolation - Fix property key interpolation: use replacement parameter instead of escaped single quotes - Fix timeProp temporal filter: use replacement parameter instead of string concatenation - Add SQL injection security tests for filter keys, values, geometry.type, timeProp, sortBy - Add functional regression tests for all filter types (equality, numeric, IN, LIKE, IS NULL, geometry.type, grouped, pagination, spatial, sortBy) - Add CRUD + filter integration test to verify parameterized queries return correct results Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…icts Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ion-filesutils fix(security): Parameterize SQL queries in Draw/Files filters to prevent injection
…e status The parameterized query fix safely handles special characters in field names (like semicolons), so the server correctly returns 'success' with no matching rows rather than 'failure'. Updated the test assertion to accept both outcomes since either is a valid non-500 response. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…eld-test fix(test): update invalid field name test to accept success or failure status
…ements in filesutils.js
- Rename geometry.type placeholders to geom_type_${idx} for consistency
- Rename propKey placeholder variable to propKeyPlaceholder for clarity
- Remove manual quote escaping (.replaceAll("'", "''")) since Sequelize
replacements handle escaping automatically (prevents double-escaping)
- Add E2E filter injection tests to draw.spec.js (6 new test cases)
- Add draw getfile filter tests to sql-injection.spec.js (12 new test cases)
- Add filter field name regex validation unit tests (3 new test cases)
Resolves SonarQube S3649 SQL injection vulnerability.
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ion-filesutils Devin/1775781632 fix sql injection filesutils
…injection tests - Fix 2a: Add dynamic column name validation for startProp/endProp in /aggregations and /intersect endpoints using describeTable() - Fix 2b: Remove unused startProp/endProp replacement keys from query objects - Fix 3a: Add filesutils-sql-injection.spec.js with tests for filter field names, geometry.type injection, timeProp sanitization, and filter values - Fix 3b: Add SQL injection tests to geodatasets.spec.js for /aggregations, /intersect, and /get endpoints with malicious startProp/endProp - Fix 3c: Extend sql-injection-prevention.spec.js with edge cases for forceAlphaNumUnder (SQL keywords, OR injection, backticks, brackets) Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ddir Replace user-tainted urlSplit with already-validated resolvedPath when constructing the fs.readdir argument in queryTilesetTimes. This breaks the taint chain that SonarQube tracks from req.query.path to fs.readdir while maintaining identical behavior (resolvedPath is derived from the same path.join(rootDir, decodedUrl) that urlSplit was). Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
… rootDir
Address Devin Review feedback: if the MMGIS installation directory
happened to contain '_time_' in its path, resolvedPath.split('_time_')
would split at the wrong occurrence. Use indexOf with allowedBase.length
offset to find the _time_ marker only within the URL portion of the
resolved path.
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ing server) Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…assertions - Apply same column allowlist validation to /get endpoint as /intersect and /aggregations - Make /get .then() callback async for await support - Remove unused startProp/endProp from /get replacements object - Remove response.json() calls from filesutils tests (server returns HTML not JSON) Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Previous tiles used average (LANCZOS) resampling which corrupted IEEE 754 float bytes at edges, producing huge values (6.54e+27). Now using near-composite resampling to preserve byte-level accuracy. Extended to zoom level 13 (was 10-12). Updated maxNativeZoom in config. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…n min/max update - Post-processed RGBA tiles to replace 23 corrupted edge pixels (from resampling) with transparent nodata. All zoom 10-13 tiles now decode to reasonable elevations (-0.23 to 267m). - Tightened populateCogScale guard: only data layers WITH shader ramps pass through (prevents TypeError crash for data layers without ramps, e.g. non-colorize shader types). - Added populateCogScale call in DataShaders.updateLegendMinMax so the LayersTool COG scale legend refreshes whenever min/max values update. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Switched from the newer gdal2customtiles.py to the legacy version which generates each zoom level as base tiles from the source DEM, avoiding the overview averaging that corrupts IEEE 754 float bytes. Legacy script also generates 2 extra zoom levels (14-15) for DEM mode. All 132 tiles verified clean: zero unreasonable pixel values at any zoom. Updated maxNativeZoom from 13 to 15. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
getTool returns a fallback object (not null) when the tool is absent, so the null check alone is insufficient. Added typeof check to prevent TypeError if LayersTool is excluded from a custom build. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…xtra ramps, fix docs - Remove zoom level 15 tiles (maxNativeZoom now 14) - Add 0 to noDataValues array so true-zero elevations are treated as nodata - Set precise bounding box from tilemapresource.xml - Add Viridis and Inferno color ramps alongside existing RdYlBu - Fix noDataValues description in Data.md docs to be accurate Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Previously bounds was only passed to the inner L.tileLayer instances (via options.options), but the outer L.GridLayer (L.TileLayer.GL) never received it. This meant L.GridLayer._isValidTile() could not filter out-of-bounds tiles, causing tile requests for the entire viewport. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…a-tiles Support data layers with plain URL rgba tiles in populateCogScale
…-encoded float tiles When --dem flag is used, float32 elevation values are encoded into RGBA bytes via IEEE 754 binary representation. The scale_query_to_tile function was using gdal.RegenerateOverview with 'average' resampling, which averages the raw RGBA byte values. Since these bytes represent IEEE 754 float encoding, averaging them produces garbage float values (e.g. 6.54e+27 instead of ~150m elevation). This is especially visible at tile edges where valid RGBA-encoded pixels neighbor transparent (0,0,0,0) nodata pixels. Fix: when options.isDEMtile is True, use 'near' (nearest-neighbor) resampling instead of 'average' in scale_query_to_tile. This preserves the RGBA byte encoding integrity during the querysize-to-tile_size downscale step. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…es.py Tiles regenerated at zoom levels 10-14 using the corrected script that uses nearest-neighbor resampling for DEM tiles instead of average. All 23,660 data pixels verified to decode to valid elevations within the source DEM range. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…esampling Fix DEM tile corruption by using nearest-neighbor resampling for RGBA-encoded float tiles
Part 1: - Move blueprints/.../Layers/Data/dem_rgba_tiles to blueprints/.../Data/dem_rgba_tiles - Update demtileurl path in config from Layers/Data/ to Data/ - Update Viewshed tool data source: correct URL, minZoom=10, maxNativeZoom=14, name Part 2: - Add optional boundingBox field to viewshed data sources - Clamp tile coordinates in updateDesiredTiles() to bbox extent - Enforce maxNumOfDataTiles (100) limit, prioritizing tiles near source - Initialize data matrix with internalNoDataValue instead of 0 - Document boundingBox in Viewshed.md Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- Add boundingBox textarray field to Viewshed config.json for Configure page - Replace G_.litho.projection.tileXYZ2LatLng with standalone web mercator helper - Remove unused G_ import - Add early return in queryDesiredTiles when totalTiles === 0 - Coerce bbox values to numbers for robustness with textarray input Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ithm.js Same fix as Manager — use standalone web mercator tileXYZ2LatLng helper instead of depending on Globe_ renderer's projection object. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ator + polar) - Create Projection_.js: replicates LithoSphere's tileXYZ2LatLng and latLngZ2TileXYZ with both Web Mercator and custom CRS/polar code paths - GlobeRenderer._initCesium(): use Projection_.buildFromConfig() instead of empty object, so projection math works with Cesium renderer - ViewshedTool: restore G_.litho.projection.tileXYZ2LatLng calls, remove standalone Web Mercator-only helper - ShadeTool: no changes needed, already uses G_.litho.projection Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
When clicking outside the bbox (north/south), the source point projects outside the clamped tile grid, causing d.data[o.y] to be undefined in initializeGrids. Clamp dataSource x/y to [1, length-2] so the algorithm's 8-neighbor access pattern stays within bounds. Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…tiles-move Move dem_rgba_tiles to Data/ and add bounding box support to ViewshedTool
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With Devin
JPL-Devin#40