#727 STAC item regex search and bulk delete by tariqksoliman · Pull Request #728 · NASA-AMMOS/MMGIS · GitHub
Skip to content

#727 STAC item regex search and bulk delete#728

Merged
tariqksoliman merged 2 commits into
developmentfrom
ts-727
Aug 13, 2025
Merged

#727 STAC item regex search and bulk delete#728
tariqksoliman merged 2 commits into
developmentfrom
ts-727

Conversation

@tariqksoliman

@tariqksoliman tariqksoliman commented Aug 13, 2025

Copy link
Copy Markdown
Member

Closes #727

With Claude 4 Sonnet and GPT-5

Enhanced STAC Collection Items Modal with Spatial Filtering and Bulk Operations

Summary

This PR significantly enhances the STAC Collection Items Modal with comprehensive filtering capabilities, bulk operations, spatial bounding box search, and improved UX. The modal has been refactored into modular components for better maintainability.

🚀 Key Features Added

Bulk Operations

  • Multi-select functionality with individual checkboxes and "Select All" option
  • Bulk delete with confirmation dialog showing count of selected items
  • Progress tracking during bulk operations with error handling

Advanced Filtering

  • Regex search with toggle switch and helpful tooltip guidance
  • Date range filtering with From/To datetime pickers (225px fixed width)
  • Spatial bounding box filtering with interactive map drawing
  • Clear Filters button to reset all filter states

Spatial Filter Map

  • Interactive Leaflet map on left side of modal (similar to JSON View Modal)
  • Bounding box drawing with toggle button ("Draw BBox")
  • Visual drawing mode with crosshair cursor and instruction overlay
  • TiTiler-PgSTAC mosaic overlay for visualizing collection items
  • Auto-disable drawing mode after bbox completion
  • 32-bit raster support with automatic rescaling and colormap application

Enhanced UX

  • Responsive design with Asset Href column hidden on screens < 1600px
  • Shortened item count message on mobile screens
  • Regex tooltip with syntax examples and styled appearance
  • Drawing instructions in lower-right corner during bbox mode
  • Active button states with blue highlighting for drawing mode
  • Disabled text selection during map interactions

🔧 Technical Improvements

Component Refactoring

The large monolithic component was broken into focused, reusable components:

StacCollectionItemsModal/
├── StacCollectionItemsModal.js (main orchestrator)
└── components/
    ├── SpatialFilterMap.js (map panel with drawing)
    ├── JsonViewModal.js (JSON view with item map)
    ├── ItemsTable.js (data table with pagination)
    ├── DeleteItemModal.js (single item deletion)
    └── BulkDeleteModal.js (multi-item deletion)

Data Flow Optimization

  • Efficient data sharing between components via props
  • Reused item metadata for 32-bit detection (no additional API calls)
  • Client-side filtering for regex, date range, and spatial intersection
  • Server-side optimization with conditional LIKE queries

32-bit Raster Support

  • Automatic detection using first item's raster:bands metadata
  • Statistics-based rescaling with fallback defaults
  • Colormap application (viridis) for better visualization
  • Consistent behavior between individual items (JsonViewModal) and collection mosaics (SpatialFilterMap)

🎨 UI/UX Enhancements

Layout Improvements

  • Reorganized header with filters moved right of title to reduce whitespace
  • Fixed-width date pickers for consistent appearance
  • Responsive column visibility based on screen size
  • Improved button styling with hover states and active indicators

Map Interactions

  • Clear visual feedback during drawing mode:
    • Blue active button state
    • Crosshair cursor over map area
    • Instructional overlay in corner
    • Disabled pan/zoom during drawing
  • Automatic cleanup of old bboxes when drawing new ones
  • Intuitive workflow with auto-disable after completion

Filter Controls

  • Regex assistance with multiline tooltip containing syntax examples
  • Date range validation with proper datetime-local inputs
  • Spatial filter integration with real-time table updates
  • One-click clear for all filter states

🔄 API & Data Changes

Enhanced Queries

  • Bulk operations with proper error handling and rollback
  • Optimized fetching up to 10,000 items with conditional server-side filtering
  • Spatial intersection calculations for bbox filtering
  • 32-bit metadata detection using existing item data

Error Handling

  • Graceful degradation when TiTiler-PgSTAC unavailable
  • User feedback via snackbar notifications
  • Validation for bulk operations and filter inputs
  • Fallback behavior for 32-bit collections without statistics

📱 Responsive Design

  • Mobile-optimized with appropriate breakpoints (theme.breakpoints.down('xl'))
  • Adaptive layouts with column hiding and message shortening
  • Touch-friendly controls and interactions
  • Consistent styling across different screen sizes

🧪 Backward Compatibility

  • Non-breaking changes to existing functionality
  • Progressive enhancement with new features gracefully degrading
  • Environment-aware features (conditional TiTiler integration)
  • Maintained API compatibility for existing STAC operations

This comprehensive enhancement transforms the STAC Collection Items Modal from a basic list view into a powerful spatial data management interface with advanced filtering, bulk operations, and interactive visualization capabilities.

Images

92385723

@tariqksoliman tariqksoliman self-assigned this Aug 13, 2025
@tariqksoliman tariqksoliman added the enhancement For making an existing feature better label Aug 13, 2025
@tariqksoliman tariqksoliman merged commit 54da03a into development Aug 13, 2025
3 checks passed
@tariqksoliman tariqksoliman deleted the ts-727 branch August 13, 2025 00:44
@github-project-automation github-project-automation Bot moved this to Done in MMGIS Aug 13, 2025
@sonarqubecloud

Copy link
Copy Markdown

tariqksoliman added a commit that referenced this pull request Sep 8, 2025
* Use PolyMeasure to draw great circle lines with Measure Tool

* Round meters to two digits on Measure tool x-axis

* Added option to not display PolylineMeasure tooltips

* Fix bug with noDataValue for single banded COGs (#700)

* Fixed issue where rubberline is not drawn with first click or after zoom

* Fix critical security vulnerabilities identified in SonarQube analysis (#701)

* Fix critical security vulnerabilities identified in SonarQube analysis

This commit addresses 8 legitimate security vulnerabilities while documenting
13 false positives that had adequate existing protections.

Security fixes implemented:

**Path Injection Vulnerabilities (3 issues fixed):**
- middleware.js: Added URL validation requiring /Missions prefix and blocking
  directory traversal sequences (../ and ..\)
- configs.js: Fixed flawed validation logic (AND→OR) and added directory
  traversal protection for mission names

**Cross-Site Scripting (1 issue fixed):**
- configs.js: Added sanitizeInput() function to escape HTML entities in error
  messages containing user-controlled data, preventing reflected XSS attacks

**Insecure Temporary File Creation (4 sample fixes):**
- Replaced insecure tempfile.mktemp() with tempfile.mkstemp() in:
  - auxiliary/demtiles/gdal2demtiles.py (lines 839, 874)
  - auxiliary/gdal2tiles4extent/gdal2tiles4extent.py (line 521)
  - auxiliary/gdal2customtiles/legacy/gdal2customtiles.py (line 601)
- Eliminates race condition vulnerabilities in GDAL processing scripts

**False Positives Documented:**
- SQL Injection (5 issues): Existing parameterized queries and input
  sanitization provide adequate protection
- Analysis details in reviewed_findings.md

All fixes maintain backward compatibility while significantly improving
security posture. Remaining auxiliary Python scripts follow the same
tempfile pattern for completion.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Tweaks to critical security vulnerability fixes

* Support .. as long as it stays within /Missions

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Tariq Soliman <Tariq.K.Soliman@jpl.nasa.gov>

* Minor fix: sort geodataset results

* Make sure polyline measurements are cleared on reset

* #702 Fix LayersTool filtering on non-dynamicExtent props-on-click geodatasets (#703)

* Minor fix: more versatile Help root pathing

* #704 Upgrade All Adjacent Servers (#705)

* Don't use polyline with LOS or else it results in two lines

* Make sure rubberline gets drawn in continuous modes

* Ensure line of sight follows great circle and polyline display cleanup

* Update Dockerfile to update certs

* Use LOS technique to draw great circle lines with varying colors

* Show great circle line even if no DEM #52

* #708 User Account Management (#711)

* #708 User Account Management 1

* #708 user account control part 2

* #708 user account management part 3

* User account control part 4

* #708 minor style tweaks

* Minor resetPassword link fix

* Use contours on all login pages

* #712 Fix some security issues (#713)

* #714 Configurable Wrapping for 2D Map (#715)

* #714 Map maxbounds

* #714 apply to projected maps too

* #716 Per Mission Permissions (#717)

* #716 Per-Mission Permission part 1

* #716 Per Mission Permissions

* #718 Globe Controls clash with Separated Tool buttons in the UI (#719)

* Added multi-platform build to support arm64 architecture

* Fix ensureUser for new Admins

* Separate platform builds and append -arm64 to the end of arm64 images

* Use separate ARM64 runner for faster ARM64 Docker builds

* Fix arm64 tag assignment syntax

* Use a prerendered image for the layer legend #658

* #721 Show, Delete, and Search for individual STAC items (#723)

* #721 STAC item UI part 1

* #721 STAC item UI part 2

* Bump version 4.0.0 -> 4.1.0

* Adjust legend width based on legend image up to 300px

* #724 Legends Max on top (#725)

* Add feature to set Layer header expanded state individually (#726)

* Expand layers feature

* Fix bug with keeping header expanded/unexpanded state

* Expand individual headers only if LayersTool.vars.expanded is not set to true

* #727 STAC item regex search and bulk delete (#728)

* #727 Stac item regex, bbox, bulk delete support part 1

* #727 Support 32bit stac items in map

* #729 Default configuration for live mode (#730)

* #731 Projection Tab Autocomplete, Case Insensitive Mission Sorting, Smart field dsiabling in /configure (#732)

* Filter out blank csv entries in csvToJSON function (#734)

* Add amd64 image suffix and build it last

* Use regular docker build instead of buildx

* Add Legend tool display options (#735)

* Add configuration options

* Add header options for legend tool

* Improve syntax

* #736 Configure Required Field Indicators (#737)

* #738 Fix GeoDataset LOCAL (#739)

* #740 Add mission planet radii (#741)

* Add legend-based property styling for vector layers

* #742 Configure Preview iframe to respect subpaths (#744)

* #709 Improved Continuous Legend Symbology Styling

* #745 Live Follow Mode (#746)

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: ac-61 <ac-61@users.noreply.github.com>
Co-authored-by: Jeff Leach <jl-0@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>
tariqksoliman added a commit that referenced this pull request Sep 9, 2025
* #699 planetcantile p1

* #699 Add planetcantile

* #699 Add untested EPSG:3413 TileMatrixSet

* #699 Update Feature Branch (#747)

* Use PolyMeasure to draw great circle lines with Measure Tool

* Round meters to two digits on Measure tool x-axis

* Added option to not display PolylineMeasure tooltips

* Fix bug with noDataValue for single banded COGs (#700)

* Fixed issue where rubberline is not drawn with first click or after zoom

* Fix critical security vulnerabilities identified in SonarQube analysis (#701)

* Fix critical security vulnerabilities identified in SonarQube analysis

This commit addresses 8 legitimate security vulnerabilities while documenting
13 false positives that had adequate existing protections.

Security fixes implemented:

**Path Injection Vulnerabilities (3 issues fixed):**
- middleware.js: Added URL validation requiring /Missions prefix and blocking
  directory traversal sequences (../ and ..\)
- configs.js: Fixed flawed validation logic (AND→OR) and added directory
  traversal protection for mission names

**Cross-Site Scripting (1 issue fixed):**
- configs.js: Added sanitizeInput() function to escape HTML entities in error
  messages containing user-controlled data, preventing reflected XSS attacks

**Insecure Temporary File Creation (4 sample fixes):**
- Replaced insecure tempfile.mktemp() with tempfile.mkstemp() in:
  - auxiliary/demtiles/gdal2demtiles.py (lines 839, 874)
  - auxiliary/gdal2tiles4extent/gdal2tiles4extent.py (line 521)
  - auxiliary/gdal2customtiles/legacy/gdal2customtiles.py (line 601)
- Eliminates race condition vulnerabilities in GDAL processing scripts

**False Positives Documented:**
- SQL Injection (5 issues): Existing parameterized queries and input
  sanitization provide adequate protection
- Analysis details in reviewed_findings.md

All fixes maintain backward compatibility while significantly improving
security posture. Remaining auxiliary Python scripts follow the same
tempfile pattern for completion.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Tweaks to critical security vulnerability fixes

* Support .. as long as it stays within /Missions

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Tariq Soliman <Tariq.K.Soliman@jpl.nasa.gov>

* Minor fix: sort geodataset results

* Make sure polyline measurements are cleared on reset

* #702 Fix LayersTool filtering on non-dynamicExtent props-on-click geodatasets (#703)

* Minor fix: more versatile Help root pathing

* #704 Upgrade All Adjacent Servers (#705)

* Don't use polyline with LOS or else it results in two lines

* Make sure rubberline gets drawn in continuous modes

* Ensure line of sight follows great circle and polyline display cleanup

* Update Dockerfile to update certs

* Use LOS technique to draw great circle lines with varying colors

* Show great circle line even if no DEM #52

* #708 User Account Management (#711)

* #708 User Account Management 1

* #708 user account control part 2

* #708 user account management part 3

* User account control part 4

* #708 minor style tweaks

* Minor resetPassword link fix

* Use contours on all login pages

* #712 Fix some security issues (#713)

* #714 Configurable Wrapping for 2D Map (#715)

* #714 Map maxbounds

* #714 apply to projected maps too

* #716 Per Mission Permissions (#717)

* #716 Per-Mission Permission part 1

* #716 Per Mission Permissions

* #718 Globe Controls clash with Separated Tool buttons in the UI (#719)

* Added multi-platform build to support arm64 architecture

* Fix ensureUser for new Admins

* Separate platform builds and append -arm64 to the end of arm64 images

* Use separate ARM64 runner for faster ARM64 Docker builds

* Fix arm64 tag assignment syntax

* Use a prerendered image for the layer legend #658

* #721 Show, Delete, and Search for individual STAC items (#723)

* #721 STAC item UI part 1

* #721 STAC item UI part 2

* Bump version 4.0.0 -> 4.1.0

* Adjust legend width based on legend image up to 300px

* #724 Legends Max on top (#725)

* Add feature to set Layer header expanded state individually (#726)

* Expand layers feature

* Fix bug with keeping header expanded/unexpanded state

* Expand individual headers only if LayersTool.vars.expanded is not set to true

* #727 STAC item regex search and bulk delete (#728)

* #727 Stac item regex, bbox, bulk delete support part 1

* #727 Support 32bit stac items in map

* #729 Default configuration for live mode (#730)

* #731 Projection Tab Autocomplete, Case Insensitive Mission Sorting, Smart field dsiabling in /configure (#732)

* Filter out blank csv entries in csvToJSON function (#734)

* Add amd64 image suffix and build it last

* Use regular docker build instead of buildx

* Add Legend tool display options (#735)

* Add configuration options

* Add header options for legend tool

* Improve syntax

* #736 Configure Required Field Indicators (#737)

* #738 Fix GeoDataset LOCAL (#739)

* #740 Add mission planet radii (#741)

* Add legend-based property styling for vector layers

* #742 Configure Preview iframe to respect subpaths (#744)

* #709 Improved Continuous Legend Symbology Styling

* #745 Live Follow Mode (#746)

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: ac-61 <ac-61@users.noreply.github.com>
Co-authored-by: Jeff Leach <jl-0@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>

* #699 Add EPSG:3413

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: ac-61 <ac-61@users.noreply.github.com>
Co-authored-by: Jeff Leach <jl-0@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement For making an existing feature better

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[New Feature]: STAC item regex search and bulk delete

1 participant