fix(vpc-endpoints): resolve critical testing infrastructure issues … (#1)

* fix(vpc-endpoints): resolve critical testing infrastructure issues and YAML operation failures

  ## Summary

  Fixed multiple critical issues in VPC endpoints testing infrastructure that were preventing
  successful test execution and YAML operations. Resolved project ID parsing failures, cloud
  provider mismatches in deletion operations, and flawed verification logic.

  ## Issues Fixed

  ### 1. VPC Endpoints YAML Project ID Resolution
  - **Problem**: VPC endpoint YAML configurations weren't being parsed for project ID resolution
  - **Error**: "failed to resolve project ID for '': project '' not found in organization"
  - **Root Cause**: getProjectID() function only handled SearchIndex resources, not VPCEndpoint
  - **Fix**: Enhanced getProjectID() to extract projectName from VPCEndpoint resource specs
  - **Impact**: YAML operations (plan/apply/destroy) now work correctly with VPCEndpoint resources

  ### 2. Multi-Provider Deletion Support
  - **Problem**: AWS endpoints deleted successfully, but GCP and Azure endpoints failed deletion
  - **Root Cause**: All deletion commands hardcoded to use "--cloud-provider AWS"
  - **Fix**: Dynamic cloud provider extraction using jq from Atlas API responses
  - **Impact**: Multi-provider tests now pass cleanup phase, preventing resource leaks

  ### 3. Verification Logic Alignment with Atlas API
  - **Problem**: Tests searched for YAML metadata names that don't exist in Atlas responses
  - **Root Cause**: Atlas VPC endpoints don't store user-defined names, only system IDs
  - **Fix**: Replaced name-based verification with endpoint count and provider validation
  - **Impact**: All verification phases now pass correctly with accurate results

  ## Technical Changes

  ### Enhanced Project ID Parsing (cmd/infra/apply.go)
  - Added VPCEndpoint support to getProjectID() function
  - Enhanced parsing for DatabaseUser, NetworkAccess, and Cluster resources
  - Improved error handling for project resolution across all resource types

  ### Multi-Provider Deletion Logic (scripts/test/vpc-endpoints-lifecycle.sh)
  - Replaced hardcoded AWS provider with dynamic extraction
  - Added jq-based provider and ID extraction from Atlas API responses
  - Enhanced cleanup_all_vpc_endpoints() with proper provider handling
  - Updated all deletion operations to use correct cloud providers

  ### Verification Logic Overhaul
  - Replaced grep searches for non-existent names with jq length counting
  - Added cloud provider verification using jq -r '.[].cloudProvider'
  - Implemented robust endpoint existence checking based on actual API responses
  - Enhanced error handling and null value validation

  ## Files Modified

  - cmd/infra/apply.go: Enhanced getProjectID() with comprehensive resource support
  - scripts/test/vpc-endpoints-lifecycle.sh: Fixed deletion logic and verification methods

  ## Test Results

  ### Before Fix
  - YAML operations: Failed with project ID resolution errors
  - Multi-provider deletion: GCP/Azure endpoints not deleted (resource leaks)
  - Verification: All attempts failed searching for non-existent names
  - Test suite: Consistent failures in YAML phase

  ### After Fix
  ✅ VPC endpoint YAML operations work correctly with proper project ID resolution
  ✅ Multi-provider deletion works for AWS, Azure, and GCP endpoints
  ✅ Verification logic checks actual endpoint existence and cloud provider types
  ✅ Complete test suite passes all phases including multi-provider scenarios

  ## Verification

  **Project ID Resolution Test:**
  ```bash
  $ matlas infra plan -f vpc-endpoint.yaml --preserve-existing
  Execution Plan  plan-1755680190
  Project         68961f3e6a4bb94d55e6404c (resolved from YAML)
  Stage 0 (1 operations)
  Resource Type  Operation  Resource Name          Risk  Duration
  VPCEndpoint    Create     test-vpc-endpoint-123  low   30s

  Multi-Provider Support:
  - AWS: ✅ Uses --cloud-provider AWS correctly
  - Azure: ✅ Uses --cloud-provider AZURE correctly
  - GCP: ✅ Uses --cloud-provider GCP correctly

  Impact

  The VPC endpoints feature is now fully operational and comprehensively tested:
  - Complete dual interface support (CLI + YAML)
  - Multi-cloud provider support (AWS/Azure/GCP)
  - Enterprise-grade testing with proper resource lifecycle management
  - Robust cleanup and preservation mechanisms
  - Reliable verification based on actual Atlas API responses

  Resolves VPC endpoint testing infrastructure issues and enables full YAML workflow support.

* fix: golangci-lint errors and windows matrix workflow

* fix: lint  and fmt

---------

Co-authored-by: Danny Teller <danny.teller@tipalti.com>

Author: d-teller

.github/workflows/release.yml

@@ -14,7 +14,7 @@ permissions:
 
 env:
   GO_VERSION: '1.24.5'
-  GOLANGCI_LINT_VERSION: 'v2.3.1'
+  GOLANGCI_LINT_VERSION: 'v2.4.0'
 
 jobs:
   # ==============================================================================
@@ -44,19 +44,31 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-go-
 
+      - name: Clean Go Module Cache (prevent lint issues)
+        run: go clean -modcache || true
+
       - name: Download Dependencies
         run: go mod download
 
       - name: Sync vendor (ensure consistent vendoring for linter)
+        shell: bash
         run: |
+          # Clean any existing vendor directory to prevent conflicts
+          if [ -d "vendor" ]; then
+            rm -rf vendor/
+          fi
           go mod tidy
           go mod vendor
+          echo "✅ Vendor directory synced successfully"
 
       - name: Run golangci-lint
         uses: golangci/golangci-lint-action@v7
         with:
           version: ${{ env.GOLANGCI_LINT_VERSION }}
           args: --no-config --enable-only=errcheck,gosec,ineffassign --timeout=5m
+          skip-pkg-cache: false
+          skip-build-cache: false
+          only-new-issues: false
 
       - name: Check Code Formatting
         run: |
@@ -100,13 +112,19 @@ jobs:
         with:
           go-version: ${{ matrix.go-version }}
 
+      - name: Download Dependencies
+        run: go mod download
+
       - name: Sync vendor
+        shell: bash
         run: |
+          # Clean any existing vendor directory to prevent conflicts
+          if [ -d "vendor" ]; then
+            rm -rf vendor/
+          fi
           go mod tidy
           go mod vendor
-
-      - name: Download Dependencies
-        run: go mod download
+          echo "✅ Vendor directory synced for ${{ matrix.os }}"
 
       - name: Create Test Temp Directory
         shell: bash
@@ -180,6 +198,8 @@ jobs:
             ~/go/pkg/mod
             ~/.cache/go-build
           key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
+          restore-keys: |
+            ${{ runner.os }}-go-
 
       - name: Build Binary
         env:
@@ -191,18 +211,20 @@ jobs:
           COMMIT=$(git rev-parse --short HEAD)
           BUILD_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
           
-          # Create dist directory
-          mkdir -p dist
+          # Create unique dist directory for this build
+          DIST_DIR="dist-${{ matrix.goos }}-${{ matrix.goarch }}"
+          rm -rf "${DIST_DIR}"
+          mkdir -p "${DIST_DIR}"
           
           # Build binary
           BINARY_NAME="matlas${{ matrix.ext }}"
           go build \
             -ldflags="-s -w -X main.version=${VERSION} -X main.commit=${COMMIT} -X main.buildTime=${BUILD_TIME}" \
-            -o "dist/${BINARY_NAME}" \
+            -o "${DIST_DIR}/${BINARY_NAME}" \
             .
           
           # Create release archive
-          cd dist
+          cd "${DIST_DIR}"
           if [ "${{ matrix.goos }}" = "windows" ]; then
             zip "matlas_${{ matrix.goos }}_${{ matrix.goarch }}.zip" "${BINARY_NAME}"
           else
@@ -216,12 +238,12 @@ jobs:
         with:
           name: matlas-${{ matrix.os }}-${{ matrix.arch }}
           path: |
-            dist/*.zip
-            dist/*.tar.gz
+            dist-${{ matrix.goos }}-${{ matrix.goarch }}/*.zip
+            dist-${{ matrix.goos }}-${{ matrix.goarch }}/*.tar.gz
           retention-days: 90
 
-  create-checksums:
-    name: 📋 Create Checksums
+  prepare-release-assets:
+    name: 📋 Prepare Release Assets
     runs-on: ubuntu-latest
     needs: build
     steps:
@@ -243,13 +265,6 @@ jobs:
           echo "Generated checksums:"
           cat checksums.txt
 
-      - name: Upload consolidated artifacts
-        uses: actions/upload-artifact@v4
-        with:
-          name: release-artifacts
-          path: dist/
-          retention-days: 90
-
   # ==============================================================================
   # Semantic Release
   # ==============================================================================
@@ -258,7 +273,7 @@ jobs:
     name: 🚀 Semantic Release
     runs-on: ubuntu-latest
     timeout-minutes: 15
-    needs: [create-checksums]
+    needs: [prepare-release-assets]
     if: github.event_name == 'push' && github.ref == 'refs/heads/main'
     outputs:
       new_release_published: ${{ steps.semantic.outputs.new_release_published }}
@@ -271,11 +286,19 @@ jobs:
           # Use a Personal Access Token to trigger other workflows if needed
           token: ${{ secrets.SEMANTIC_RELEASE_TOKEN || secrets.GITHUB_TOKEN }}
 
-      - name: Download consolidated artifacts
+      - name: Download all artifacts to recreate dist/
         uses: actions/download-artifact@v4
         with:
-          name: release-artifacts
-          path: dist/
+          path: artifacts/
+      
+      - name: Recreate dist directory for semantic-release
+        run: |
+          mkdir -p dist
+          find artifacts/ -name "*.zip" -exec cp {} dist/ \;
+          find artifacts/ -name "*.tar.gz" -exec cp {} dist/ \;
+          # Generate checksums
+          cd dist
+          sha256sum *.zip *.tar.gz > checksums.txt || shasum -a 256 *.zip *.tar.gz > checksums.txt
 
       - name: Set up Node.js
         uses: actions/setup-node@v4

.gitignore

@@ -65,4 +65,7 @@ coverage.html
 config.local.yaml
 .matlas/
 test-reports/
-new_gaps.md
+new_gaps.md
+WARP.md
+CLAUDE.md
+.claude/*

CHANGELOG.md

@@ -21,7 +21,31 @@
 
 ## [Unreleased]
 
+### Added
+- New `atlas search` command group for Atlas Search index management
+  - `matlas atlas search list` - List search indexes in cluster or collection
+  - `matlas atlas search create` - Create new search indexes (basic implementation)
+  - Support for both full-text search and vector search indexes
+- New YAML kinds: `SearchIndex` and `VPCEndpoint` for ApplyDocument support
+- Comprehensive examples for search and VPC endpoint configurations
+- Multi-cloud provider support for VPC endpoints (AWS, Azure, GCP)
+
+### Fixed
+- **VPC Endpoints YAML Project ID Resolution**: Fixed critical project ID parsing error where VPCEndpoint YAML configurations weren't being processed for project resolution, causing all YAML operations to fail with "project '' not found in organization" error
+- **VPC Endpoints Multi-Provider Deletion**: Fixed cloud provider mismatch where all deletion operations were hardcoded to use AWS provider, causing GCP and Azure endpoints to fail deletion and creating resource leaks
+- **VPC Endpoints Test Verification Logic**: Fixed verification logic that searched for non-existent YAML metadata names in Atlas API responses, replacing with actual endpoint count and cloud provider validation
+- Enhanced project ID parsing support for DatabaseUser, NetworkAccess, and Cluster resources in YAML configurations
+- Improved VPC endpoints test infrastructure with proper timing mechanisms for Atlas backend processing delays
+- Added robust resource cleanup procedures with dynamic cloud provider extraction
+
+### Changed
+- Updated documentation with Atlas Search and VPC endpoint examples
+- Added support for Atlas Search APIs in SDK v20250312005
+- VPC endpoints testing infrastructure now uses Atlas API-compatible verification methods
+- Enhanced test scripts with comprehensive multi-provider cleanup and verification logic
+
 ### Fixed
+- Updated release process documentation to accurately reflect the consolidated workflow implementation
 - Fixed semantic-release not detecting conventional commits by consolidating workflows
 - Completely redesigned release process following current best practices
 - Consolidated CI/CD, testing, building, and releasing into single workflow

Makefile

@@ -71,4 +71,13 @@ generate-mocks:
 install-hooks:
 	mkdir -p .git/hooks
 	cp scripts/pre-commit .git/hooks/pre-commit
-	chmod +x .git/hooks/pre-commit 
+	chmod +x .git/hooks/pre-commit
+
+install:
+	./install.sh
+
+install-user:
+	MATLAS_INSTALL_DIR=~/.local/bin ./install.sh
+
+uninstall:
+	./uninstall.sh