feat: config refactoring (#406)

* feat: add new CLI args structure and error handling

- Add cli_args.rs with structured CLI argument definitions
- Add error.rs with comprehensive error handling for CLI operations
- Update CLI dependencies in Cargo.toml
- Export new modules in lib.rs

* feat: implement factory pattern for components

- Add DA layer factory for creating Celestia and in-memory DA layers
- Add storage factory for creating database instances
- Add light client factory for node creation
- Add prover factory for prover node creation
- Add telemetry error handling module

* refactor: modernize CLI configuration system

- Refactor cfg.rs to use new factory pattern and improved error handling
- Update main.rs to use new CLI args structure and factories
- Simplify node_types.rs with factory pattern integration
- Improve configuration validation and error reporting

* refactor: remove redis

* refactor: enhance Celestia DA layer integration

- Update Celestia full node and light client implementations
- Improve error handling and configuration in DA layer
- Add better integration with factory pattern
- Enhance utility functions and module exports

* refactor: improve storage layer architecture

- Update database abstraction layer
- Enhance Redis and RocksDB implementations
- Add better error handling and configuration
- Integrate with factory pattern

* refactor: update light client node type

- Add new dependencies to light client Cargo.toml
- Update light client lib.rs to use factory pattern
- Improve module organization and exports

* refactor: modernize prover node implementation

- Update prover lib.rs with factory pattern integration
- Refactor prover module and test implementations
- Update prover engine, sequencer, syncer, and webserver
- Improve error handling and configuration management

# Conflicts:
#	crates/node_types/prover/src/prover/mod.rs
#	crates/node_types/prover/src/prover/tests/mod.rs
#	crates/node_types/prover/src/prover_engine/sp1_prover.rs

* feat(keys): improve key management system

- Update payload handling and key operations
- Enhance signing and verifying key implementations
- Add comprehensive tests for key operations
- Improve error handling and validation

* refactor: improve telemetry system

- Update telemetry dependencies in Cargo.toml
- Enhance telemetry initialization with better error handling
- Update lib.rs to export new error module
- Improve configuration and setup processes

* refactor: update tests and tree implementation

- Update test library with new configuration system
- Enhance key directory tree implementation
- Improve integration with refactored components

* refactor(telemetry): introduce factory pattern for telemetry initialization

- Add TelemetryInstance struct to manage telemetry lifecycle
- Move telemetry creation logic to factory module
- Remove old init.rs module
- Export factory functions from lib.rs

* refactor(da): improve serde configuration and factory pattern

- Change LightClientDAConfig to use tagged enum serialization
- Update Celestia DA configs for better serialization
- Remove commented code and clean up factory implementation

* refactor(storage): update storage factory configuration

* fix(keys): improve error handling and key management

- Add detailed error message to PemCreationError
- Update payload, signing keys, and verifying keys implementations
- Enhance error reporting in key operations

* refactor(lightclient): update dependencies and factory configuration

* refactor(prover): update dependencies and factory configuration

* refactor(cli): major config system refactoring

- Simplify imports and remove unused dependencies (dotenvy, lumina_node)
- Remove old Config struct and WebServerConfig dependency
- Streamline configuration loading with new structure
- Clean up imports from prism_da, prism_prover, and prism_telemetry

* refactor(cli): update CLI argument handling

- Update CLI argument parsing to work with new config system
- Modify argument structures to align with refactored configuration

* refactor(cli): update main.rs to use new factory pattern

- Integrate with new telemetry factory pattern
- Update main application flow to work with refactored config system
- Adapt to new node factory implementations

* build: update Cargo.lock after dependency changes

* feat: add preset system for node configurations

- Add new presets crate with LightClientPreset, FullNodePreset, and ProverPreset
- Implement ApplyPreset trait for configuration management
- Add preset variants for Development and Specter environments
- Update workspace Cargo.toml to include presets crate

* chore: update Cargo.lock for new dependencies

- Add darling, dyn-clone, and ident_case dependencies
- Update serde feature flags for chrono and indexmap
- Lock versions for configuration system refactoring

* feat: integrate preset system with DA layer configurations

- Add preset dependency to DA crate
- Implement ApplyPreset trait for Celestia and memory DA configs
- Update Celestia configurations to support preset-based initialization
- Refactor factory patterns to work with new preset system

* feat: add preset support to storage layer

- Add preset dependency to storage crate
- Implement ApplyPreset trait for DatabaseConfig
- Update storage factory to work with preset-based configurations

* feat: integrate preset system with node type factories

- Add preset dependencies to lightclient and prover crates
- Update factory implementations to support preset-based configuration
- Implement ApplyPreset traits for node-specific configurations

* feat: add configuration management to WASM and UniFFI clients

- Add configuration modules to wasm-lightclient and uniffi-lightclient
- Update WASM client with preset support and error handling improvements
- Update UniFFI bindings to support new configuration system
- Update worker example to use new configuration approach

* feat: implement modular CLI argument system

- Add modular CLI arguments structure with separate modules for DA, database, webserver, and commands
- Implement argument parsing for different node types (light client, full node, prover)
- Add support for preset-based argument configuration
- Replace monolithic CLI args with composable argument modules

* feat: implement configuration application system

- Add apply_args modules to handle configuration application from CLI arguments
- Implement ApplyArgs trait for converting CLI arguments to configurations
- Add modular application logic for DA, database, webserver, and command configurations
- Provide traits and implementations for argument-to-config mapping

* feat: add unified CLI configuration structures

- Add CliLightClientConfig, CliFullNodeConfig, and CliProverConfig structures
- Implement preset application for each configuration type
- Add serde support for configuration serialization/deserialization
- Provide flattened configuration structure for seamless integration

* refactor: remove old CLI configuration system and update dependencies

- Remove legacy cfg.rs and cli_args.rs files
- Add new dependencies: config, darling, dyn-clone, dirs, and serde features
- Update CLI crate to support new modular configuration system

* feat: integrate new configuration system with CLI application

- Update lib.rs to export new configuration modules and structures
- Refactor main.rs to use new modular CLI argument parsing and configuration application
- Integrate preset system with command-line interface
- Update application flow to support new configuration architecture

* test: add test modules and update key verification tests

- Add test module for CLI configuration system
- Update verifying_keys.rs with enhanced test coverage
- Add test infrastructure for validating configuration parsing and application

* refactor: remove obsolete StorageBackend

It has already been replaced by DatabaseConfig.

* chore: update zkvm ELF and keys

* feat: light client works with latest lumina changes

* chore(lightclient): .gitignore for PrismLightClient Swift folder

* refactor(database): remove obsolete imports

* build(deps): bump sp1 crates from 5.0.8 to 5.2.1

* refactor: reorganize workspace dependencies and add testing dependencies

* feat(cli): add file utility helpers for config management

* refactor(cli): extract file utilities and improve error handling

* docs(da): add comprehensive crate documentation with examples

* feat(da): add factory pattern and configuration types

* docs(storage): add comprehensive crate documentation with examples

* feat(storage): add factory pattern and configuration types

* feat(telemetry): add factory pattern and configuration types

* feat(keys): add configuration support and improve serialization

* feat(lightclient): add factory pattern and configuration types

* feat(prover): add factory pattern and configuration types

* refactor(cli): update configuration integration with factory patterns

* chore(ai): add commit instructions to CLAUDE.md

* refactor(prover): Use only one config struct for webserver args

* fix(wasm-lightclient): Do not shadow std Result type

* feat(prover): trace when signing key was not found

* fix(presets): align possible names for dev preset

* build(deps): use prism-telemetry 0.3.0 instead of local again

* feat(cli): improve webserver argument validation

Replace requires with required_if_eq for better conditional validation
of host and port arguments when webserver is active

* refactor(prover): simplify module exports and imports

- Make factory module private and re-export its contents at crate root
- Update imports in CLI to use simplified paths
- Improve API ergonomics while maintaining functionality

* refactor(storage): standardize error handling in RocksDB

Replace anyhow::Result with our standard Result type alias
for consistent error handling across Database trait methods

* feat(da): improve light client configuration and error handling

- Add wasm32 disk storage validation with proper error messaging
- Clean up unused imports and commented code in light client
- Fix directory creation logic to handle base path correctly
- Remove excessive documentation comments for better readability

* refactor: remove unnecessary dependencies

* test: Use correct node parameters in integration test

* test(wasm-lightclient): Adapt to latest changes in test impl of LightClientWorker

* feat(cli): Use dotenv again

* build: move target-specific dependencies and remove unused deps

- Move dirs dependency to non-wasm32 target in da crate
- Remove unused lumina-node dependency from lightclient crate

* refactor(da): reorganize imports in factory module

- Group imports by origin (external, prism_*, std)
- Move target-specific imports to conditional blocks
- Simplify celestia import paths

* refactor(da): improve celestia light client organization

- Move target-specific imports to conditional blocks
- Make LightClientConnection fields private for better encapsulation
- Remove redundant comments and clean up formatting
- Use env::current_dir instead of current_dir import

* refactor(cli): use let-chains in expand_tilde function

Replace nested if-let with modern Rust let-chains pattern for cleaner code

* refactor(da): improve module organization and exports

- Export CelestiaConnection and CelestiaFullNodeDAConfig from celestia module
- Add re-exports for cleaner imports in other modules
- Improve module documentation and structure

* refactor(prover): remove unused get_verifying_key method

Method was not referenced elsewhere in the codebase

* refactor(cli): minor formatting and style improvements

- Adjust line breaks and spacing for better readability
- Consistent parameter formatting across CLI modules

* fix(prover): avoid double key creation

* fix(prover): incorrect prover factory default

* refactor(prover): improve module organization

* build: remove more unused deps

Using cargo machete instead of udeps this time.

* build(da): avoid unused-warnings in wasm

* chore: update zkvm ELF and keys

* build(lightclient): move dirs dependency under non-wasm target

* refactor(lightclient): storage path setter only for non-wasm targets

* refactor(cli): dotenv before cli parsing

* docs(telemetry): Shorten shutdown doc

* test(cli): use nextest instead of serial_test crate

* chore: remove obsolete comments

* chore(ai): specify first rule for writing tests

* build: add clippy.toml to specify exceptions for doc_markdown lint

* feat: activate workspace linting configuration

Add rust.unexpected_cfgs lint with coverage configuration check

* fix(cli): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Fix Self usage in error conversion

* fix(da): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Fix various clippy warnings and improve code quality

* fix(prover): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Fix import statements and remove wildcard imports
- Add allow(dead_code) for legitimate unused struct fields
- Improve clippy compliance

* fix(lightclient): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Add allow(dead_code) for legitimate unused struct fields and imports

* fix(storage): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Add tokio dev dependency for async tests
- Improve clippy compliance and code quality

* fix(telemetry): resolve compiler warnings and activate lints

- Add coverage_attribute feature flag
- Add allow(dead_code) for meter field needed to keep provider alive

* test(da): add comprehensive factory function tests

Add extensive test coverage for:
- Light client DA configuration and preset application
- Full node DA configuration with different backends
- Error handling for configuration mismatches
- Memory DA layer functionality

* test(cli): add comprehensive test coverage

Add test coverage for:
- DA argument application with various configurations
- Database configuration handling
- Webserver configuration
- File utilities functionality
- Coverage attribute annotations for test exclusion

* test(prover): add comprehensive factory configuration tests

Add extensive test coverage for:
- Prover configuration creation and validation
- Preset application for different environments
- Storage path configuration
- Error handling for invalid configurations

* test(lightclient): add comprehensive factory configuration tests

Add test coverage for:
- Light client configuration creation
- Preset application for different networks
- Storage path configuration
- Error handling and validation
- Coverage attribute annotations

* test(storage): add comprehensive factory configuration tests

Add test coverage for:
- Database configuration creation and validation
- Preset application for different storage backends
- Storage path handling and validation
- Error handling for invalid configurations

* test(telemetry): add comprehensive factory configuration tests

Add test coverage for:
- Telemetry configuration creation and validation
- Preset application for different environments
- Service name and endpoint configuration
- Error handling and configuration validation

* build: update Cargo.lock

* refactor(database): re-instate "not found" path for sync height

Author: jns-ps

.config/nextest.toml

@@ -1,3 +1,10 @@
+[test-groups]
+serially = { max-threads = 1 }
+
 [[profile.default.overrides]]
 filter = 'test(/^prover::tests::/)'
 slow-timeout = { period = "60s", terminate-after = 30 }
+
+[[profile.default.overrides]]
+filter = 'package(prism-cli) and test(/^tests/)'
+test-group = 'serially'

CLAUDE.md

@@ -4,7 +4,12 @@ This guide provides comprehensive instructions for AI agents working on the Pris
 
 ## Project Overview
 
-Prism is a high-performance key transparency solution written in Rust, creating SNARKs using the SP1 zkVM over a key directory tree. The codebase is organized into well-defined crates with clear boundaries and responsibilities.
+Prism is a high-performance key transparency solution written in Rust, creating SNARKs using a zkVM over a key directory tree. The codebase is organized into well-defined crates with clear boundaries and responsibilities.
+
+## Github
+- Repository name: prism
+- Repository owner: deltadevsde
+- Repository is https://github.com/deltadevsde/prism
 
 ## Architecture Overview
 
@@ -50,6 +55,20 @@ Prism is a high-performance key transparency solution written in Rust, creating
 - Documentation: Add comments for public APIs and complex logic
 - File organization: Group related functionality in modules
 
+### Testing
+- Do not write tests for derived traits (Clone, Debug, PartialEq, etc), unless there's a custom implementation for them
+
+### Commits
+- Always combine related changes in focused, granular commits
+- Each commit should focus on a single logical change
+- Break changes into multiple smaller commits that each represent a complete, working step
+- Aim for commits that modify ~50 lines or less when possible (does not apply for non-code files)
+- Use conventional commits
+- Allowed types are feat, refactor, fix, build, ci, chore, docs, test, release
+- Use canonical crate names as scopes for conventional commits (e.g. lightclient, prover, keys, da, etc..)
+- Do not mention AI in the description
+- If updated, always make a separate commit "chore: update zkVM ELF and keys" for keys.json and elf files as last commit
+
 ### Performance Considerations
 
 1. **Avoid Allocations in Hot Paths**: Use references and borrowing