OCI Image Specs support — resolve #5 vs main + finish read/write paths#13
Closed
sashml wants to merge 51 commits into
Closed
OCI Image Specs support — resolve #5 vs main + finish read/write paths#13sashml wants to merge 51 commits into
sashml wants to merge 51 commits into
Conversation
Enhance CI workflow with security permissions and GitHub Pages deploy…
- Introduced bfc_compress.h for compression context and result structures. - Implemented compression and decompression functions in bfc_reader.c and bfc_writer.c. - Added logic to handle compression type selection and thresholds in bfc_add_file. - Enhanced bfc_create to initialize compression settings. - Implemented unit tests for compression functionality in test_compress.c. - Updated CMakeLists.txt to include new test file and link against ZSTD if enabled. - Added functions to set and get compression settings in the BFC writer. - Improved error handling for compression and decompression processes.
Add compression support to BFC library
- Added `test_encrypt.c` to implement unit tests for encryption support, key management, data encryption/decryption, and error handling. - Introduced `test_encrypt_integration.c` for integration tests focusing on encryption context lifecycle, key derivation edge cases, and large data encryption. - Updated `CMakeLists.txt` to include the new encryption test files and link against libsodium if enabled. - Temporarily disabled integration tests in `test_main.c` due to API mismatches.
…encryption key handling in bfc_writer.c
…mments for clarity
…r encryption functions
Add unit tests for encryption functionality and integration tests
… improved error handling
Reorder directory change and container opening in extract command for…
- Add bfc_oci.h header with OCI data structures and function declarations - Add bfc_oci.c implementation with OCI manifest, config, and layer support - Add oci_example.c demonstrating OCI functionality - Add OCI_SUPPORT.md documentation - Update CMakeLists.txt to include OCI support option - Add examples/CMakeLists.txt for building examples This enables BFC to be used as a storage backend for OCI-compliant container images, making it suitable for integration with container runtimes and registries.
- Implemented symlink addition in the container with `bfc_add_symlink`. - Added symlink extraction functionality in `extract_symlink`. - Updated directory processing to handle symlinks during extraction. - Enhanced info display to include symlink statistics. - Modified file mode formatting to recognize symlinks.
Add comprehensive symlink support to BFC containers
- Add comprehensive OCI image manifest and config support - Implement bfc_extract_to_oci function for extracting BFC containers to OCI format - Add dynamic architecture and OS detection for cross-platform compatibility - Create OCI example with proper CMakeLists.txt integration - Update examples/README.md with OCI example documentation - Add proper error handling and memory management - Support for multiple architectures: x86_64, ARM64, RISC-V, PowerPC - Support for multiple OS: Linux, Windows, macOS, BSD variants This enables BFC to be used as an OCI image storage format while maintaining compatibility with existing BFC functionality.
- Fix code formatting according to project standards - Apply consistent indentation and spacing - Sort includes alphabetically - Ensure all C/C++ files follow clang-format conventions This addresses the formatting issues reported in CI/CD pipeline.
Enable BFC to build successfully on FreeBSD by addressing platform-specific compiler requirements and API differences. Changes: - Add -mcrc32 flag for CRC32 intrinsics on x86_64/amd64 architectures - Detect FreeBSD's 'amd64' architecture identifier (in addition to x86_64/AMD64) - Fix unused parameter warning in bfc_os_advise_nocache() on FreeBSD - Fix benchmark_encrypt.c to use correct reader API (bfc_reader_set_encryption_password) - Update documentation to mention FreeBSD support and pkgconf requirement Technical details: - FreeBSD's Clang requires both -msse4.2 and -mcrc32 for _mm_crc32_* intrinsics - CMAKE_SYSTEM_PROCESSOR returns "amd64" on FreeBSD (not "x86_64") - Changed from set_source_files_properties to target_compile_options for proper flag application Tested on FreeBSD 14.3-RELEASE with Clang 19.1.7.
Add FreeBSD build support
- Move src/bfc_oci.c to src/lib/bfc_oci.c for better organization - Update src/lib/CMakeLists.txt to use correct path (bfc_oci.c instead of ../bfc_oci.c) - Add BFC_WITH_OCI compile definition for targets - Create comprehensive test suite in tests/unit/test_oci.c - Tests for validation functions (manifest, config) - Tests for creation functions (from manifest, from index) - Tests for retrieval and extraction functions - NULL pointer validation tests - Memory management tests - Update test infrastructure to include OCI tests - All changes applied from updated_changes.patch
Resolve conflicts across 20 files by unioning two orthogonal feature lines: main's encryption subsystem + Windows/MSVC portability + TOCTOU hardening, and the PR's OCI image-specs support. Remove dead duplicate src/bfc_oci.c (only src/lib/bfc_oci.c is built).
- Add _GNU_SOURCE to bfc_oci.c/oci_example.c/test_oci.c (strdup, fmemopen) - Implement manifest + index JSON serialization via open_memstream; the write path previously passed NULL to bfc_add_file (which rejects NULL) and never worked. bfc_create_from_oci_manifest/index now emit real OCI JSON. - Read path now returns new BFC_E_NOSYS instead of a false BFC_OK; full read needs a JSON parser (follow-up). - Guard all path-building snprintf against truncation (fixes -Werror=format- truncation and the reviewer's digest buffer-overflow concern). - Correct OCI schemaVersion '2.0.1' -> '2' (per OCI image-spec it MUST be 2). - Fix triple-pointer arg in test_oci.c; use valid 64-hex digests in the example (the '...' placeholders tripped the '..' path-traversal guard).
Replace the BFC_E_NOSYS stubs with working readers:
- bfc_get_oci_manifest reads manifest.json (bfc_stat + bfc_read) and parses
schemaVersion/mediaType/config{digest,size}/layers[].digest with libcjson.
- bfc_get_oci_config parses architecture/os/created/author from config.json.
- bfc_list_oci_layers parses layers[] into a bfc_oci_layer_t array.
Wire libcjson as the OCI feature's dependency via pkg-config under BFC_WITH_OCI,
mirroring the existing libzstd/libsodium optional-dep pattern (lib, tests,
example). Add a write->read round-trip test covering manifest, config and
layers.
Document the OCI-module -> BFC-core boundary (gated BFC_WITH_OCI): bfc-oci.calm.json + bfc-oci-c4.md.
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves PR #5 (OCI Image Specs support) against current
mainand finishes the implementation. This branch (oci-image-specs-resolve) ismain+ #5's OCI feature, conflict-free, building clean under-Werror, with the previously-incomplete paths now actually working.What was wrong with #5 (found by building it under
-Werror)mainacross 20 files (main added encryption + Windows/MSVC + TOCTOU hardening since Add OCI Image Specs support #5 branched).BFC_WITH_OCI=ON: missing_GNU_SOURCE(strdup/fmemopen),-Werror=format-truncationon path building.NULLtobfc_add_file(which rejects NULL) and never serialized the manifest/index to JSON.bfc_get_oci_manifest/config,bfc_list_oci_layersreturnedBFC_OKwithout doing anything."2.0.1"— per the OCI image-specschemaVersionMUST be2.src/bfc_oci.c; a triple-pointer bug + invalid...digests in tests/example (tripped the..path-traversal guard).Changes
Merge: union of
main's encryption/portability work and #5's OCI feature (20 files).Write path:
bfc_create_from_oci_manifest/_indexnow serialize real OCI JSON viaopen_memstream.Read path (new): implemented with libcjson
bfc_get_oci_manifest— parsesschemaVersion/mediaType/config{digest,size}/layers[].digestbfc_get_oci_config— parsesarchitecture/os/created/authorbfc_list_oci_layers— parseslayers[]intobfc_oci_layer_t[]Dependency: libcjson is wired as the OCI feature's dep via
pkg-configunderBFC_WITH_OCI, mirroring the existinglibzstd/libsodiumoptional-dep pattern. Build OCI with:apt install libcjson-dev(or vcpkg on Windows).Correctness:
schemaVersion→"2"; truncation guards on all pathsnprintfs; removed deadsrc/bfc_oci.c; fixed the test triple-pointer + example digests.Tests: added a write→read round-trip covering manifest, config and layers.
Docs: CALM + Mermaid C4 model of the OCI↔core boundary under
docs/architecture/.Architecture (critical parts)
OCI module ↔ BFC core boundary — the OCI layer only touches BFC's public API; gated behind
BFC_WITH_OCI(default OFF):Write path — struct → JSON → container entry:
Read path — container entry → JSON → struct (was a
BFC_OKno-op, now real):Resulting BFC container layout for an OCI image:
Verification
-Werror)oci_exampleend-to-endBFC_WITH_OCIdefaults OFF.