In containerized environments, resource sharing is typically achieved through volumes or bind mounts. However, these methods lack versioning, dynamic attachment capabilities, and isolation, which are critical for modern containerized applications.
We decided to implement Resource Capsules, a novel approach to resource sharing that provides versioning, dynamic attachment, and isolation. This decision aligns with the goals of enhancing flexibility, security, and efficiency in resource management.
- Versioning: Enables containers to use specific versions of shared resources.
- Dynamic Attachment: Allows capsules to be attached or detached from running containers without restarting them.
- Isolation: Ensures resources are secure and consistent across containers.
- Reusability: Capsules can be reused across multiple containers, reducing duplication.
- Complexity: Adds additional components like Capsule Manager and Capsule Store.
- Overhead: Requires managing capsule lifecycle and storage.
- Traditional Volumes:
- Pros: Simple and widely used.
- Cons: No versioning or dynamic attachment capabilities.
- Bind Mounts:
- Pros: Direct access to host resources.
- Cons: Security risks and lack of isolation.
graph TD
A[Identify Resource Sharing Needs] --> B[Evaluate Existing Methods]
B -->|Lack Versioning| C[Consider Resource Capsules]
C --> D[Design Capsule System]
D --> E[Implement Capsule Manager]
E --> F[Integrate AddResourceCapsule Functionality]
F --> G[Validate with TestAddResourceCapsuleWithDockerPsAndInspect]
G --> H[Deploy Resource Capsules]
This updated diagram includes the AddResourceCapsule function and its corresponding test case, highlighting their role in the validation and deployment of Resource Capsules.
- Capsule Store:
- Acts as a centralized repository for storing and managing capsules.
- Capsules are stored in a compressed and immutable format to ensure integrity.
- Capsule Manager:
- Handles the creation, retrieval, and attachment of capsules.
- Ensures that capsules are properly managed throughout their lifecycle.
- Capsule API:
- Provides a user-friendly interface for interacting with capsules.
- Simplifies the process of creating, updating, and managing capsules.
- Create Capsule: Developers can create capsules containing specific resources, such as libraries or configurations.
capsule create --name libssl --version 1.1.1 --path /usr/lib/libssl.so
- Store Capsule: Capsules are stored in the Capsule Store for easy retrieval.
- Request Capsule: Containers can request specific capsules at runtime, ensuring they have access to the required resources.
basic-docker run --capsule libssl:1.1.1 /bin/myapp
- Attach Capsule: The Capsule Manager dynamically attaches the requested capsule to the container, making it available for use.
flowchart TD
subgraph CapsuleSystem
CapsuleManager["Capsule Manager"]
CapsuleStore["Capsule Store"]
CapsuleAPI["Capsule API"]
end
subgraph ContainerRuntime
Container["Container"]
end
CapsuleManager -->|Manages| CapsuleStore
CapsuleAPI -->|Interacts| CapsuleManager
CapsuleManager -->|Attaches| Container
CapsuleStore -->|Provides| Container
sequenceDiagram
participant Developer
participant CapsuleAPI
participant CapsuleManager
participant Container
Developer->>CapsuleAPI: Create Capsule
CapsuleAPI->>CapsuleManager: Store Capsule
Developer->>CapsuleAPI: Request Capsule
CapsuleAPI->>CapsuleManager: Retrieve Capsule
CapsuleManager->>Container: Attach Capsule
graph TD
A[Create Capsule] --> B[Store Capsule]
B --> C[Attach Capsule]
C --> D[Detach Capsule]
D --> E[Delete Capsule]
These diagrams provide a visual representation of the Resource Capsules architecture, workflow, and lifecycle.
Capsule System:
- Capsule Manager: Manages the lifecycle of capsules.
- Capsule Store: Stores capsules in a compressed, immutable format.
- Capsule API: Provides an interface for capsule operations.
Container Runtime:
- Container: Requests and uses capsules.
Relationships:
- Capsule Manager -> Capsule Store: Manages capsules.
- Capsule API -> Capsule Manager: Interacts with the manager.
- Capsule Manager -> Container: Attaches capsules to containers.
Developer -> Capsule API: Create Capsule
Capsule API -> Capsule Manager: Store Capsule
Developer -> Capsule API: Request Capsule
Capsule API -> Capsule Manager: Retrieve Capsule
Capsule Manager -> Container: Attach Capsule
Create Capsule -> Store Capsule -> Attach Capsule -> Detach Capsule -> Delete Capsule
Capsule Creation
cm := NewCapsuleManager()
cm.AddCapsule("libssl", "1.1.1", "/usr/lib/libssl.so")Capsule Retrieval
capsule, exists := cm.GetCapsule("libssl", "1.1.1")
if !exists {
fmt.Println("Capsule not found")
}Capsule Attachment
err := cm.AttachCapsule("container-1234", "libssl", "1.1.1")
if err != nil {
fmt.Printf("Failed to attach capsule: %v\n", err)
}These textual diagrams and code snippets provide a clear and concise representation of the Resource Capsules feature.
The benchmark tests for the basic-docker-engine were conducted to evaluate the performance of the Resource Capsules feature. Below are the results:
- Benchmark Name:
BenchmarkVolumeAttachment-2 - Iterations: 547,869
- Average Time per Operation: 2,141 ns/op
- Total Execution Time: ~6.476 seconds
| Feature | Basic Docker Engine (Resource Capsules) | Docker System (Volumes) |
|---|---|---|
| Attachment Time | ~2,141 ns/op | Typically higher |
| Dynamic Attachment | Supported | Limited |
| Versioning | Supported | Not Supported |
| Isolation | High | Moderate |
| Flexibility | High | Moderate |
| Security | Enhanced | Standard |
- Performance: The
basic-docker-enginedemonstrates superior performance in attaching resources dynamically, with an average operation time of ~2,141 ns/op. - Feature Set: Resource Capsules provide advanced features such as versioning and enhanced isolation, which are not available in Docker's volume system.
- Use Case Suitability: The
basic-docker-engineis particularly well-suited for scenarios requiring high flexibility, security, and resource versioning.
These results highlight the efficiency and advanced capabilities of the basic-docker-engine compared to the traditional Docker system.
To ensure compatibility and practicality, Resource Capsules will be selectively implemented in real Docker production environments and Kubernetes clusters. This approach allows us to:
- Leverage Existing Infrastructure: Integrate Resource Capsules without disrupting existing workflows.
- Target High-Impact Use Cases: Focus on scenarios where versioning, dynamic attachment, and isolation provide the most value.
- Minimize Overhead: Avoid unnecessary complexity in environments where traditional methods suffice.
-
Docker Production Environments:
- Introduce Resource Capsules as an optional feature.
- Provide a configuration flag to enable or disable capsules per container.
- Ensure backward compatibility with volumes and bind mounts.
-
Kubernetes Clusters:
- Extend Kubernetes storage classes to support Resource Capsules.
- Implement a Capsule Controller to manage capsule lifecycle within the cluster.
- Integrate with Kubernetes APIs for seamless deployment and scaling.
┌───────────────────────────────────────────────────────────────────────────┐
│ Resource Capsules System │
├───────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────┐ │
│ │ Core Components │ │ Platform Drivers │ │ CLI & Management │ │
│ ├─────────────────┤ ├─────────────────┤ ├─────────────────────┤ │
│ │ │ │ │ │ │ │
│ │ Capsule │ │ Docker │ │ CLI Interface │ │
│ │ ├─ Metadata │ │ Driver │ │ │ │
│ │ ├─ Content │◄───►│ │◄───►│ Management API │ │
│ │ └─ Version │ │ Kubernetes │ │ │ │
│ │ │ │ Driver │ │ Dependency │ │
│ │ Dependencies │ │ │ │ Resolver │ │
│ │ │ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐ ┌──────────────────────┐
│ Capsule │ │ CapsuleManager │
├──────────────────────┤ ├──────────────────────┤
│ +name: string │ │ +capsules: Map │
│ +version: string │◄──────────┤ │
│ +path: string │ │ +AddCapsule() │
│ +metadata: Map │ │ +GetCapsule() │
└──────────────────────┘ │ +ListCapsules() │
│ +DeleteCapsule() │
┌────────────────────────┐ └──────────────────────┘
│ CapsuleDependency │ ▲
├────────────────────────┤ │
│ +sourceName: string │ │
│ +sourceVersion: string │ ┌─────────────────────────────┐
│ +targetName: string │ │ │
│ +targetVersion: string │ ┌─────┤ KubernetesCapsuleManager │
│ +isOptional: bool │ │ │ │
└────────────────────────┘ │ ├─────────────────────────────┤
▲ │ │ +client: k8s.Interface │
│ │ │ +namespace: string │
┌─────────────────────────┐ │ │ │
│CapsuleDependencyManager │ │ │ +CreateConfigMapCapsule() │
├─────────────────────────┤ │ │ +GetConfigMapCapsule() │
│ +manager: CapsuleManager│ │ │ +CreateSecretCapsule() │
│ +dependencies: List │◄─┘ │ +GetSecretCapsule() │
│ │ │ +DeleteCapsule() │
│ +AddDependency() │ │ +ListCapsules() │
│ +GetDependencies() │ │ +AttachCapsuleToDeployment()│
│ +ResolveDependencies() │ └─────────────────────────────┘
└─────────────────────────┘
┌────────┐ ┌────────────────┐ ┌─────────────┐ ┌────────────┐
│ Client │ │ KCapsuleManager│ │ Kubernetes │ │ Deployment │
└───┬────┘ └───────┬────────┘ └──────┬──────┘ └──────┬─────┘
│ │ │ │
│ AttachCapsule │ │ │
│───────────────────────> │ │
│ │ │ │
│ │ Get Deployment │ │
│ │─────────────────────────>│ │
│ │ │ │
│ │ │ Get │
│ │ │──────────────────────> │
│ │ │ │
│ │ │ Return Deployment │
│ │ │<──────────────────────┐│
│ │ │ │
│ │ Return Deployment │ │
│ │<─────────────────────────│ │
│ │ │ │
│ │ Add Volume & Mount │ │
│ │─┐ │ │
│ │ │ │ │
│ │<┘ │ │
│ │ │ │
│ │ Update Deployment │ │
│ │─────────────────────────>│ │
│ │ │ │
│ │ │ Update │
│ │ │──────────────────────> │
│ │ │ │
│ │ │ Confirmation │
│ │ │<──────────────────────┐│
│ │ │ │
│ │ Confirmation │ │
│ │<─────────────────────────│ │
│ │ │ │
│ Success │ │ │
│<─────────────────────── │ │
│ │ │ │
┌─────────────────────────────────────────────────────────────────────┐
│ Container │
│ │
│ ┌─────────────────────┐ ┌─────────────────────────┐ │
│ │ Application │ │ /capsules/ │ │
│ │ │ access │ │ │
│ │ ┌─────────────┐ │◄──────────┤ /capsules/config/1.0/ │ │
│ │ │ Application │ │ │ ├─ config.yaml │ │
│ │ │ Code │ │ │ └─ secrets.yaml │ │
│ │ └─────────────┘ │ │ │ │
│ │ │ │ /capsules/api-keys/2.1/ │ │
│ └─────────────────────┘ │ └─ credentials.json │ │
│ └─────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
When a Resource Capsule is created in Kubernetes, it's represented as either a ConfigMap or Secret:
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config-1.0
labels:
capsule.docker.io/name: app-config
capsule.docker.io/version: "1.0"
data:
config.yaml: |
database:
host: db.example.com
port: 5432When attached to a Deployment, the following changes are made:
- A volume is added:
spec:
template:
spec:
volumes:
- name: capsule-app-config-1.0
configMap:
name: app-config-1.0- Volume mounts are added to each container:
spec:
template:
spec:
containers:
- name: app
# ... other container config ...
volumeMounts:
- name: capsule-app-config-1.0
mountPath: /capsules/app-config/1.0
readOnly: true- Compatibility: Ensuring Resource Capsules work alongside existing storage solutions.
- Performance: Minimizing the performance impact of capsule management in high-load environments.
- Adoption: Encouraging users to adopt Resource Capsules without mandating changes to their workflows.
- Extend Capsule API for remote management.
- Gather feedback from early adopters to refine the implementation.
- Implement garbage collection for unused capsules.
- Add support for capsule dependency resolution.
All four ADR-001 claims were formally verified by running
scripts/verify-adr-001.sh against a live AKS cluster
(basic-docker-aks, East US, Kubernetes v1.34.4, single node Standard_B2s).
| Item | Value |
|---|---|
| Cloud | Azure for Students |
| Cluster | basic-docker-aks (East US) |
| Kubernetes version | v1.34.4 |
| Node | aks-nodepool1-22820865-vmss000000 (Standard_B2s) |
| Go version | 1.24 |
| Run date | 2026-04-29 |
| Script | scripts/verify-adr-001.sh |
16 checks passed, 0 failed.
| Claim | Check | On-cluster result |
|---|---|---|
| C1 Versioning | v1.0 ConfigMap carries correct capsule.docker.io/version label |
✔ |
| v2.0 ConfigMap carries correct label | ✔ | |
| v1.0 and v2.0 ConfigMap data are independent (no bleed-through) | ✔ | |
CRD ResourceCapsule object persists spec.version field |
✔ | |
CRD object persists spec.rollback.enabled flag |
✔ | |
Unit: TestKubernetesConfigMapCapsule |
✔ | |
| C2 Dynamic Attachment | Baseline Deployment app-a is Available before any capsule is attached |
✔ |
Unit: TestAttachCapsuleToDeployment — volume and VolumeMount verified |
✔ | |
Live Deployment spec shows volume capsule-attach-cap-1-0 after patch |
✔ | |
| Live container spec shows matching VolumeMount | ✔ | |
| C3 Isolation | Capsules in test namespace are invisible from a second namespace | ✔ |
Cross-namespace kubectl get returns NotFound |
✔ | |
| C4 Reusability | Deployment app-b mounts mylib-1.0 and becomes Available |
✔ |
Deployment app-c mounts mylib-1.0 and becomes Available |
✔ | |
| Single ConfigMap object backs both Deployments (no duplication) | ✔ | |
Unit: TestResourceCapsuleCRDTypes, TestResourceCapsuleCRDDeepCopy, TestResourceCapsuleOperatorCreation |
✔ |
C1 — Versioning holds solid value.
Two ConfigMap-backed capsules (mylib-1.0, mylib-2.0) coexisted in the same
namespace without conflict. The capsule.docker.io/version label was correctly
stored and retrievable. The CRD ResourceCapsule object persisted both the
spec.version string and the spec.rollback.enabled boolean through the
Kubernetes API server. Versioning is implemented cleanly and is directly
queryable via label selectors (kubectl get cm -l capsule.docker.io/name=mylib).
C2 — Dynamic Attachment is real but has an implementation constraint.
The AttachCapsuleToDeployment function correctly adds a volume and VolumeMount
to a live Deployment via a Kubernetes API Update call. However, a bug was
discovered and fixed during this verification: version strings containing dots
(e.g. 1.0) produced volume names like capsule-attach-cap-1.0, which
Kubernetes rejects (DNS subdomain rules prohibit dots in volume names). The fix
replaces dots with dashes in the generated volume name while keeping the mount
path (/capsules/attach-cap/1.0) unchanged. The "without restarting" claim in
the ADR requires clarification — Kubernetes rolling-restarts pods when a
Deployment spec is updated; the capsule attach operation does not bypass this.
C3 — Isolation is enforced by Kubernetes RBAC and namespace scoping, not by
capsule-specific logic. ConfigMaps and Secrets are namespace-scoped objects;
the capsule system inherits this isolation for free. The claim holds, but its
strength comes from the platform rather than from any capsule-specific access
control. Adding dedicated RBAC Role/RoleBinding objects would make this
capsule-owned rather than platform-inherited.
C4 — Reusability holds solid value. Two separate Deployments mounting the
same ConfigMap-backed capsule both reached Available state. Kubernetes
confirmed a single ConfigMap object (one API resource) backing multiple consumers
simultaneously. This is the strongest validated claim — it works exactly as
described in the ADR with zero additional mechanism needed.
kubernetes.go AttachCapsuleToDeployment generated Kubernetes volume names
containing dots (e.g. capsule-name-1.0). Kubernetes volume names must comply
with DNS label syntax and may not contain dots. Fixed by sanitizing the version
string: strings.ReplaceAll(capsuleVersion, ".", "-") in the volume name only;
the mount path retains the original version string.
| File | Purpose |
|---|---|
scripts/setup-azure-aks.sh |
Provision AKS cluster + app registration + set GitHub secrets |
scripts/verify-adr-001.sh |
Automated ADR-001 claim verification on a live cluster |
.github/workflows/aks-lifecycle.yml |
workflow_dispatch with deploy / verify / deploy-and-verify / destroy options |
.github/workflows/azure-aks-verify.yml |
Existing focused verify-only workflow |
April 12, 2025 - Planned August 2, 2025 - Implementation with Kubernetes done April 29, 2026 - All four claims verified on Azure AKS (see Experimental Verification section above)
April 12, 2025