Skip to content

Commit 23d8e59

Browse files
committed
[fix.] Docker normal image overlay mount and simplify docs
Use container snapshot's ParentIDs directly instead of looking up init snapshot for normal image layer paths in Docker mode. Return proper overlay mount fallback instead of nil for normal images. Simplify DOCKER.md by adding storage-driver config and removing outdated troubleshooting/migration sections. Signed-off-by: Yifan Yuan <tuji.yyf@alibaba-inc.com>
1 parent 17fc0f5 commit 23d8e59

2 files changed

Lines changed: 39 additions & 144 deletions

File tree

docs/DOCKER.md

Lines changed: 30 additions & 136 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,8 @@ Edit `/etc/docker/daemon.json`:
4949
{
5050
"features": {
5151
"containerd-snapshotter": true
52-
}
52+
},
53+
"storage-driver": "overlaybd"
5354
}
5455
```
5556

@@ -84,23 +85,35 @@ sudo systemctl restart containerd
8485
Once configured, you can run overlaybd images with Docker:
8586

8687
```bash
87-
# Pull an overlaybd image
88-
docker pull registry.hub.docker.com/overlaybd/redis:7.2.3_obd
89-
9088
# Run with overlaybd snapshotter
91-
docker run --rm -it --snapshotter=overlaybd registry.hub.docker.com/overlaybd/redis:7.2.3_obd
92-
```
93-
94-
Or set overlaybd as the default snapshotter:
95-
96-
```bash
97-
# In daemon.json
98-
{
99-
"features": {
100-
"containerd-snapshotter": true
101-
},
102-
"snapshotter": "overlaybd"
103-
}
89+
docker run --rm -it registry.hub.docker.com/overlaybd/redis:7.2.3_obd
90+
91+
Unable to find image 'registry.hub.docker.com/overlaybd/redis:7.2.3_obd' locally
92+
7.2.3_obd: Pulling from overlaybd/redis
93+
Digest: sha256:d4c391ded1cdd1752f0ec14a5c594c8aa072983af10f29032294f4588dc6a5fc
94+
Status: Downloaded newer image for registry.hub.docker.com/overlaybd/redis:7.2.3_obd
95+
1:C 26 Apr 2026 05:09:54.784 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
96+
1:C 26 Apr 2026 05:09:54.784 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
97+
1:C 26 Apr 2026 05:09:54.784 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
98+
1:C 26 Apr 2026 05:09:54.784 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
99+
1:M 26 Apr 2026 05:09:54.785 * monotonic clock: POSIX clock_gettime
100+
_._
101+
_.-``__ ''-._
102+
_.-`` `. `_. ''-._ Redis 7.2.3 (00000000/0) 64 bit
103+
.-`` .-```. ```\/ _.,_ ''-._
104+
( ' , .-` | `, ) Running in standalone mode
105+
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
106+
| `-._ `._ / _.-' | PID: 1
107+
`-._ `-._ `-./ _.-' _.-'
108+
|`-._`-._ `-.__.-' _.-'_.-'|
109+
| `-._`-._ _.-'_.-' | https://redis.io
110+
`-._ `-._`-.__.-'_.-' _.-'
111+
|`-._`-._ `-.__.-' _.-'_.-'|
112+
| `-._`-._ _.-'_.-' |
113+
`-._ `-._`-.__.-'_.-' _.-'
114+
`-._ `-.__.-' _.-'
115+
`-._ _.-'
116+
`-.__.-'
104117
```
105118

106119
### Verifying Device Creation
@@ -118,122 +131,3 @@ lsblk
118131
# Check mounts
119132
mount | grep overlaybd
120133
```
121-
122-
## How It Works
123-
124-
### Container Startup Flow
125-
126-
1. **Image Pull**: Docker pulls overlaybd format layers through containerd
127-
2. **Init Layer Preparation**:
128-
- Docker creates an init layer (key ends with `-init`)
129-
- Snapshotter attaches overlaybd device at parent's mountpoint
130-
- Returns bind mount to init layer's `fs` directory
131-
3. **Container Layer Preparation**:
132-
- Parent is the init layer
133-
- Returns overlay mount with `lowerdir=init/fs:overlaybd_mountpoint`
134-
4. **Container Startup**: Docker starts container with the prepared rootfs
135-
136-
### Device Lifecycle
137-
138-
- **Creation**: Device is created during init layer preparation
139-
- **Sharing**: Multiple containers from the same image share the same overlaybd device
140-
- **Cleanup**: Device is destroyed when the init layer is removed (after all containers exit)
141-
142-
## Limitations
143-
144-
### 1. Read-Write Mode Restrictions
145-
146-
Docker support **only works with `rwMode: "overlayfs"`**. Other modes (`dir`, `dev`) are not supported because:
147-
- Docker requires overlayfs for its layer management
148-
- Init layer mechanism is designed for overlayfs-based storage
149-
150-
### 2. Init Layer Overhead
151-
152-
Docker creates an extra init layer that:
153-
- Adds one additional layer to the overlay stack
154-
- Consumes minimal storage (usually contains only Docker metadata)
155-
- May add slight latency during container preparation
156-
157-
### 3. Device Cleanup Timing
158-
159-
Unlike containerd where devices are cleaned up immediately after container exit:
160-
- Docker may delay init layer removal
161-
- Device remains attached until init layer is garbage collected
162-
- This is normal Docker behavior and doesn't affect functionality
163-
164-
### 4. Storage Compatibility
165-
166-
- Works with **overlaybd format images** (converted or turboOCI)
167-
- Works with **OCI images** (converted on-the-fly, slower startup)
168-
- Does not support:
169-
- Native OCI images with ZFile (use turboOCI instead)
170-
- Direct block device mode (`rwMode: "dev"`)
171-
172-
### 5. Performance Considerations
173-
174-
- First container startup from an image: Full overlaybd device initialization
175-
- Subsequent containers from same image: Reuses existing device (fast)
176-
- Image pull performance: Same as containerd mode
177-
178-
### 6. Known Issues
179-
180-
- **Device busy errors**: May occur if trying to remove an image while containers are still using it. This is handled gracefully by the snapshotter.
181-
- **Init layer identification**: Relies on `-init` suffix in snapshot keys. Unusual Docker configurations may not be detected correctly.
182-
183-
## Troubleshooting
184-
185-
### Container fails to start
186-
187-
1. Check snapshotter logs:
188-
```bash
189-
sudo journalctl -u overlaybd-snapshotter -f
190-
```
191-
192-
2. Verify runtimeType configuration:
193-
```bash
194-
cat /etc/overlaybd-snapshotter/config.json | grep runtimeType
195-
```
196-
197-
3. Ensure overlaybd-tcmu is running:
198-
```bash
199-
sudo systemctl status overlaybd-tcmu
200-
```
201-
202-
### Device not cleaned up
203-
204-
Check if containers/init layers are still holding references:
205-
```bash
206-
# List active snapshots
207-
sudo ctr snapshot --snapshotter=overlaybd ls
208-
209-
# Check mounts
210-
mount | grep overlaybd
211-
```
212-
213-
### Performance issues
214-
215-
1. Enable trace recording for frequently used images
216-
2. Check network connectivity to registry
217-
3. Verify overlaybd cache settings
218-
219-
## Migration from Containerd
220-
221-
If migrating existing setups from containerd to Docker:
222-
223-
1. Images remain compatible (same overlaybd format)
224-
2. Existing pulled images can be reused
225-
3. Configuration changes required:
226-
- Add `runtimeType: "docker"` to snapshotter config
227-
- Enable containerd snapshotter in Docker daemon
228-
4. Restart services in order:
229-
- overlaybd-tcmu
230-
- overlaybd-snapshotter
231-
- containerd
232-
- Docker
233-
234-
## References
235-
236-
- [QUICKSTART](QUICKSTART.md) - General overlaybd setup
237-
- [EXAMPLES](EXAMPLES.md) - Running overlaybd containers
238-
- [TurboOCI](TURBO_OCI.md) - On-the-fly OCI image acceleration
239-
- [Configuration](../script/config.json) - Full configuration options

pkg/snapshot/docker.go

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -135,13 +135,12 @@ func (o *snapshotter) prepareDockerContainerLayer(ctx context.Context, s storage
135135
log.G(ctx).Infof("Docker container layer: using overlaybd mountpoint=%s", mountpoint)
136136
lowerPaths = append(lowerPaths, mountpoint)
137137
} else {
138-
// Normal image: use all parent layers' fs directories
139-
initSnapshot, err := storage.GetSnapshot(ctx, initInfo.Name)
140-
if err != nil {
141-
return nil, fmt.Errorf("failed to get init snapshot: %w", err)
142-
}
143-
for _, pid := range initSnapshot.ParentIDs {
144-
lowerPaths = append(lowerPaths, o.upperPath(pid))
138+
// Normal image: use parent layers' fs directories from container snapshot
139+
// s.ParentIDs[0] is init layer (already added above), [1:] are image layers
140+
if len(s.ParentIDs) > 1 {
141+
for _, pid := range s.ParentIDs[1:] {
142+
lowerPaths = append(lowerPaths, o.upperPath(pid))
143+
}
145144
}
146145
}
147146
} else {
@@ -295,5 +294,7 @@ func (o *snapshotter) dockerContainerLayerMount(ctx context.Context,
295294
}, nil
296295
}
297296

298-
return nil, nil
297+
// Normal image: fall back to standard overlay mount
298+
// s.ParentIDs already contains [initLayerID, imageLayer1ID, ...]
299+
return o.normalOverlayMount(s), nil
299300
}

0 commit comments

Comments
 (0)