Update pattern deletion logic #211

Job	Run time
build-container (amd64, ubuntu-latest)	5m 3s
build-container (arm64, ubuntu-24.04-arm)	3m 45s
pre-push-check	0s
push-multiarch-manifest	0s
	8m 48s

Provide feedback