server : fix cancel pending task (#16467) #19

Job	Run time
ubuntu-22-vulkan	4m 55s
ubuntu-22-cpu (x64, ubuntu-22.04)	2m 25s
windows-cpu (arm64)	1m 51s
windows-cpu (x64)	3m 6s
macOS-arm64	4m 47s
macOS-x64	3m 14s
windows (vulkan, x64, -DGGML_VULKAN=ON, ggml-vulkan)	6m 34s
windows-hip (radeon, gfx1151;gfx1200;gfx1201;gfx1100;gfx1101;gfx1102;gfx1030;gfx1031;gfx1032)	2m 51s
ios-xcode-build	20m 12s
windows-sycl	8m 52s
windows (opencl-adreno, arm64, -G "Ninja Multi-Config" -D CMAKE_TOOLCHAIN_FILE=cmake/arm64-window...	2m 20s
windows-cuda (12.4)	3m 12s
release	1m 54s
	1h 6m 13s

Provide feedback