clean up some runtime potential bugs (#19447) #4570

Job	Run time
test-coreml / test-backend-macos (coreml, models) / macos-job	13m 55s
test-coreml / test-backend-macos (coreml, operators) / macos-job	10m 53s
test-coreml / test-backend-linux	-1s
test-coreml / package-golden-artifacts	0s
	24m 47s

Provide feedback