audio: crossover: Improve robustness for invalid configuration #23519

Job	Run time
LP64-WIP	3m 49s
manifest-check	3m 41s
build-linux (mnfst, ptl)	5m 18s
build-linux (mnfst, tgl)	4m 33s
build-linux (-d, mnfst, lnl)	4m 31s
build-windows (-d, mnfst, lnl)	9m 35s
build-windows (mnfst, tgl)	8m 46s
build-windows (mnfst, ptl)	11m 1s
build-windows (mnfst, imx8 imx8x)	10m 20s
build-windows (mnfst, tgl-h)	8m 24s
build-windows (mnfst, lnl)	9m 48s
build-linux (mnfst, imx8m imx8ulp)	4m 46s
build-windows (mnfst, mtl)	9m 8s
build-linux (mnfst, tgl-h)	4m 34s
build-windows (mnfst, imx8m imx8ulp)	10m 1s
build-linux (mnfst, lnl)	4m 27s
build-linux (mnfst, imx8 imx8x)	4m 49s
build-linux (mnfst, mtl)	4m 45s
build-linux (mnfst, wcl imx95)	7m 29s
compare-linux-win	39s
	2h 10m 24s

Provide feedback