audio: crossover: Improve robustness for invalid configuration #23645

Job	Run time
manifest-check	2m 47s
LP64-WIP	3m 13s
build-windows (mnfst, tgl)	8m 17s
build-windows (mnfst, imx8 imx8x)	8m 48s
build-windows (mnfst, lnl)	7m 33s
build-windows (-d, mnfst, lnl)	7m 38s
build-linux (mnfst, mtl)	4m 24s
build-linux (mnfst, ptl)	5m 33s
build-linux (mnfst, lnl)	4m 22s
build-linux (mnfst, imx8m imx8ulp)	4m 29s
build-linux (mnfst, tgl-h)	3m 55s
build-linux (mnfst, tgl)	3m 32s
build-windows (mnfst, mtl)	7m 34s
build-windows (mnfst, ptl)	8m 24s
build-windows (mnfst, tgl-h)	7m 17s
build-windows (mnfst, imx8m imx8ulp)	8m 55s
build-linux (mnfst, imx8 imx8x)	4m 7s
build-linux (mnfst, wcl imx95)	5m 59s
build-linux (-d, mnfst, lnl)	4m 24s
compare-linux-win	29s
	1h 51m 40s

Provide feedback