fix an issue for real inputs of `irfft` #418

Job	Run time
build_linux (3.9)	2m 13s
build_linux (3.10)	2m 12s
build_linux (3.11)	1m 55s
build_linux (3.12)	2m 30s
build_windows (3.9)	18m 35s
build_windows (3.10)	9m 17s
build_windows (3.11)	18m 35s
build_windows (3.12)	12m 15s
test_linux (3.9, false, ubuntu-latest)	1m 12s
test_linux (3.10, false, ubuntu-latest)	1m 11s
test_linux (3.11, false, ubuntu-latest)	1m 16s
test_linux (3.12, false, ubuntu-latest)	1m 15s
test_windows (3.9, false, windows-2019)	3m 25s
test_windows (3.10, false, windows-2019)	3m 39s
test_windows (3.11, false, windows-2019)	3m 58s
test_windows (3.12, false, windows-2019)	3m 16s
	1h 26m 44s

Provide feedback