[Feature request] native sse2avx option support

[Andarwinux]

Clang and GCC support use -msse2avx to convert legacy SSE instructions into VEX-prefixed, which avoids AVX-SSE transition penalties and allows unnecessary vzeroupper to be omitted. For projects that use x86inc.asm, -DFORCE_VEX_ENCODING=1 can be used, but native option support in NASM would be preferable.

[lab-bat]

The VideoLAN Gitlab Projects seems to use NASM/YASM syntax to generate assembly source code. The switch FORCE_VEX_ENCODING was used in macros such as INIT_XMM under the /common/x86 of x264 GIT repository. The switch avx_enabled was carried out to more macros everywhere.

The AVX instruction set expansion can in place SSE instruction set expansion. Lots of legacy code rewrite that needs efforts can be free I guess.

However, I would not be happy to have assembler program to cross the third rail. The developer of x264 utializes the NASM macro. Macros are used to make a sequence of computing instructions available to the programmer as a single program statement. The NASM macro is nicely used for niche assembly optimazation here, but I believe NASM macros can be used more extensively in general scenarios.

[Andarwinux]

Yes, FORCE_VEX_ENCODING is a convenience macro provided by x86inc.asm, but not all projects or code use x86inc.asm. I noticed this issue when investigating the performance of x86-64-v3 distributions like CachyOS.

I think it would be useful if NASM itself provided a --sse2avx option to transparently transcode legacy SSE to VEX-prefixed when emitting object files in backend, rather than relying on frontend macro magic. This would allow even code that doesn't use x86inc.asm to migrate to AVX-128 for free, and still allow source-level support for baseline x86-64 to be retained.

[hpax]

So I have pushed an "sse2avx" experimental branch for this. It is not production quality at this point; in fact, it seems to cause some unrelated regressions.

If you would like to test it out and see if it does what you need, that would be useful.

[Andarwinux]

So I have pushed an "sse2avx" experimental branch for this. It is not production quality at this point; in fact, it seems to cause some unrelated regressions.

If you would like to test it out and see if it does what you need, that would be useful.

I tried sse2avx branch, but it looks like sse2avx isn't a cli option? The expected use case is enable via cli option append by build system, without the need to modify asm source.

[ludloff]

As desirable as optional SSE2AVX promotion might seem,
SSE and AVX actually do have rather different behavior.

See e.g. SDM 092 Vol1 Section 14.1.1 and 14.9.

---

Here is some 15 year old code snippet which achieves the
promotion with macros rather than assembler changes:

%macro reptok 0-*.nolist
%rep %0
m %1
%rotate 1
%endrep
%endmacro

; Without %REPTOK support it is a
; little more tricky to pass that
; desired macro name in. We can't
; use %?, because it would expand
; to m. So we make use of another
; trick: we sneak in a silent 3rd
; argument with the macro name.

%macro m 1

%imacro %1 2-3.nolist %1
%ifdef USE_SSE2
%3 %1,%2
%elifdef USE_AVX1
v%3 %1,%1,%2
%endif
%endmacro

%endmacro

reptok addpd,addps ; ... (3-op)

;;;%undefm m 1

%macro m 1

%imacro %1 2-3.nolist %1
%ifdef USE_SSE2
%3 %1,%2
%elifdef USE_AVX1
v%3 %1,%2
%endif
%endmacro

%endmacro

reptok ptest,movntdqa ; ... (2-op)

;;;%undefm m 1

;;;%undefm reptok 0-*

;---------

bits 64

%define EA [rsi+rax*2+0x12345678]

%define USE_SSE2

addpd xmm0,xmm1
addpd xmm0,EA
addps xmm0,xmm1
addps xmm0,EA

ptest xmm0,xmm1
ptest xmm0,EA
movntdqa xmm0,EA

%undef USE_SSE2

%define USE_AVX1

addpd xmm0,xmm1
addpd xmm0,EA
addps xmm0,xmm1
addps xmm0,EA

ptest xmm0,xmm1
ptest xmm0,EA
movntdqa xmm0,EA

%undef USE_AVX1

; EOF

[hpax]

@ludloff : we know, this isn't intended as a general option.

@Andarwinux : use nasm --before "[sse2avx]" ...

[Andarwinux]

@ludloff : we know, this isn't intended as a general option.

@Andarwinux : use nasm --before "[sse2avx]" ...

Wow, I didn't know it could be used that way before.

It looks like there's a bug.

test.asm.txt

I tried to apply sse2avx to this asm extracted from libaom, but only movdqu was converted to VEX encoding, and other instructions didn't work.

[hpax]

As I said, experimental... :)