Has someone (partially) worked on ARM64EC builds for Windows?

[Olivier Mascia]

Hello,

> I will have to work on adjusting the quite significant build mechanism in order to produce OpenSSL 3.0.x binaries in the ARM64EC mode.  Fundamentally it is an ARM64 build, with '/arm64EC' on the 'cl' commands, and with '/MACHINE:ARM64EC' on the 'link' commands.

Just for sharing information, as this is the users list, here are my very preliminary results attempting such a build.


*) In real short: doable (and it runs), albeit not very valuable at the moment.  My limited understanding of OpenSSL code structure and the details of its build system are obviously my number one limiting factor.


Windows on ARM has been very limited to few very specific configurations for a long time (smaller handheld devices, tablets, and the like).  Windows on ARM64 has broaden the view a little bit (more powerful tablets and other small form-factor computers).  Windows 11 "desktop" on ARM64 is still not an off-the-shelf commercial product which anyone could install on whatever ARM-based PC (which one?, as a starter), though things have been moving much faster over the last months (<12).  There are some specific end-users hardware products, based on ARM design, delivered with Windows 11 already.

https://learn.microsoft.com/en-us/windows/arm/overview

Windows 11 ARM(64 only) is also available to any user enrolled in the Windows Insiders program, through its beta release cycle or its dev release cycle. There are developer-oriented devices available, such as what's called 'Windows Dev Kit 2023' (code name "Project Volterra").  Though a very common way to experiment around Windows 11 on ARM64 and start getting software nearly ready for wider deployments, is to run it inside a virtual machine on some host ARM-based. There are multiple solutions available for doing that more or less easily, more or less comfortably. The Parallels Desktop 18 for Mac, running on Mac Silicon (M1 or M2) is one very decent solution, albeit purely commercial and I'm not writing this to recommend this or any other path.

https://learn.microsoft.com/en-us/windows/arm/dev-kit/

The ARM64EC ABI itself (https://learn.microsoft.com/en-us/windows/arm/arm64ec-abi) is designed to provide ARM native-level functionality and performance, while providing transparent and direct interoperability with x64 code running under emulation.


1) I managed to build OpenSSL 3.0.8 for that ARM64EC ABI. First I had to locally fix the small 3.0.8 bug regarding usage of _umul128 in ARM builds made using Visual Studio.  Line 122 of crypto/bn/rsa_sup_mul.c reads:

#elif (BN_BYTES == 8) && (defined _MSC_VER)

and as such triggers, in that #elif section, the use of the intrinsic _umul128, despite the build possibly targeting ARM instructions.  There a are a number of ways to workaround that, I simply chose the simplest / most immediate path to temporarily move on :

#elif (BN_BYTES == 8) && (defined _MSC_VER && !(defined _M_ARM || defined _M_ARM64 || defined _M_ARM64EC))

One might be tempted to use _M_AMD64 here, though when _M_ARM64EC is defined, _M_AMD64 is too, by MS compilers supporting _M_ARM64EC.  As the last one is my specific goal for now, this simply does it.  But there probably is a cleaner way to fix that.


2) For the build system, I awfully hacked the Configurations/10-main.conf file, by adding the following around line 1421, between the definitions for "VC-WIN64A" and "VC-WIN32" :

    "VC-WIN64-ARMEC" => {
        inherit_from     => [ "VC-WIN64-common" ],
        AS               => "armasm64",
        ASFLAGS          => "-nologo",
        asoutflag        => "-o",
        asflags          => "-machine ARM64EC",
	ARFLAGS		=> add("/MACHINE:ARM64EC"),
	cflags		=> add("/arm64EC"),
        lflags          => add("/MACHINE:ARM64EC"),
        sys_id           => "WIN64_ARMEC",
        uplink_arch      => 'aarch64',
        asm_arch         => 'aarch64',
        perlasm_scheme   => "auto",
        multilib         => "-arm64ec",
    },	

Okay, that's probably not a recommended way to go in order to submit a patch one day, but at the moment I'm only experimenting for myself, while sharing findings : more experienced people here with the build system and the code base might comment and give alternate paths to explore.

I then build, from a clean directory, this way:

(having set appropriate environments variables for using the right MS command-line tools)

perl \osrc\Configure VC-WIN64-ARMEC --openssldir=C:\ProgramData\OpenSSL no-deprecated no-asm
nmake
nmake test
nmake DESTDIR=\odll install_sw

As you can see, I couldn't identify if there is ASM code appropriate for ARM64 (I guess there is) and which could be assembled by the 'armasm64' assembler tool, provided with Visual Studio.  This is why I had to use 'no-asm' on the Configure command-line.

Though with that in place, the build runs apparently fine, can link into some existing programming, compiled in ARM64EC ABI.


3) The end result of this preliminary test (with 'no-asm') is that the TLS I/O performance of my host test application heavily decreases (10 times slower or about, that's noticeable) when linking that ARM64EC OpenSSL 'no-asm' build, compared to linking the X86 build (using proper ASM code where available) and let it be emulated.


I'll dig deeper soon, as I learn.  Next step will be to tweak to build the x86_x64 asm files as such (through nasm) as ARM64EC ABI specifically allows that, mixing x64 and ARM64, function by function. I expect the ASM portions might exhibit higher performance, even though through the X86 emulation compared to their no-asm C equivalents compiled as ARM64.  That test is interesting to run anyway, just per se.

-- 
Olivier Mascia


> Le 7 févr. 2023 à 20:56, Olivier Mascia via openssl-users <openssl-users at openssl.org> a écrit :
> 
> Hello,
> 
> I will have to work on adjusting the quite significant build mechanism in order to produce OpenSSL 3.0.x binaries in the ARM64EC mode.  Fundamentally it is an ARM64 build, with '/arm64EC' on the 'cl' commands, and with '/MACHINE:ARM64EC' on the 'link' commands.
> 
> As to not reinvent the wheel, would anyone have already started, or done, that before?
> 
> Fundamentally the ARM64EC stuff is ARM64 code production, albeit with an adjusted ABI which allows an ARM64EC executable, essentially made of ARM64 code, to load and use, including be statically linked with, portions of code compiled for x64 (AMD64).  The x64 bits are runtime translated.  It is very useful to easily get large software booted on ARM64, even though some dependents bits have to remain Intel/AMD x64 for some time (third party libraries).  The limitation is that pure ARM64 bits cannot be linked with ARM64EC bits, because they do not share the exact same ABI.
> 
> Today, I use WIN64A builds of OpenSSL 3.0.7 without any issue into larger works built as ARM64EC.  To get some more performance from OpenSSL, I'd like to experiment with it built as ARM64EC.
> 
> If anyone has past experience with building OpenSSL 3.0 as ARM64EC on Windows world and wishes to share ideas, gotchas, or hints, I'm all ears.  In between I will start some experiment, while diving into OpenSSL build procedure.
> 
> Thanks!
> -- 
> Olivier Mascia