Dimensional Data

MSVC ARM64 Optimizations in Visual Studio 2022 17.7 Hongyon Suauthai (ARM) September 28th, 20230 2 In Visual Studio 2022 version 17.6 we added a host of new ARM64 optimizations. In this 2nd edition of our blog, we will highlight some of the performance improvements to MSVC ARM64 compiler backend, we will discuss key optimizations in the Visual Studio 2022 version 17.7 for both scalar ISA and SIMD ISA (NEON). We started introducing these performance optimizations in 17.6 release and we have landed them in 17.7 release. By element operation ARM64 supports by-element operation in several instructions such as fmul, fmla, fmls, etc.  This feature allows a SIMD operand to be computed directly by a SIMD element using an index to access it. In the example below where we multiply a scalar with an array, MSVC duplicated the vector element v0.s[0] into all lanes of a SIMD register, then multiplied that with another SIMD operand represented by array b.  This is not efficient because the dup instruction will add 2 more execution latency before executing the fmul instruction. To better understand this optimization let’s take the following sample code: void test(float * __restrict a, float * __restrict b, float c) {   for (int i = 0; i < 4; i++)        a[i] = b[i] * c; } Code generated by MSVC 17.6:        dup         v17.4s,v0.s[0]        ldr         q16,[x1]        fmul        v16.4s,v17.4s,v16.4s         str         q16,[x0] In Visual Studio 2022 17.7, we eliminated a duplicate instruction. It now can multiply by a SIMD element directly and the code generation is:        ldr         q16,[x1]        fmul        v16.4s,v16.4s,v0.s[0]         str         q16,[x0] Neon supports for shift right and accumulate immediate This instruction right-shifts a SIMD source by an immediate and accumulates the final result with the destination SIMD register. As mentioned earlier, we started working on this optimization in the 17.6 release and completed the feature in the 17.7 release. In the 17.5 release MSVC turned right shifts into left shifts using a negative shift amount. To implement it that way, it copied a constant -2 into a general register, and then duplicated a general register into the SIMD register before the left shift. We eliminated the duplicate instruction and used right shift with an immediate value in the 17.6 release. It was an improvement, but not yet good enough. We further improved the implementation in 17.7 by combining right shift and add by using usra for unsigned and ssra for signed operation. The final implementation is much more compact than the previous ones. To better understand how this optimization works let’s look at the sample code below: void test(unsigned long long * __restrict a, unsigned long long * __restrict b) {    for (int i = 0; i < 2; i++)        a[i] += b[i] >> 2; } Code generated by MSVC 17.5:        mov         x8,#-2        ldr         q16,[x1]        dup         v17.2d,x8        ldr         q18,[x0]        ushl        v17.2d,v16.2d,v17.2d        add         v18.2d,v17.2d,v18.2d         str         q18,[x0] Code generated by MSVC 17.6:        ldr         q16,[x1]        ushr        v17.2d,v16.2d,#2        ldr         q16,[x0]        add         v16.2d,v17.2d,v16.2d         str         q16,[x0] Code generated by MSVC 17.7:        ldr         q16,[x0]        ldr         q17,[x1]        usra        v16.2d,v17.2d,#2         str         q16,[x0] Neon right shift into cmp A right shift on a signed integer shifts the msb, with the result being either -1 or 0, and […]

Microsoft C++ Team at CppCon 2023 Sy Brand September 28th, 20230 2 As always our team will be at CppCon this year with a host of presentations. Many of us will also be present at our team’s booth in the main hall for the first two days of the conference. Come say hi and let us know if you have any questions about our talks, products, or anything else! You can also join the #visual_studio channel on the CppCon Discord to talk to us (note: to join, head to #directory channel first, and check the checkbox next to “Visual Studio” box). We’re also running a survey on the C++ ecosystem. If you have a moment, please take our survey. It’s quick and you could win a utility backpack or duffel bag. Here’s the lineup: Monday 2nd​ Lifetime Safety in C++ – Gabor Horvath – 11am ​ Informal Birds of a Feather for Cpp2/cppfront – Herb Sutter – 12:30pm​ Tuesday​ 3rd​ What’s New in Visual Studio – David Li & Mryam Girmay – 3:15pm​ ​Thursday 5th​​ Cooperative C++ Evolution: Towards a Typescript for C++ – Herb Sutter (Keynote) – 10:30am​ How Visual Studio Code Can Help You Develop More Efficiently in C++ – Alexandra Kemper & Sinem Akinci – 3:15pm​ Regular, Revisited – Victor Ciura – 3:15pm​ Friday 6th​​ Getting Started with C++ – Michael Price – 1:30pm   Sy Brand C++ Developer Advocate, C++ Team Follow

C11 Threads in Visual Studio 2022 version 17.8 Preview 2 Charlie Barto September 26th, 20232 3 Back in Visual Studio 2022 version 17.5 Microsoft Visual C gained preliminary support for C11 atomics. We are happy to announce that support for the other major concurrency feature of C11, threads, is available in Visual Studio version 17.8 Preview 2. This should make it easier to port cross-platform C applications to Windows, without having to drag along a threading compatibility layer. Unlike C11 atomics, C11 threads do not share an ABI with C++’s facilities, but C++ programs can include the C11 threads header and call the functions just like any C program. Both are implemented in terms of the primitives provided by Windows, so their usage can be mixed in the same program and on the same thread. The implementations are distinct, however, for example you can’t use the C11 mutexes with C++ condition variables. C11 contains support for threads and a variety of related concurrency primitives including mutexes, condition variables, and thread specific storage. All of these are implemented in Visual Studio version 17.8 Preview 2. Threads Threads are created with thrd_create, to which you pass a pointer to the desired entry point and a user data pointer (which may be null), along with a pointer to a thrd_t structure to fill in. Once you have a thrd_t created with thrd_create you can call functions to compare it to another thrd_t, join it, or detach it. Functions are also provided to sleep or yield the current thread. int thread_entry(void* data) { return 0; } int main(void) { thrd_t thread; int result = thrd_create(&thread, thread_entry, NULL); if(result != thrd_success) { // handle error } result = thrd_join(thread, NULL); if(result != thrd_success) { // handle error } return 0; }   A key difference between our implementation and C11 threads implementations based on pthreads is that threads can not detach themselves using thrd_current() and thrd_detach(). This is because of a fundamental difference in how threads work on Windows vs Unix descendants and we would require a shared datastructure that tracks thread handles to implement the typical behavior. On Unix derivatives the integer thread ID is the handle to the thread and detaching just sets a flag causing the thread to be cleaned up immediately when it finishes. This makes detached threads somewhat dangerous to use on Unix derivatives, since after a detached thread exits any other references to that thread ID will be dangling and could later refer to a different thread altogether. On Windows the handle to a thread is a win32 HANDLE and is reference counted. The thread is cleaned up when the last handle is closed. There is no way to close all handles to a thread except by keeping track of them and closing each one. We could implement the Unix/pthreads behavior by keeping a shared mapping of thread-id to handle, populated by thrd_create. If you need this functionality then you can implement something like this yourself, but we don’t provide it by default because it would incur a cost even if it’s not used. Better workarounds may also be available, such as passing a pointer to the thrd_t populated by thrd_create via the user data pointer to the created thread. Mutexes Mutexes are provided through the mtx_t structure and […]

MSVC Machine-Independent Optimizations in Visual Studio 2022 17.7 Troy Johnson September 25th, 20232 3 This blog post presents a selection of machine-independent optimizations that were added between Visual Studio versions 17.4 (released November 8, 2022) and 17.7 P3 (released July 11, 2023). Each optimization below shows assembly code for both X64 and ARM64 to show the machine-independent nature of the optimization. Optimizing Memory Across Block Boundaries When a small struct is loaded into a register, we can optimize field accesses to extract the correct bits from the register instead of accessing it through memory. Historically in MSVC, this optimization has been limited to memory accesses within the same basic block. We are now able to perform this same optimization across block boundaries in many cases. In the example ASM listings below, a load to the stack and a store from the stack are eliminated, resulting in less memory traffic as well as lower stack memory usage. Example C++ Source Code: #include bool compare(const std::string_view& l, const std::string_view& r) {    return l == r; } Required Compiler Flags: /O2 X64 ASM: 17.4 17.7 sub        rsp, 56 movups  xmm0, XMMWORD PTR [rcx] mov        r8, QWORD PTR [rcx+8] movaps  XMMWORD PTR $T1[rsp], xmm0 cmp        r8, QWORD PTR [rdx+8] jne        SHORT $LN9@compare mov        rdx, QWORD PTR [rdx] mov        rcx, QWORD PTR $T1[rsp] call       memcmp test       eax, eax jne        SHORT $LN9@compare mov        al, 1 add        rsp, 56 ret        0 $LN9@compare: xor        al, al add        rsp, 56 ret        0 sub         rsp, 40 mov        r8, QWORD PTR [rcx+8] movups  xmm1, XMMWORD PTR [rcx] cmp        r8, QWORD PTR [rdx+8] jne         SHORT $LN9@compare mov        rdx, QWORD PTR [rdx] movq      rcx, xmm1 call        memcmp test        eax, eax jne         SHORT $LN9@compare mov        al, 1 add         rsp, 40 ret         0 $LN9@compare: xor         al, al add         rsp, 40 ret         0   ARM64 ASM: 17.4 17.7 str         lr,[sp,#-0x10]! sub         sp,sp,#0x20 ldr         q17,[x1] ldr         q16,[x0] umov        x8,v17.d[1] umov        x2,v16.d[1] stp         q17,q16,[sp] cmp         x2,x8 bne         |$LN9@compare| ldr         x1,[sp] ldr         x0,[sp,#0x10] bl          memcmp cbnz        w0,|$LN9@compare| mov         w0,#1 add         sp,sp,#0x20 ldr         lr,[sp],#0x10 ret |$LN9@compare| mov         w0,#0 add         sp,sp,#0x20 ldr         lr,[sp],#0x10 ret str         lr,[sp,#-0x10]! ldr         q17,[x1] ldr         q16,[x0] umov        x8,v17.d[1] umov        x2,v16.d[1] cmp         x2,x8 bne         |$LN9@compare| fmov        x1,d17 fmov        x0,d16 bl          memcmp cbnz        w0,|$LN9@compare| mov         w0,#1 ldr         lr,[sp],#0x10 ret |$LN9@compare| mov         w0,#0 ldr         lr,[sp],#0x10 ret   Vector Logical and Arithmetic Optimizations We continue to add patterns for recognizing vector operations that are equivalent to intrinsics or short sequences of intrinsics. An example is recognizing common forms of vector absolute difference calculations. A long series of bitwise instructions can be replaced with specialized absolute value instructions, such as vpabsd on X64 and sabd on ARM64. Example C++ Source Code: #include void s32_1(int * __restrict a, int * __restrict b, int * __restrict c, int n) { for (int i = 0; i < n; i++) { a[i] = (b[i] - c[i]) > 0 ? (b[i] – c[i]) : (c[i] – b[i]); } } Required Flags: /O2 /arch:AVX for X64, /O2 for ARM64 X64 ASM: 17.4 17.7 $LL4@s32_1: movdqu  xmm0, XMMWORD PTR [r11+rax] add     ecx, 4 movdqu  xmm1, XMMWORD PTR [rax] lea     rax, QWORD PTR [rax+16] movdqa  xmm3, xmm0 psubd   xmm3, xmm1 psubd   xmm1, xmm0 movdqa  xmm2, xmm3 pcmpgtd xmm2, xmm4 movdqa  xmm0, xmm2 andps   xmm2, xmm3 andnps  xmm0, xmm1 orps    xmm0, xmm2 movdqu  XMMWORD PTR [r10+rax-16], xmm0 cmp     ecx, edx jl      SHORT $LL4@s32_1 $LL4@s32_1: vmovdqu xmm1, XMMWORD PTR [r10+rax] vpsubd     xmm1, xmm1, XMMWORD PTR [rax] vpabsd     xmm2, […]