Multiple evaluations of a GEP

[momchil-velikov]

Bugzilla Link	51184
Version	trunk
OS	Windows NT
Attachments	test case
CC	@preames,@RKSimon,@shubhamnarlawar77,@sjoerdmeijer

Extended Description

In the attached test case there is this instruction

  %h3 = getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, i64 0, i32 0

and 12-13 more similar instructions like

   %h3.3 = getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, i64 3, i32 0

which begin with the same part getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom

When the attached test case is compiled with

opt --passes="default<O3>" store.ll -S -o - | lcc

the output assembly file still contains multiple evaluations of that part, e.g.

	mov	w15, #48
	madd	x15, x9, x15, x8
 ...
	mov	w13, #48
	madd	x13, x9, x13, x8

etc.

[momchil-velikov]

assigned to @shubhamnarlawar77

[shubhamnarlawar77]

Can you please share C/C++ testcase of above store.ll file?

[shubhamnarlawar77]

@momchil-velikov - Can you please share the expected assembly?

Like,

Does current assembly -

mov	w15, #​48
madd	x15, x9, x15, x8
...
mov w13, #​48
madd x13, x9, x13, x8

should look something like this?

mov	w15, #​48
madd	x15, x9, x15, x8

[fhahn]

I think a simple reproducer is the following, there should be no need for the C/C++ sources (https://llvm.godbolt.org/z/Tf9hEY1EM):

%struct = type { i32, i32, i32 }

define i32 @bad(%struct* %ptr, i64 %idx, i1 %c) {
  %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
  %lv.1 = load i32, i32* %gep.1
  br i1 %c, label %then, label %else

then:
  %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
  %lv.2 = load i32, i32* %gep.2
  %res = add i32 %lv.1, %lv.2
  ret i32 %res

else:
  ret i32 0
}

define i32 @good(%struct* %ptr, i64 %idx, i1 %c) {
  %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
  %lv.1 = load i32, i32* %gep.1
  %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
  %lv.2 = load i32, i32* %gep.2
  %res = add i32 %lv.1, %lv.2
  ret i32 %res
}

llc -mtriple=arm64-apple-darwinos produces:

_bad:                                   ; @bad
        tbz     w2, #0, LBB0_2
        mov     w8, #12
        mov     w9, #12
        madd    x8, x1, x8, x0
        madd    x9, x1, x9, x0
        ldr     w8, [x8, #4]
        ldr     w9, [x9, #8]
        add     w0, w8, w9
        ret
LBB0_2:
        mov     w0, wzr
        ret
_good:                                  ; @good
        mov     w8, #12
        madd    x8, x1, x8, x0
        ldp     w9, w8, [x8, #4]
        add     w0, w9, w8
        ret

Note the following sequence, where both madds compute the same values

        mov     w8, #12
        mov     w9, #12
        madd    x8, x1, x8, x0
        madd    x9, x1, x9, x0

[fhahn]

One option would be to fix it in the backend.

The problem with splitting up the GEPs in the middle-end in common base and offsets in the middle-end is that transformations like instcombine will fold the split up GEPs together, as in https://llvm.godbolt.org/z/57nYKxqPT

[shubhamnarlawar77]

Hello,

From above suggestions, I think #50528 and #50529 seems related. Here is the detailed analysis -

---------------------------------Basic Block layout analysis--------------------------------------------

Test case reproducing issue 50528 -
Case a:
—---------------------LLVM-IR—-------------------------------------

%struct = type { i32, i32, i32 }

define i32 @bad(%struct* %ptr, i64 %idx, i1 %c) {
 %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
 %lv.1 = load i32, i32* %gep.1
 br i1 %c, label %then, label %else


then:
 %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
 %lv.2 = load i32, i32* %gep.2
 %res = add i32 %lv.1, %lv.2
 ret i32 %res


else:
 ret i32 0
}


define i32 @good(%struct* %ptr, i64 %idx, i1 %c) {
 %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
 %lv.1 = load i32, i32* %gep.1
 %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
 %lv.2 = load i32, i32* %gep.2
 %res = add i32 %lv.1, %lv.2
 ret i32 %res
}

On compiling above LLVM IR to AArch64 assembly we get -

-—--------------------Assembly—----------------------------------------

_bad:                                   ; @bad
       tbz     w2, #0, LBB0_2
       mov     w8, #12
       mov     w9, #12
       madd    x8, x1, x8, x0
       madd    x9, x1, x9, x0
       ldr     w8, [x8, #4]
       ldr     w9, [x9, #8]
       add     w0, w8, w9
       ret
LBB0_2:
       mov     w0, wzr
       ret
_good:                                  ; @good
       mov     w8, #12
       madd    x8, x1, x8, x0
       ldp     w9, w8, [x8, #4]
       add     w0, w9, w8
       ret

—-------------------------------------------------------------------------------------------------------

Case b:

But, if we run instcombine pass on above case a LLVM IR and generate assembly for the same, then bug is gone -

–-----case (b) LLVM IR obtained after running -instcombine on above case (a) LLVM IR—-----

%struct = type { i32, i32, i32 }

define i32 @bad(%struct* %ptr, i64 %idx, i1 %c) {
 br i1 %c, label %then, label %else


then:                                             ; preds = %0
 %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
 %lv.1 = load i32, i32* %gep.1, align 4
 %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
 %lv.2 = load i32, i32* %gep.2, align 4
 %res = add i32 %lv.1, %lv.2
 ret i32 %res


else:                                             ; preds = %0
 ret i32 0
}

—------------------------------------------------------------------------------------------------------------------
Generating assembly on case(b) llvm IR leads us to see only one GEP evaluation

—--------------------------------------------------assembly------------------------------------------------

_bad:                                   ; @bad
       tbz     w2, #0, LBB0_2
       mov     w8, #12
       madd    x8, x1, x8, x0
       ldp     w9, w8, [x8, #4]
       add     w0, w9, w8
       ret
LBB0_2:
       mov     w0, wzr
       ret

—-----------------------------------------------------------------------------------------------------------------
Observations on basic block layout using below testcase -
—-------------------------------Original C++ code—--------------------------------------------------------

extern "C" {
int abs(int);
void use(void *);
}


struct state_t {
 int f;
 unsigned h;
 unsigned int a;
 unsigned int b;
};


struct bucket_t {
 unsigned int h;
 unsigned int : 32;
 unsigned char d;
 unsigned char a;
};


struct entry_t {
 bucket_t buckets[4];
};


const unsigned size = 15000000;
entry_t *table;
unsigned int a;


state_t s;


void store() {
 int i, mi, d, md;
 unsigned h;
 entry_t *t;
 bucket_t *e;


 if (!s.f)
   h = s.h + 1;
 else
   h = s.h;


 t = &table[h % size];


 md = 65535;


 for (i = 0; i < 4; i++) {
   /*if (t->buckets[i].h == h) {
     s.a++;
     mi = i;
     break;
   }


   s.a++;
   if (t->buckets[i].a == a)
     s.b++;


   d = t->buckets[i].d - abs((int)(t->buckets[i].a - a)) * 1024;


   if (d < md) {
     md = d;
     mi = i;
   }*/
 }


 e = &t->buckets[mi];
 use(e);
}

—------------------------------------------------------------------------------------------------------

Case 1 : mi global

mi - global and testcase remains same i.e. for loop is empty - I can see new instructions here - "adrp" and "ldrsw" probably related to global offset computation

https://clang.godbolt.org/z/bW13nch45

Case 2: mi - local and explicitly assigned

If we assign local mi then we are getting mov and smaddl for the assignment instruction and no change in basic block layout.

https://clang.godbolt.org/z/a8M5T3P69

Case 3: mi - global and explicitly assigned

This case seems similar to case 1 as there is only assignment to mi

https://clang.godbolt.org/z/hq8svneWc

—------------Now changing below snippet in above original testcase—----------------------------

if (!s.f)
{
    h = s.h + 1;
mi = 0;
}
  else
{
    h = s.h;
mi = 1;
}

—----------------------------------------------------------------------------

if mi is local - https://clang.godbolt.org/z/jd1qM5ors

if mi is global - https://clang.godbolt.org/z/efqP5zn4x

From above observations, it seems centred more around basic block laying outs.

Hence, issue #50528 and #50529 seems to be related.

[shubhamnarlawar77]

@fhahn - Can you please give comment on basic block layout on above analysis?

[momchil-velikov]

I don't think I agree with this analysis.

Taking your first example, you can see redundant evaluations even though the
code ended up in the same basic block

    bad:                                    // @bad
            tbz     w2, #0, .LBB0_2
            mov     w8, #12
            mov     w9, #12
            madd    x8, x1, x8, x0
            madd    x9, x1, x9, x0
            ldr     w8, [x8, #4]
            ldr     w9, [x9, #8]
            add     w0, w8, w9
            ret
    .LBB0_2:
            mov     w0, wzr
            ret

This example can be optimised at the LLVM IR level:

    define i32 @bad1(%struct* %ptr, i64 %idx, i1 %c) {
     %gep.0 = getelementptr %struct, %struct* %ptr, i64 %idx
     %gep.1 = getelementptr %struct, %struct* %gep.0, i64 0, i32 1
     %lv.1 = load i32, i32* %gep.1
     br i1 %c, label %then, label %else
    then:
     %gep.2 = getelementptr %struct, %struct* %gep.0, i64 0, i32 2
     %lv.2 = load i32, i32* %gep.2
     %res = add i32 %lv.1, %lv.2
     ret i32 %res
    else:
     ret i32 0
    }

which yields:

    bad1:                                   // @bad1
            tbz     w2, #0, .LBB1_2
            mov     w8, #12
            madd    x8, x1, x8, x0
            ldp     w9, w8, [x8, #4]
            add     w0, w9, w8
            ret
    .LBB1_2:
            mov     w0, wzr
            ret

https://gcc.godbolt.org/z/4WGfMear5

This would be even more applicable to the original test case, where the GEP
instructions are quite bigger, e.g

    %h3 = getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, i64 0, i32 0

and contain a substantial common part like:

    getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, ...

Moreover, in all cases those GEPs are dominated by the above %h3 = ...
instruction in the entry block, this the redundancy can be trivially eliminated.

While the basic block layout (i.e. linearisation of the CFG) can end up with a
layout where two basic blocks are merged, because one falls through to the
other, I don't think it's relevant, as it happens almost as the last pass in the
optimisation pipeline and thus won't open new optimisation opportunities.

As for CFG transformation that happen earlier, I don't think they can
restructure the test case in a way as to group all (or most of) the redundant
GEPs in one basic block.

For this issue, it was suggested to investigate SeparateConstOffsetFromGEP and
other passes that are enabled by -aarch64-enable-gep-opt=true, even though
from a quick glance I couldn't see it making a difference.

I'm afraid I don't understand the examples with the mi variable ("Case 1",
"Case 2", etc). As far as I can see, there's nothing special regarding that
variable, it's allocated to a register and there are a few constant assignments
to it and that's as good as it gets.

[navinksinha]

Dear @momchil-velikov,

Thanks for your comments. I would want to clarify something on the points that you have mentioned above, and let us take it one point at a time.

The first example in the listing given by @shubhamnarlawar77 is the original test-case and its output.

We surmised that the problem is primarily because the code was spread across basic blocks. To verify this, we ran InstrCombine explicitly and figured out that we got exactly the output that you have been suggesting.

This is what we got -

_bad:                                   ; @bad
       tbz     w2, #0, LBB0_2
       mov     w8, #12
       madd    x8, x1, x8, x0
       ldp     w9, w8, [x8, #4]
       add     w0, w9, w8
       ret
LBB0_2:
       mov     w0, wzr
       ret

Although this was achieved via a different IR-pathway than you mention, the output indeed is equivalent.

What we have been proposing is that the LLVM does already have the necessary tools to get this in place, and we might have to ensure that those passes/optimizations are run in proper order.

Let me know if I am right so far. Once confirmed on this, I would like to seek some clarification on other points as well.

[momchil-velikov]

The first example in the listing given by @shubhamnarlawar77 is the original test-case and its output.

No, I don't think so.

One of the examples is the one given by @fhahn and the other is the original C++ testcase, but with the interesting part (which generates the redundant madds) commented out.

We surmised that the problem is primarily because the code was spread across basic blocks. To verify this, we ran InstrCombine explicitly and figured out that we got exactly the output that you have been suggesting.

Looking at the first example, opt -instcombine sinks the first load (and thus its GEP) towards its use in the then basic block. This is possible, since the branch condition does not depend on the load.

However, I'm a quite sceptical if instcombine would have the same effect on the original testcase, as the branch conditions generally depend on the loaded value.

[shubhamnarlawar77]

I think a simple reproducer is the following, there should be no need for the C/C++ sources (https://llvm.godbolt.org/z/Tf9hEY1EM):

%struct = type { i32, i32, i32 }

define i32 @bad(%struct* %ptr, i64 %idx, i1 %c) {
  %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
  %lv.1 = load i32, i32* %gep.1
  br i1 %c, label %then, label %else

then:
  %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
  %lv.2 = load i32, i32* %gep.2
  %res = add i32 %lv.1, %lv.2
  ret i32 %res

else:
  ret i32 0
}

define i32 @good(%struct* %ptr, i64 %idx, i1 %c) {
  %gep.1 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 1
  %lv.1 = load i32, i32* %gep.1
  %gep.2 = getelementptr %struct, %struct* %ptr, i64 %idx, i32 2
  %lv.2 = load i32, i32* %gep.2
  %res = add i32 %lv.1, %lv.2
  ret i32 %res
}

llc -mtriple=arm64-apple-darwinos produces:

_bad:                                   ; @bad
        tbz     w2, #0, LBB0_2
        mov     w8, #12
        mov     w9, #12
        madd    x8, x1, x8, x0
        madd    x9, x1, x9, x0
        ldr     w8, [x8, #4]
        ldr     w9, [x9, #8]
        add     w0, w8, w9
        ret
LBB0_2:
        mov     w0, wzr
        ret
_good:                                  ; @good
        mov     w8, #12
        madd    x8, x1, x8, x0
        ldp     w9, w8, [x8, #4]
        add     w0, w9, w8
        ret

Note the following sequence, where both madds compute the same values

        mov     w8, #12
        mov     w9, #12
        madd    x8, x1, x8, x0
        madd    x9, x1, x9, x0

If we pass -aarch64-enable-gep-opt=true -O3 to llc then this case is also being handled - https://clang.godbolt.org/z/hcjsh1vex

Also, using flags -aarch64-enable-gep-opt=true -O3 is as good as enabling SeparateConstOffsetFromGEPPass() at -O3 and -O2

[shubhamnarlawar77]

Bugzilla Link 51184
Version trunk
OS Windows NT
Attachments test case
CC @preames,@RKSimon,@shubhamnarlawar77,@sjoerdmeijer

Extended Description

In the attached test case there is this instruction

%h3 = getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, i64 0, i32 0

and 12-13 more similar instructions like

%h3.3 = getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom, i32 0, i64 3, i32 0

which begin with the same part getelementptr inbounds %struct.entry_t, %struct.entry_t* %2, i64 %idxprom

When the attached test case is compiled with

opt --passes="default<O3>" store.ll -S -o - | lcc

the output assembly file still contains multiple evaluations of that part, e.g.

mov	w15, #48
madd	x15, x9, x15, x8

... mov w13, #​48 madd x13, x9, x13, x8

etc.

Does anyone has small reproducer for this bug after using -aarch64-enable-gep-opt -O3? I am trying to reduce original testcase to replicate the problem at my end.

Using -aarch64-enable-gep-opt -O3 removes most of redundant GEP's but there is one redundant GEP still left. (It is probably phi node case)