[RFC] "asm goto" vs branch target enforcement

The asm goto language extension lets you write inline assembly that branches to a specified set of labels within the containing function. Currently, LLVM disagrees with gcc on whether those branches are permitted to be indirect, and therefore, on whether an asm goto targeting a label means that a bti or endbr64 instruction (in AArch64 and x86-64 respectively) needs to be generated at the label, in compilation modes where those instructions are being used.

gcc doesn’t generate a bti at a label just because an asm goto mentions it as a possible destination. LLVM does.

This isn’t a one-sided question of safety (“the code might branch indirectly, so we must insert a bti to prevent a crash”). It’s a tradeoff. Every bti is a potential gadget for exploit code to try to use, so it’s a (quantitative) security improvement to keep the number of bti to a minimum. Also, bti isn’t completely free in performance terms: in sufficiently hot code, just having it there at all is a noticeable slowdown.

In particular, the Linux kernel uses asm goto to implement tracepoints in hot code: the asm generates a NOP, plus a data record in a separate section that points at the NOP and tells the kernel how to modify it into a branch instruction at run time. But the branch is always direct, so no bti is needed at its target label, and I’m told that some of the uses of this pattern are hot enough that adding one anyway costs noticeable performance (measured by benchmarking clang-built kernel code against gcc-built without the bti).

So it would be useful to have a way to avoid those bti being generated, for both security and performance. (Unusually, since those two are often opposed!)

Proposal

In the immediate short term: bring LLVM’s implementation of asm goto into line with gcc’s. A label in the list of targets of the asm goto should not force generation of a bti or endbr64 at the label. (One may still be generated for some other reason, of course, like if the label was also address-taken via && and used for a computed goto.)

In the longer term: consider whether it’s necessary to extend the asm goto syntax to provide a way for the user to specify that the code will make an indirect branch to a particular target label. If we decide it is necessary, come up with a syntax and semantics, and agree it with GNU. (Not necessarily for immediate implementation in both compilers, but at least arrange that the syntax is reserved for that use.)

Rationale

Why this way round, and not the other way round (keeping the default bti and adding some kind of a syntax option to suppress it)?

First: asm goto is a GNU language extension, so the de facto reference implementation is what gcc does. If they don’t emit bti in this situation then we should match them – that is, unless they can be persuaded to change their own default, and since a major use case is the Linux kernel which objects to the performance hit, that seems unlikely.

In particular, if anyone currently has code that does make an indirect branch to an asm goto destination label, that code is already at risk if it’s ever compiled with gcc instead of clang. Only source code that is 100% committed to clang (or to not using branch-target enforcement at all) is safe from this latent bug.

Second: if we do want to extend the syntax to provide both options (generate a bti or leave it out), it seems to me the natural approach is to add a system of constraints applying to labels.

Syntactically, you could (for example) replace a plain identifier in the labels section with a token-sequence such as "constraints" (label), similar to the syntax for inputs and outputs. (Of course you’d still have to permit a single identifier in place of that whole sequence, for backwards compatibility with existing source code.)

But in this case, the constraints should have the sense of constraints: each one should impose a requirement on the compiler’s code generation, rather than removing one. So it makes more sense to have a constraint saying “you must be prepared for an indirect call” than one saying “I promise not to indirectly call this label”.

Sounds reasonable to me. Most potentially interesting uses of indirect branches are not actually possible with asm goto anyhow, since we permit block-splitting. (Thus a given label won’t necessarily have the same address in multiple asm-goto in the same function, so passing an indirect target from one to the other can’t work).

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80053#c16 has my thoughts on this. And that bug report is the interaction between computed gotos and asm gotos even.

One thing to consider is what happens with frontends that only use LLVM, and not GCC, such as Rust?[1] Rust’s asm_goto tracking issue. Those frontends would theoretically have no issues using indirect branches in inline asm goto, but if this change is made, that’ll break.


  1. I know Rust has backends for Cranelift and GCC, but nearly everyone only uses the LLVM backend ↩︎

Interesting point about Rust. But at present I’m only concerned with how the asm goto syntax in C should behave, because that’s an input syntax that might be fed to either of clang or gcc and which they ought to agree about.

Perhaps we should more proactively add a configuration option to the asm operation in LLVM IR, because adding an option to LLVM IR is easier. Then anyone generating an IR asm with label arguments would be able to specify whether each label is (potentially) used in a way that requires a bti.

And then, if Rust decides that its asm goto language semantics require the ability to branch indirectly to a label, the Rust frontend can set that option in its output IR. Meanwhile clang leaves it unset (according to my proposal), and everyone’s happy.

1 Like

I’d recommend that we NOT implement such an option until/unless someone comes with an actual convincing use-case for what they need it for.

We do need to ensure we’ve properly documented the requirement to only use asm-goto labels in direct branches, both in Clang and LLVM IR documentations, however.