Offloading on nvptx64 target with flang-new leads to `Undefined reference`s

Subprojects Flang
,
1

Hi everyone,

I’ve been trying to test offloading with flang-new by compiling a simple hello world.

PROGRAM Parallel_Hello_World
USE OMP_LIB

!$OMP PARALLEL

    PRINT *, "Hello from process: ", OMP_GET_THREAD_NUM()

!$OMP END PARALLEL

END

When compiling without offloading

flang-new -fopenmp test.f90 -o a.out

there is no problem, but when turning on offloading with -fopenmp-targets=nvptx64, it results in many Undefined reference during the call to nvlink.
The missing symbols all looks to be a mangled version of openmp ones (eg _QMomp_libEComp_allocator_handle_kind).

I tried to inspect the compilation steps with -v and the symbols are already present in the temporary BC file after the first compilation step

$ llvm-bcanalyzer --dump /tmp/test-9ded23.bc | less
...
<STRTAB_BLOCK NumWords=714 BlockCodeSize=3>
  <BLOB abbrevid=4/> blob data = '_QQclX8588ff86aeb59aac301a91fa2adffb54_QQclX48656C6C6F2066726F6D2070726F636573733A20_QMomp_libEComp_allocator_handle_kind_QMomp_libEComp_alloctrait_key_kind_QMomp_libEComp_alloctrait_val_kind...

while the mangling is not present in the object file produced by the first step of the compilation without offloading (exactly the same command with -emit-obj instead of -emit-llvm-bc)

Offloading a similar HW.ccp program with clang seems to be working fine.

I was wondering if this is not supported yet, or if I am missing some crucial options or flags during the compilation.

As a reference, during testing i did get a few failed libomptarget tests:

  libomptarget :: nvptx64-nvidia-cuda :: jit/empty_kernel_lvl1.c
  libomptarget :: nvptx64-nvidia-cuda :: jit/empty_kernel_lvl2.c
  libomptarget :: nvptx64-nvidia-cuda :: jit/type_punning.c
  libomptarget :: nvptx64-nvidia-cuda :: offloading/ctor_dtor.cpp
  libomptarget :: nvptx64-nvidia-cuda-LTO :: jit/empty_kernel_lvl1.c
  libomptarget :: nvptx64-nvidia-cuda-LTO :: jit/empty_kernel_lvl2.c
  libomptarget :: nvptx64-nvidia-cuda-LTO :: jit/type_punning.c
  libomptarget :: nvptx64-nvidia-cuda-LTO :: offloading/ctor_dtor.cpp

In case it could be of use this is the result of ldd flang-new

linux-vdso.so.1 (0x00007ffe82f4c000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa70b308000)
libclang-cpp.so.18.1 => /home/crivella/.local/easybuild/software/LLVMcore/18.1.7-offload/lib/libclang-cpp.so.18.1 (0x00007fa7072f3000)
libLLVM.so.18.1 => /home/crivella/.local/easybuild/software/LLVMcore/18.1.7-offload/lib/libLLVM.so.18.1 (0x00007fa7033fe000)
libc++.so.1 => /home/crivella/.local/easybuild/software/LLVMcore/18.1.7-offload/lib/x86_64-unknown-linux-gnu/libc++.so.1 (0x00007fa70f43d000)
libc++abi.so.1 => /home/crivella/.local/easybuild/software/LLVMcore/18.1.7-offload/lib/x86_64-unknown-linux-gnu/libc++abi.so.1 (0x00007fa70f3f6000)
libunwind.so.1 => /home/crivella/.local/easybuild/software/LLVMcore/18.1.7-offload/lib/x86_64-unknown-linux-gnu/libunwind.so.1 (0x00007fa70f3e6000)
libm.so.6 => /lib64/libm.so.6 (0x00007fa70307c000)
libc.so.6 => /lib64/libc.so.6 (0x00007fa702cb7000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa70f31d000)
librt.so.1 => /lib64/librt.so.1 (0x00007fa702aaf000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fa7028ab000)
libz.so.1 => /home/crivella/.local/easybuild/software/zlib/1.3.1/lib/libz.so.1 (0x00007fa70268f000)
libzstd.so.1 => /lib64/libzstd.so.1 (0x00007fa7023bc000)
libncurses.so.6 => /home/crivella/.local/easybuild/software/ncurses/6.5/lib/libncurses.so.6 (0x00007fa70215f000)
libatomic.so.1 => /lib64/libatomic.so.1 (0x00007fa701f57000)
2

Hi,
I added patch for searching mlink-builtin-bitcode files for AMD GPU ( [Flang-new][OpenMP] Add bitcode files for AMD GPU OpenMP by DominikAdamski · Pull Request #96742 · llvm/llvm-project · GitHub ). My patch searches for required AMD GPU bitcode files and adds AMD GPU specific target options.

You can similar try similar patch for Nvidia GPUs:

//file: clang/lib/Driver/ToolChains/Flang.cpp
void Flang::addTargetOptions(const ArgList &Args,
                             ArgStringList &CmdArgs) const {
/* ... */
switch (TC.getArch()) {
+case llvm::Triple::nvptx:
+case llvm::Triple::nvptx64:
+    TC.addClangTargetOptions(Args, CmdArgs, Action::OffloadKind::OFK_OpenMP);

Toolchain for Nividia GPUs can add some flags which are not supported by flang-new (see file: clang/include/clang/Driver/Options.td ) .