Properly locate libomp.so when installed in clang target subdirectory.

[estewart08]

An upstream llvm change enables
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default for the openmp build. This installs the openmp libraries into
/opt/rocm-ver/llvm/lib/x86_64-unknown-linux-gnu instead of /opt/rocm-ver/llvm/lib. Currenty, hipBLAS only uses /lib.

Since hipBLAS uses gcc by default, find_package(OpenMP) will locate libgomp from the gnu installation. We would prefer to use libomp from the ROCm install. Use find_package(LLVM) to provide PATHS to find_library(omp).

On the other hand, the user can choose to use hipcc, which allows find_package(OpenMP) to properly locate libomp inside ROCm llvm.

Summary of proposed changes:
-If compiler is gcc, look for ROCm llvm to resolve libomp.so
-If compiler is hipcc, use find_package(OpenMP).
-Allow proper detection of new lib installation subdirectory x86_64-unknown-linux-gnu

[estewart08]

I think this is a better approach to #839.

[TorreZuk]

Okay if it passes seems good to me. @daineAMD you want to review as well

[TorreZuk]

Looks like CMake errors on LLVM, you can add some message status lines to help you debug

[estewart08]

It's complaining about not finding libffi and the C language not being enabled. But it looks like it grabbed the wrong llvm cmake config. I added a second commit. Do I need to retrigger the checks, or does that happen automatically?

[TorreZuk]

Every commit will trigger another build and test

[TorreZuk]

CMake Error at /opt/cmake-3.25.2/share/cmake-3.25/Modules/Internal/CheckSourceCompiles.cmake:44 (message):
check_source_compiles: C: needs to be enabled before use.

You can add C to LANGUAGES for the hipblas-clients project it should be fine.

[daineAMD]

CI compiles with g++ typically. There is a multi-compiler job that would test out hipcc. After some discussion with Sam it sounds like it was broken from the org. change, but it should start running now; so please wait until that passes.

If it doesn't get picked up, I opened PR #841 so you can specify the compiler CI uses once it's merged.

[estewart08]

@daineAMD Can you confirm whether or not these failures are related to this patch. The multi-compiler build has undefined references to the thread library, which I did not change.

[TorreZuk]

Those errors can be seen in develop branch but @estewart08 but that means your changes aren't getting tested either on that build.

[daineAMD]

Working out some problems with hipcc on CI. Tested with --compiler=hipcc with my local ROCm 6.0 and it passed tests, so fine to merge by me and I'll work on getting multi-compiler in a good state.