Inclusion Request: vulkan_lunarg + vulkan_lunarg_installer

[madebr]

Package Details

• Package Name: LunarG Vulkan SDK (runtime libraries + build tools) url
• Operating Systems+versions: Linux, Windows, macOS
• Compiler support: gcc-X clang-X xcode Visual Studio
• Conan recipe repository ies repository:
  • https://github.com/madebr/conan-lunarg_vulkan_sdk
  • https://github.com/madebr/conan-lunarg_vulkan_sdk_installer
  • => https://github.com/madebr/conan-vulkan_lunarg <=
• Version: 1.1.114.0 1.1.106.0

Inclusion request

• Fill all checkboxes with x if you agree with the statement or if it applies to your Conan recipe
• I want that my Conan recipe is getting included into the Bincrafters organisation
• I understand that if my request is accepted we are cloning your repository with all its history into Bincrafters, adapt some configurations and meta data if needed, and publishing it under the Bincrafters name and accounts
• I have used the Bincrafters templates to develop my Conan recipe - https://github.com/bincrafters/conan-templates
• My Conan recipe and all parts of it are licensed under the MIT license - https://choosealicense.com/licenses/mit/
  • Bincrafters are releasing all recipes under MIT license. If you want to have your recipe within Bincrafters we kindly ask you to do the same
  • it is okay if your test_package contains code from the upstream project under the upstream project's license, as long as it is clearly mentioned within the test_package files
• My Conan recipe covers (at least) the latest stable version which is available for the package
• I know that my Conan recipe is working on all operating system and compilers mentioned above right now
  • if not: please provide here links to your CI platforms and / or build logs and describe the problems you encounter
• I'm willing to help to maintain the recipe even after it is included into Bincrafters

Notes

• I suggest renaming the repositories to: conan-vulkan_lunarg and conan-vulkan_lunarg_installer (or similar)
• I have used the ConanMultiPackager because it is repackaging pre-built binaries.
• I have separated the sdk in a runtime element vulkan_lunarg and a build element vulkan_lunarg_installer. The build method is very similar, but not the same.
  Ideally, these 2 repos could be combined in one repo. But I do not find a good example. (see [discussion] Should we create new projects for installer packages #625) Installer and runtime conanfile are in the same repo.
• The download step is a bit more complicated because lunarg limits the number downloads per file per ip per day. It just caches the download. Also the downloads are 100-300MB which is a bit much when developing these recipes.
• bincrafters-conventions fails because I used the Various license type.
  The LICENSES file contains exctacts of the Apache, BSD, BSD-2, MIT and GPL-3 licenses (probably more). This is because lunarg's sdk uses multple projects.
• ci appveyor
• ci travis
• bintray: vulkan_lunarg
• bintray: vulkan_lunarg_installer

Checklist for the Bincrafters team

• Ignore this if you are not part of the Bincrafters team
• There is no such Conan package yet in conan-center, conan-community or Bincrafters
• [ The project is either popular or deserves to be maintained for another good reason (we can't package every single project out there)
• bincrafters-conventions reports no complains; the recipe fulfills our general quality requirements and conventions
• The recipe is cloned into our organisation; CI is enabled and configurated and succeeds
• There is a branch protection rule for stable/* branches, which requires a pull request with at least one approval first before things can get published
• Considering granding the inclusion request author push rights to the recipe repository (there is no guarantee that everyone is getting push rights, we decide this on a case-to-case base)

[Malacath92]

I'd really love to see this move forward. Is there anything I can do to speed it up or make it happen more likely?

[madebr]

@Malacath92 Do these packages work as intended for you?
@SSE4 @uilianries

[Malacath92]

@Malacath92 Do these packages work as intended for you?

I'm sorry, I have no experience with conan package development. Do you have a link to instructions for testing packages during development?
Either way, I could currently only test on Windows 10, could test on Ubunutu in a couple of days.

[Malacath92]

@Malacath92 Do these packages work as intended for you?

Alright, I used conan create to test both packages and they didn't give me any errors besides CMake warning when testing vulkan_lunarg:

CMake Warning:
    Manually-specified variables were not used by the project:
        CMAKE_EXPORT_NO_PACKAGE_REGISTRY

My setup is:

• Windows 10
• Visual Studio 2017 15.9.8
• CMake 3.12.4

Hope this helps

[madebr]

Thanks @Malacath92 .
I suppose this also means your project builds, links and runs fine using this vulkan lunarg runtime?

The warning you're mentioning is emitted by the test_package project which is ok.

[Malacath92]

Thanks @Malacath92 .
I suppose this also means your project builds, links and runs fine using this vulkan lunarg runtime?

Yeah exactly, that's what I meant. I had absolutely no issues building or linking and I shortly stepped through the program to see that the instance is created and destroyed as expected.

[Malacath92]

@madebr Running conan create . vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg on Ubuntu bionic 18.04, which is the same command I executed on Windows, I get the following error ERROR: vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg: 'settings.compiler' value not defined

This is emitted as part of the conan test . vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg step, to which I had to add -s compiler=gcc -s compiler.libcxx=libstdc++11 -s compiler.version=8, maybe you can provide some defaults?

But now it emitted

vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg: ERROR: Package 'b80d46004713aa37d6a90b42e2a326a056a237b5' build failed
vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg: WARN: Build folder /home/user/.conan/data/vulkan_lunarg/1.1.101.0/me/test_vulkan_lunarg/build/b80d46004713aa37d6a90b42e2a326a056a237b5
ERROR: vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg: Error in build() method, line 13
	cmake.configure()
	ConanException: Error 256 while executing cd '/home/user/.conan/data/vulkan_lunarg/1.1.101.0/me/test_vulkan_lunarg/build/b80d46004713aa37d6a90b42e2a326a056a237b5' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_EXPORTED="1" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="gcc" -DCONAN_COMPILER_VERSION="8" -DCONAN_CXX_FLAGS="-m64" -DCONAN_SHARED_LINKER_FLAGS="-m64" -DCONAN_C_FLAGS="-m64" -DCONAN_LIBCXX="libstdc++11" -DCMAKE_INSTALL_PREFIX="/home/user/.conan/data/vulkan_lunarg/1.1.101.0/me/test_vulkan_lunarg/package/b80d46004713aa37d6a90b42e2a326a056a237b5" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -Wno-dev '/home/user/.conan/data/vulkan_lunarg/1.1.101.0/me/test_vulkan_lunarg/build/b80d46004713aa37d6a90b42e2a326a056a237b5'

And I'm not sure where that fault comes from. Other projects that have handle dependencies through conan work fine on this machine. If I run conan install and cmake . by hand I get

CMake Error at CMakeLists.txt:13 (add_executable):
  Target "test_package" links to target "CONAN_PKG::vulkan_lunarg" but the
  target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?

Any idea what I could be doing wrong?

[madebr]

@Malacath92
What is your default profile? I assume default 😄
What is the output of conan profile show default?
Mine is: (fedora ships with gcc 8)

$ conan profile show default
Configuration for profile default:

[settings]
os=Linux
os_build=Linux
arch=x86_64
arch_build=x86_64
compiler=gcc
compiler.version=8
compiler.libcxx=libstdc++
build_type=Release
[options]
[build_requires]
[env]

[Malacath92]

@madebr

What is your default profile? I assume default smile

Alright, I modified my default profile to be similar to yours, but that doesn't change anything about the errors I get.
It still gives me
ERROR: vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg: Error in build() method, line 13

[madebr]

@Malacath92

If I run conan install and cmake . by hand I get ...

This fails because the test_package has an implicit dependency on the vulkan_lunarg conan package.
You can fix this locally and temporarily by adding an explicit requires = "vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg", to the body of the ConanFile class.

What is the complete output of conan test . vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg?
I don't know where the error appears right now. (it could be in either conan, the test recipe or the cmake script)

This looks like a problem with your conan install/configuration. Can you try:

• updating conan to the latest version ( 1.13.2 as of today)
• rm $HOME/.conan/settings.yml (unless you have modified it for your needs)
• mv $HOME/.conan $HOME/conan_original, conan user and conan create conan-vulkan_lunarg/vulkan_lunarg me/test_vulkan_lunarg

[Malacath92]

@madebr
Sorry for bothering you with this.
My conan install was completely fresh, so I don't suppose that was an issue. I reinstalled, which didn't help. I ran your suggested commands, which also didn't help.

The error I mentioned happens directly after cmake is done writing build files, though it also emits the Target "test_package" links to target "CONAN_PKG::vulkan_lunarg" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?
Not entirely sure if it said that before as well, though it would explain why building fails.

adding an explicit requires = "vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg", to the body of the ConanFile class

I'm sorry for the stupid question, but where exactly should I add this? I tried in several place inside test_package/conanfile.py, particularly implementing a build_requirements function, but to no avail.

[madebr]

@Malacath92
I meant adding it like this.

The error about CONAN_PKG::vulkan_lunarg means that you have run conan install on test_package. conan install does only install the explicit dependencies. (vulkan_lunarg is implicit here because it is a test_package, thus it HAS to have vulkan_lunarg as a dependency)

To make the dependency explicit, the first lines of vulkan_lunarg/test_package/conanfile.py should look like this:

...
class VulkanTestConan(ConanFile):
    settings = 'os', 'compiler', 'build_type', 'arch'
    generators = 'cmake'
    requires = 'vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg',

    def build(self):
...

Can you execute the following commands when inside the test_package folder after you the change above and post the exact output to a github gist or some pastebin service.
These commands assume you ran conan create on vulkan_lunarg before.

# cd vulkan_lunarg/test_package
rm -rf tmp && mkdir tmp && cd tmp
conan install ..
conan build ..

[Malacath92]

@madebr
I feel really stupid now. I cleaned my local git repo again and removed the current vulkan_lunarg packages. After that I ran conan create . vulkan_lunarg/1.1.101.0@me/test_vulkan_lunarg again, which succeeded, including building and running the test.
So I have to assume your suggestion that something was off with my installation or setup is right and I didn't properly clean up everything when reinstalling.
Sorry to waste your time on this with my silly issues. I hope the confirmation that this works on my distro is worth something at least.

[madebr]

@Malacath92 I'm happy to hear everything is working now.

@uilianries @SSE4
review? (Needs bincrafters/bincrafters-package-tools#21 to use bincrafters-package-tools python package)

[madebr]

Use default+installer hybrids

[Lawrencemm]

@madebr Assuming you have a project that depends on this package,

in you project, what is the way that you utilise the glslangvalidator to compile your shaders?
At the moment, in my project, I build shaders with cmake. Unfortunately this means that I have to copy the glslangvalidator executable into my cmake binary directory, add_executable(glslangvalidator) and then specify an imported location for it.

[madebr]

@Lawrencemm I forgot to reply to your question. Thanks for your patience.

I have split lunarg's sdk in a runtime and installer component:

• The installer (vulkan_lunarg_installer) contains the executables that should be run on your development machine, such as glslangValidator (please mind the capital V)
  It is meant to be added as a build requirement (added in the function build_requirements->build_requires)
• The runtime (vulkan_lunarg) contains all the files that should be run on the target machine (such as the vulkan-1.lib, vulkan-1.dll, libvulkan.so.1, ...). It also contains the header files.
  It is meant to be added as a requirement (added in the function requirements->requires)

The installer will add its binary directories to the environment PATH of consuming conan projects.
That way the glslangValidator executable should be runnable automatically by using e.g.
self.run("glslangValidator -h") in your conanfile.py or find_executable(GLSLANG_VALIDATOR "glslangValidator") in your CMakeLists.txt.
The test of the installer tests this:
https://github.com/madebr/conan-vulkan_lunarg/blob/35bdc4e2d6a25e27fdced53cb2881288cf0f6f53/test_package_installer/conanfile.py#L10

[madebr]

I've begun building vulkan from the khronos repo's:

• https://github.com/madebr/conan-vulkan_headers
• https://github.com/madebr/conan-vulkan_loader
• https://github.com/madebr/conan-glslang (contains glslang and glslang_installer)
• https://github.com/madebr/conan-spirv_cross (contains spirv_cross and spirv_cross_installer)
• https://github.com/madebr/conan-spirv_tools (contains spirv_tools and spirv_tools_installer)
• https://github.com/madebr/conan-spirv-headers
  @Croydon: I've updated the bincrafters' repo to 1.4.1: see https://github.com/madebr/conan-spirv-headers/tree/testing/1.4.1
• https://github.com/madebr/conan-vulkan_validation_layers
• https://github.com/madebr/conan-vulkan_tools

I'm working on the loaders and tools now.

I'm rate limited by jfrog's bintray, so I'm pausing this work for now

[Croydon]

@madebr I think that some recipes of those are already in CCI and that all recipes can be directly contributed into CCI, right?

If so, we can close this one :)

[madebr]

Indeed, I haven't worked on vulkan recipes since long time.