Replies: 12 comments 16 replies
-
|
Hi psyinf?
In the last few months I started to use conan <https://conan.io> quite
intensively to manage my dependencies. Since it easily integrates with
CMake I wonder if the vsg community is interested to make vsg available
here <https://conan.io/center>. Having some minor experience with the
process of creating a conan package I'd sort of volunteer to make one for
the vsg 1.0.0 release.
There are some open questions however:
- we need to specify the minimum Vulkan version
The VSG is design/implement to be backwards compatible back to Vulkan 1.1.
It's 1.2 when Vulkan really gets in its stride. Driver availability would
be the big question.
Does conan have Vulkan as part of the suite of dependencies it can pull
in? If so what is the Vulkan and glslLang version range?
- What build options, apart from static/dynamic linking must be
available
glslLang or no glslLang would be the big one. For most desktop apps I
think shader compiling and composition is useful so would expect glsgLang
There is also the ability to set the VSG_MAX_DEVICES to a higher number
than the default of 1. Leaving it at 1 is probably OK for 99% of users.
Multi-device support is only really something I've discussed within the
community. Perhaps as time goes on this will become more important, but
frankly if you want to use multiple devices then you are power user so
should be compiling the VSG from scratch anyway.
- An initial version would not include any of the companion projects
vsgXchange is pretty important for any desktop application development so
if the VSG is included vsgXchange needs to be distributed as well. This
then brings up the permulatios on the various dependencies that it supports.
What are your thoughts on this?
Death by permutations :-)
Cheers,
Rpbert/
… Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Robert, Sorry for the alias-name and not introducing properly. Actually my name is Sebastian Messerschmidt, and it might ring a bell, since I've been part of the OSG community for a couple of years.
That's one item I was going to explore. If I got this correctly, the base Vulkan SDK basically is header only. Correct me If I'm wrong here.
Since those are optional, they could be added as an optional dependency, so users could compose the set of libraries they see fit.
When using conan one would usually add a dependency and some options. This would pull in the dependencies and If available for the requested platform/options/permutation the package usually is pulled as is. If not available, conan provides a build script and will build the dependent packages. So basically conan is exactly built to cover permutation hell introduced by C++'s ABI/compiler mismatches. If you feel this could be a valuable contribution to vsg, I'll invest at least some initial time to test it locally. It surely helped me in my previous projects to have OSG available at conan-center. Cheers |
Beta Was this translation helpful? Give feedback.
-
|
Hi Sebastian,
Conan being able to compile combinations that don't have binaries is
certainly helpful.
Would one normally put conan files into a repo? Or can this be part fo the
Cmake build?
I am planning on jumping back over to the OSG side to make stable release,
this I will have to juggle with client and VSG work, but one of things I'd
like to do, where possible, is to converge how both projects are managed at
C++, CMake and github level.
I don't know how much we'll be able to share, but I'd like to avoid having
to keep track of two different approaches as well as two different code
bases. I mention this as there may be opportunities on the packaging side
as well. I expect the OSG/VSG communities would find familiarity helpful
too.
Cheers,
Robert.
Message ID: <vsg-dev/VulkanSceneGraph/repo-discussions/590/comments/4184663@
github.com>
… |
Beta Was this translation helpful? Give feedback.
-
|
Hi, about the conan "files". In order to have the package available, one need to create a python configuration that follows the template I've linked in the initial post. Since my first attempts with conan only used a header-only library I need to figure out if there are any changes required to the CMake infrastructure in vsg. But since this wasn't really needed for OSG AFAIR I don't expect problems here. With your consent, I'd try to establish an initial version at conan-center and we iterate from there. Cheers |
Beta Was this translation helpful? Give feedback.
-
|
HI Sebastian,
Is it possible to trial packaging before it goes mainstream/officially part
of conon?
I am thinking it might be useful for me to try it out to better understand
the process and what end users will experience.
Cheers,
Robert.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Robert, Cheers, Sebastian |
Beta Was this translation helpful? Give feedback.
-
|
I've finished the first conan package descriptor (featuring the options to build static/dynamic library for all build types). Next steps are to add the glslang package as an option. The fork can be found here. For testing the local test guide can be used. TLDR; After building the package locally, it can be used like a package remotely available on conan-center. |
Beta Was this translation helpful? Give feedback.
-
|
Not much to add here other than Conan absolutely slaps. Glad to see this initiative. |
Beta Was this translation helpful? Give feedback.
-
|
I've stepped forward publishing/merge-requesting the VSG-conan package for 1.0.0. This is rather unfortunate, since it really is orthogonal to what conan tries to solve: offering packages with adjacent dependencies. As of now this changes the game, as it would require changes to the CMake files of VSG at least to keep it compatible with the conan-approach. I think a rather minimal approach would be to use the CMake tooling to check if a certain version of a dependency is available via CMake itself before actually "falling back" by checking out the submodule. Any thoughts? |
Beta Was this translation helpful? Give feedback.
-
|
Could you explain what specific problem there is as it should just be a case of removing an external dependency from the VSG build. Building glslang within the VSG itself has solved a number of problems, including crashes in user application due to inconsistently SPIRV-Tools/glslang dll's. Having glslang as a external dependency was always a constant pain due to the way it's so packaged on different platforms. Adding a special case for conan build I think would be a backwards step, the whole point of building it locally is to avoid the pain that it caused and ensuring that we have a consistent build across all platforms. It might not be the "conan" way but it works. Personally I'd suggest just simplifying the rules for the VSG build and standardizing on 1.0.3 onwards. |
Beta Was this translation helpful? Give feedback.
-
|
Digging deeper into the challenge I might have found a non-intrusive option here. Conan solves such problems by offering a mechanism to patch the sources, including the CMake files before internally building it. Currently I'm trying this. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Robert, I've tackled the issues for now by adding a checkout-step for the submodules in question.
All after all I'm now waiting for the pull request being reviewed. I personally consider VSG being available via conan a big step forward to make it more accessible to C++ developers. It also seems that this doesn't require any careful considerations from your side, as conan seems to built explicitly to be non-intrusive, offering possibilities to patch sources on the fly and harness the power of scripting to get around most of the issues I've encountered. You might want to take a look: |
Beta Was this translation helpful? Give feedback.
-
|
VSG_SUPPORTS_ShaderCompiler = true is the default configuration I've chosen for the default built of the VulkanSceneGraph from source so would recommend following this for the conan. Shader compilation make the VSG much more convenient to use as it enable easy runtime shader composition and simplifies the process of developing applications. VSG_SUPPORTS_ShaderCompiler = false is something I would expect that will only be need by small portion of the VSG community that require minimal code sizes and will be exclusively using pre-compiled SPIR-V shaders. If you have a embedded application with minimal resources available then this might be of interest. BUILD_SHARED_LIBS is something users might be more likely to want to toggle. The VSG_MAX_DEVICES may also be something users might want to adjust from the default of 1. Using multiple GPUs on one system might be quite niche though. As the VSG project provide multiple libraries that a high % of users will want to use like vsgXchange, and also may want to pull in vsgImGui, vsgQt, vsgCs etc. then there may be tension in using conan for a sub-portion of these, and the missing ones compiled by source. The answer to this would be long term for conan support for these ancillary projects. At a personal level I see conan as one of many 3rd party packaging options, while it's great to see it made possible with the VSG my focus will remain on trying to get the VSG to work well out of the box when compiled from source and to minimize friction points that developers compiling from source, or for a 3rd part package system. |
Beta Was this translation helpful? Give feedback.
-
|
@psyinf in the VulkanSceneGraph master, the VulkanSceneGrasph-1.0 branch and the v1.0.5-rc2 tag I have just made the glslang component is now cloned from CMake rather than using a submodule. I have had to make this change as the GitHub release system doesn't package submodules so you end up with broken tarball. Changing to using CMake to clone the glslang repo when required resolves this issue as the gslang repo is cloned automatically on the first CMake invocation. If git isn't found or there is an issue with cloning the glslang repo the VSG_SUPPORTS_ShaderCompiler cmake var is automatically set to 0 to disable shader compilation support. This automatic fallback will mean the VSG should build even if you don't have git or path to it, or there is no internet access, it won't have full capabilities w.r.t shader compilation. I decided this is marginally better than just breaking the build requiring end user action to resolve the issue as I think users first experience of the VSG should be as seamless as we can get it. |
Beta Was this translation helpful? Give feedback.
-
|
@robertosfield Thanks for keeping me in the loop here. I've changed the still-pending pull request here to incorporate your changes. Since git is prerequisite for conan AFAIK, I took the liberty to model this without the preliminary check. I'm quite positive the pull-request will be merged any time soon (there was a lot of work for the reviewers due to conan 2.0 transition work). I also started preparing vsgXchange and vsgImGui to be integrated with conan (as I need them for my upcoming projects anyways). From there it should be pretty trivial to add the remaining repositories if needed by anyone. Long term goal is to have VSG fully available. Cheers |
Beta Was this translation helpful? Give feedback.
-
|
@robertosfield Fortunately I finally found some time to drive the conan packages here a bit closer to being accepted. There is one caveat for now: I had to drop the glslang support, as conan more or less dictates dependencies to be part of conan. Therefore simply checking out glslang etc simply doesn't work. In order to circumvent this I need to use glslang as a dependencies. Unfortunately the conan-package doesn't seem to export the public-headers (why on earth?). So I'd resort to create a vsg-glslang package here. For this I need exactly one thing from you: A tag/release of the VSG-1.0.x branch. Could you please provide this? Cheers |
Beta Was this translation helpful? Give feedback.
-
|
What is the issue with checking out glslang? Is there something we can do to help with how we check out glslang? What do you specifically mean by a tag/release of the VSG-1.0.x branch? There is the VulkanSceneGraph-1.0 branch and v1.0.x tags, what is missing from these? |
Beta Was this translation helpful? Give feedback.
-
|
@robertosfield The issue is that conan is relatively strict about how to handle "dependecies". It relies on using conan-packages for external code and such. P.S. does the vsg-built glslang drop-in make use of the SPIRV optimizer? Cheers |
Beta Was this translation helpful? Give feedback.
-
|
So you need a tag of vsg-dev/glslang? I presume a VSG release would also need reference this explicitly, |
Beta Was this translation helpful? Give feedback.
-
|
Exactly, that would be nice. You don't explicitly need to reference this from the VSG but it would sure help to keep things consistent. Cheers |
Beta Was this translation helpful? Give feedback.
-
|
Will this do? |
Beta Was this translation helpful? Give feedback.
-
|
This will do fine. Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
@robertosfield The initial [PR|https://github.com/conan-io/conan-center-index/pull/16528#event-9667944268] has been merged into the conan main branch and vsg should be available as a package soon. If you don't mind I'd prepare an example on how to get started with conan and VSG. Personally I think having VSG available in different package/dependency-managers will help to proliferate the library thus increasing the the number of people that might contribute. Next steps I'm planning as soon as I find some time:
|
Beta Was this translation helpful? Give feedback.
-
|
Just to add to this discussion. Our company is heavily invested in conan/artifactory. It is integral to our CI build process. |
Beta Was this translation helpful? Give feedback.
-
|
I stopped working on this, since the conan integration process is rather slow and patching the vsg cmake and sources to work around git-checkouts via cmake is rather tedious. Since VSG is split up into multiple parts, each time a new version is released one needs to bump up all those subprojects, make sure it builds with the submodules etc. I'd be willing to put some effort into this again, but that would need some support on the VSG-side and I won't be able to do this with priority. |
Beta Was this translation helpful? Give feedback.
-
|
Believe me, I know how much effort you would have put into this. It is appreciated! I have had to develop the equivalent for packages that are not on conan.io so they can be used in our "main" conanfile.py. Often having to patch CMakefiles.txt and source code in the package conanfile.py I can see how unless VSG is restructured to be conan friendly then it just becomes too much ongoing work. |
Beta Was this translation helpful? Give feedback.
-
|
I have too much work to manage with core VSG tasks to take on tasks like conan support and other 3rd party packaging schemes so have to rely upon the community for help. Where it's possible I'm happy to see changes to the CMake to make it easier for these 3rd party packaging schemes but it has to be clean and straight forward to maintain, if it looks like one of these packaging schemes put too much strain on VSG's existing CMake support then I'll not merge them, it's already a challenge to keep things building in all the different platform/build combinations that VSG users come up with - it's all too easy to fix things one set of users but break it for another set. So if you want to propose changes let me know I can make suggestions of whether that looks viable for long term support. As a general comment, in recent VSG-1.1.x developer releases glslang has been made an external dependency, this was meant to resolve build issues for end users, but it's actually caused it's own problems as glslang is so inconsistently distributed by 3rd party packaging, this might also be an issue for conan unless others have shaken down the problems with glslang. For myself, my priority is making CMake build itself easy for users to integrate it directly. |
Beta Was this translation helpful? Give feedback.
-
|
Since the removal of the bundled glslang, the vcpkg package definition is super simple (see https://github.com/microsoft/vcpkg/blob/master/ports/vsg/portfile.cmake) so the same should be true of an updated Conan package. In principle, everything should Just Work™ without any fuss. |
Beta Was this translation helpful? Give feedback.
-
Hi,
In the last few months I started to use conan quite intensively to manage my dependencies. Since it easily integrates with CMake I wonder if the vsg community is interested to make vsg available here. Having some minor experience with the process of creating a conan package I'd sort of volunteer to make one for the vsg 1.0.0 release.
I've already contributed some header-only math library so I at least know the process and might be able to provide the initial work required for this. How to contribute is described here
I'd also document some key aspects on how to maintain/create packages for future releases.
There are some open questions however:
What are your thoughts on this?
Beta Was this translation helpful? Give feedback.
All reactions