How to force vpckg to not cache a specific dependency

[dylanclark]

I'm using vcpkg in manifest mode to manage a set of dependencies.

For one of the dependencies, I actively work on the repo with the dependency code. I'd like to develop both repos side by side. As such, I have a custom portfile written. When I want a tight inner loop while making changes in both repositories, I edit the portfile for the dependency to remove the call to 'vcpkg_from_git' and manually set the SOURCE_PATH to my local clone of the repository I'm depending on.

I'd like to make changes in the the repository I depend on and have them apply every time I run 'vcpkg install'. What I find is that without a manual change to the portfile, vcpkg can't detect if the source code I'm using changed and so won't rebuild it every time.

Is there a flag or strategy I can use to get vcpkg to rebuild this one dependency every time without having to edit the portfile or delete the built artifacts? Or perhaps what I'm doing is an anti-pattern and I should consider another approach? I don't want to have to rebuild every dependency every time. Just the one.

Thanks in advance for suggestions!

[autoantwort]

If you create 100 files in your portfile dir the binary cache will not be used for that port, but that is not a behavior I would rely on :D

[dylanclark]

Ha, I do not want to rely on this but I will file this knowledge away, I'm sure it will come in handy at some point.

Is the magic number really 100? I wonder why that is?

[autoantwort]

There is now microsoft/vcpkg-tool#1104 to remove this behavior

        // If there is an unusually large number of files in the port then
        // something suspicious is going on.  Rather than hash all of them
        // just mark the port as no-hash
        constexpr int max_port_file_count = 100;

[dg0yt]

Did you try to redirect this port to an overlay port? Instead of repeatedly touching the portfile, I would use another file to hold nothing but the timestamp (as content, not as metadata). It would still trigger the rebuild because the content takes part in ABI hashing. This can be automated from the main project.
However, rebuilds only take place when vcpkg is actually run again.

[dylanclark]

This is an interesting idea, could you please elaborate a bit? I have a few custom ports already, and I pass the directory that contains them to vcpkg via the --overlay-ports command line argument. The portfile in question here is one of those ports.

Are you suggesting having the build system generate a file with the timestamp, and including that in my portfile? That would be sufficient to invalidate the cache and force a rebuild of that target the next time vcpkg runs?

[dg0yt]

"including" isn't needed. vcpkg takes the hashes of the files in the port directory (up to the magical number).

[dylanclark]

Ah, I see, I didn't realize it took the hashes of all files in the directory. This worked great! Thanks.

I ended up adding a 'timestamp.txt' file to the directory that gets written by the portfile.

# File contents of timestamp.txt.in
Built at @CURRENT_TIME@

And then the following in the portfile that writes the timestamp file for each build.

    #  In portfile.cmake, uncommented when I want to force the dependency to get rebuilt everytime vcpkg runs
    string(TIMESTAMP CURRENT_TIME UTC)
    configure_file(
        ${CMAKE_CURRENT_LIST_DIR}/timestamp.txt.in
        ${CMAKE_CURRENT_LIST_DIR}/timestamp.txt
        @ONLY)

[dylanclark]

Ended up using @dg0yt's suggestion to write a file with the timestamp into the portfile directory to force vcpkg to rebuild it each time.

[RealLitb]

Here's a solution that doesn't require changing the port directory:

ABI_INFLUENCER=$RANDOM ./vcpkg install zlib:x64-linux-dynamic-tricks

The environment variable can be set to anything per-build like your CURRENT_TIME value. I think it should work if you set it within your CMakeLists, as long as it's before the project() call, but haven't tested. Then in the triplet you can say

if(PORT STREQUAL "zlib")
   list(APPEND VCPKG_ENV_PASSTHROUGH ABI_INFLUENCER)
endif()

[dg0yt]

This requires changing the triplet (once), i.e. decouples the triplet file from vcpkg and the cached artifacts from other uses.

[RealLitb]

Certainly! It's a tradeoff with pros and cons! Your solution of changing the port directory will allow installing different ports on different vcpkg runs and afterwards reuse the cache, provided that the file in the port won't anymore chance. My solution can also reuse the cache, but will require the shared environment variable not to change, which is the same for all ports. So all ports must be installed with the same value!