curl package update taps a new keg and installs 85 new dependencies?!

[jacktose]

I figured curl 7.73.0 -> 7.73.0_1 probably means a minor update, so imagine my surprise when brew upgrade curl started tapping linuxbrew/xorg and installing eighty-five dependencies!
I think the change is in this commit. Is that intended? Is it safe? Is there a way around it? I'll uninstall curl and build it myself before I quadruple the size of my linuxbrew installation.

$ brew upgrade curl
==> Upgrading 1 outdated package:
curl 7.73.0 -> 7.73.0_1
==> Upgrading curl 7.73.0 -> 7.73.0_1 
==> Tapping linuxbrew/xorg
Cloning into '/data/shared/lusers/jacktose/.linuxbrew/Library/Taps/linuxbrew/homebrew-xorg'...
remote: Enumerating objects: 129, done.
remote: Counting objects: 100% (129/129), done.
remote: Compressing objects: 100% (80/80), done.
remote: Total 7628 (delta 82), reused 70 (delta 49), pack-reused 7499
Receiving objects: 100% (7628/7628), 1.26 MiB | 16.38 MiB/s, done.
Resolving deltas: 100% (5917/5917), done.
Tapped 105 formulae (144 files, 1.7MB).
...
==> Installing dependencies for curl: gmp, mpfr, libmpc, binutils, [email protected], gcc, brotli, libunistring, libidn2, libmetalink, libssh2, cunit, [email protected], sphinx-doc, c-ares, jansson, jemalloc, libev, libevent, libxml2, nghttp2, texinfo, jpeg, libtiff, libpng, freetype, gperf, json-c, util-linux, fontconfig, libidn, ghostscript, cmake, util-macros, xtrans, libpthread-stubs, xorgproto, libxau, libxdmcp, libxcb, libx11, libxext, libxfixes, libxi, libxrender, libxrandr, libxxf86vm, ninja, meson, libxdamage, bison, help2man, flex, libedit, libelf, llvm, lm-sensors, libxshmfence, libxv, libxvmc, docbook, docbook-xsl, libxslt, libpciaccess, libdrm, linuxbrew/xorg/libomxil-bellagio, linuxbrew/xorg/wayland, linuxbrew/xorg/libva-internal, linuxbrew/xorg/libvdpau, linuxbrew/xorg/wayland-protocols, mesa, mesa-glu, libxinerama, xinput, freeglut, jasper, netpbm, psutils, uchardet, pcre, glib, groff, openldap, rtmpdump and zstd

[gromgit]

==> Installing dependencies for curl: gmp, mpfr, libmpc, binutils, [email protected], gcc, brotli, libunistring, libidn2, libmetalink, libssh2, cunit, [email protected], sphinx-doc, c-ares, jansson, jemalloc, libev, libevent, libxml2, nghttp2, texinfo, jpeg, libtiff, libpng, freetype, gperf, json-c, util-linux, fontconfig, libidn, ghostscript, cmake, util-macros, xtrans, libpthread-stubs, xorgproto, libxau, libxdmcp, libxcb, libx11, libxext, libxfixes, libxi, libxrender, libxrandr, libxxf86vm, ninja, meson, libxdamage, bison, help2man, flex, libedit, libelf, llvm, lm-sensors, libxshmfence, libxv, libxvmc, docbook, docbook-xsl, libxslt, libpciaccess, libdrm, linuxbrew/xorg/libomxil-bellagio, linuxbrew/xorg/wayland, linuxbrew/xorg/libva-internal, linuxbrew/xorg/libvdpau, linuxbrew/xorg/wayland-protocols, mesa, mesa-glu, libxinerama, xinput, freeglut, jasper, netpbm, psutils, uchardet, pcre, glib, groff, openldap, rtmpdump and zstd

It looks like curl is preparing to build from source, which shouldn't happen, since there are bottles for everything.

What do the following say?

• brew info curl
• brew config
• brew doctor

[jacktose]

@SMillerDev I wish I could, but the whole reason I use linuxbrew is that I only have access to my userspace on this system.

I'm aware of that warning message, though I will note that official docs seem to be of two minds on the supportedness, including statements like

Just extract (or git clone) Homebrew wherever you want.
Pick another prefix at your peril!
Create a Homebrew installation wherever you extract the tarball.
Can install software to your home directory and so does not require sudo

And that it was placed here automatically by the official installation script:

The installation script installs Homebrew to /home/linuxbrew/.linuxbrew using sudo if possible and in your home directory at ~/.linuxbrew otherwise.

In any case, I'm happy to do the troubleshooting legwork, but I'm not an expert and need someone to point me in the right direction. If this is an inappropriate venue for that, please just tell me where to go or whom to ask.

[gromgit]

Manually tapping linuxbrew/xorg fixed that error. Is something borked with the auto-tapping for dependencies?

The error references libcurl.so.4, and you're upgrading curl, so I'd say it was probably a chicken-n-egg problem.

But obviously it's still installing a huge pile of deps that it didn't seem to need for the previous version.

That's a by-product of Homebrew's design as an evergreen CI-tested package distribution. Each formula is always tested against the latest version of each dependency, so the web of interlocking formulae and versions is ever-shifting. Installing/upgrading a specific formula at a specific time requires installing/upgrading specific versions of other formulae at the same time, so if a formula's dependency tree happens to hit a recently-updated "foundation" formula like python, the "working set" can get very large indeed.

[jacktose]

Okay, so we're back to my original question: is this change to curl's dependencies Good?

Here's the commit. It adds nine new (direct) dependencies to a package that previously had one (two on Linux), apparently to add a richer feature set to the build. But now curl, a very basic utility, requires a whole other tap and 85 (total) other packages, including various graphics libraries, all of python, etc. (BTW, one of those is failing for me, so I still haven't managed to upgrade.)
Let's also consider that curl is required by a lot of brewy things, and so has a large user base, close to half a million installs in a year. Hopefully most of those are bottles, but this change is going to make a large difference to a significant fraction of brew users' installations.

IANA maintainer and I don't know how these decisions are made (and I don't see any comments on the commit or PR), but I hope somebody can tell me I'm not crazy: it's intentional and justified, or it's an unintended consequence, or there's a simpler alternative now, or ???

Thanks for walking me through this stuff.

[gromgit]

is this change to curl's dependencies Good?

It depends. In your case, several factors would tend to increase the number of involved formulae on average:

• A non-standard HOMEBREW_PREFIX: some of the formulae you see may actually be build dependencies, because some other formula involved can't be poured from a bottle, and/or were originally built from source for whatever reason
• Linuxbrew: this evolved semi-independently from macOS Homebrew, and design decisions that were sensible in the early days (e.g. the linuxbrew/xorg tap) may now be somewhat problematic, especially w.r.t. closer integration with its parent
• General upgrade frequency: brew always tries very hard to ensure that any changes to be made leave your Homebrew in a consistent state, so every install/upgrade is transitive by nature, i.e. it expands as necessary to cover the minimum consistent set of tested versions. I'm not saying you've been falling behind on your brew upgrades, but homebrew/core is an extremely active tap, and lots of software depends on lots of other software.

it's intentional and justified, or it's an unintended consequence, or there's a simpler alternative now, or ???

Not a maintainer, but from all the comms I've seen from maintainers over time, I get the impression that it's the inevitable consequence of the following tenets:

• Disk space is cheap, while time spent figuring out which one of a group of similarly-named packages (each with its own subset of capabilities) you need for a specific situation is neither cheap nor recoverable --> "full-fat, kitchen-sink" packages tend to be preferred
• Homebrew shall be user-driven, so anyone can add (within reason) whatever software they deem fit for everyone's benefit, and/or whatever capabilities are added by the original software authors over time --> maintainers usually cull submissions for quality rather than quantity (in both packages and features)
• Homebrew software shall be kept up-to-date (within reason) --> "livecheck" detection of new releases, ASAP version bumps, and a general prohibition against Homebrew-specific patches for long-term maintainability
• Homebrew software shall be tested (within reason), not just by itself, but also against everything it depends on, every time any of the formulae involved is version-bumped --> formulae are tightly coupled in an interlocking "version web"
• Open-source software interdependence increases monotonically --> formulae that were previously independent are suddenly linked, when one side decides to ride on the other's efforts, either to replace their own implementation or add new features

[jacktose]

Okay, I see how all the principles lead us here. But when the emergent effect is a tendency to make homebrew a bloat factory, there's got to be some principle to push back on that too.

This reminds me of the recent debate about installation options, which makes me think there isn't a way to do a lean install either?

[jacktose]

To put the point a different way: stats say 500,820 people have installed curl this year, and 156 have bothered to add additional features. That's a rounding error! I would think there'd have to be a good reason to make big changes to the formula for things relevant to 0.038% of the users, and I don't see that reasoning anywhere.

[iMichka]

Should we ask the curl maintainers to stop pushing updates?

[jonchang]

Now that Homebrew/homebrew-core#67412 is merged, once that appears in linuxbrew/core users who install Homebrew to a nonstandard prefix should no longer need to build curl from source to get a working installation.

On a broader note, opening discussions like these are not helpful. We are aware of these issues; what's needed most is not more complaints, but people to actually do the work and figure out how to fix these problems. I should note that I figured out this fix while trying to help a contributor merge a pull request to fix an unrelated curl issue. Getting patches is encouraging, getting hyperbolic complaints like this one is demotivating.