Decide on a path forward for the project #110
Comments
|
I certainly have neglected this suite of packages. I hate that it has happened but I switched jobs and no longer work with Qt. Combined with this always having been a painful project to maintain... here we are. If I do, I will probably start with reducing the supported matrix as much as possible to simplify the CI and build needs. The number of jobs spread across four layers of projects (pyqt-tools uses pyqt-plugins uses qt-tools uses qt-applications) is... big. I had been supporting many Python versions and many Qt versions across all platforms. I would expect to drop supporting older Qt versions, maybe just latest Qt5 and Qt6, or maybe even drop Qt5 entirely. Given the size of the qt-applications releases, there was also complexity around properly building new python and platform support for existing releases to avoid re-releasing the same data for already supported platforms and Python versions under a new version. Note the four layers of this project created to isolate the slow builds of the Qt application package and the PyQt plugins from the pure python Qt tools and PyQt tools layers. The separation also allows for isolation of releases of wrapper features from including duplicates of the big Qt applications files. Note that at one point we were in the top 40 packages on PyPI for disk usage... https://github.com/altendky/pyqt-tools If other developers are interested in maintenance, I would have the normal author hesitation to give up control... You have already proposed a few possible changes here, which is inline with my suggestion in the previous paragraph.
I think a PR describing the state of the project in the readme is a great idea. It could say 'unmaintained', or 'unmaintained with discussions around fixing that situation', or...
Is there a maintained fork? I haven't been looking around.
I think the issue might be less the upstream pinned versions and more the lack of builds of This doesn't fix the situation, but do note that the PyQt and Qt versions uses can be set when building via environment variables. This is how CI does it. pyqt-tools/.github/workflows/ci.yml Lines 131 to 133 in 2583303 env:
QT_VERSION: ${{ matrix.qt.qt_version }}
PYQT_VERSION: ${{ matrix.qt.pyqt_version }} env:
QT_VERSION: ${{ matrix.qt.qt_version }}
PYQT_VERSION: ${{ matrix.qt.pyqt_version }} env:
QT_VERSION: ${{ matrix.qt.env }} env:
QT_VERSION: ${{ matrix.qt.env }}Though yeah, there's a little more to go with that like the Lines 90 to 95 in 2583303 Welp, there's a first pass anyways at 1) acknowledging the badness of the present situation, 2) agreeing with your interest in having other folks involved, and 3) laying out some of the reasons, hassles, and mechanisms around supporting this and keeping it updated. |
|
Wow, thanks for such a detailed, thoughtful and comprehensive response! And hope it didn't take up too much of your time. Just for some context, the reason I brought this up was that we had users asking about it on the Python Discourse, and I'd looked into it a bit further, particularly since I'm one of the maintainers of QtPy. However, unfortunately I am not a user this package myself nor am I actually a Qt expert (nor do I really use it much outside the context of Spyder)—I just help maintain things where I can (and something like this lies pretty far outside the realm of my expertise, just to be clear). The reason I reached out was mostly because I recognized your name from interacting with you previously as a QtPy contributor (and we were eventually able to merge your PR, BTW, I just had to take some time off myself for other reasons and came back to it later), and figured I'd send you a ping on this on behalf of the various users asking.
Yeah, it's definitely tough for sure. Even just with QtPy, which itself is pure-Python, it's a lot with 2 major Qt versions (5, 6) x 3 Qt minor versions (5.9, 5.12, 5.15 + 6.2, 6.3, 6.4) x 2 bindings (PyQt, PySide) x 2 Python versions (3.7, 3.11) x 3 platforms (Mac, Linux, Windows). It's still not perfect but I've refactored and simplified our CI scripts and workflows to keep most of the matrix stuff in one place in the declarative config; you're welcome to steal any of that you like in case it is helpful.
You're probably familiar with it, but especially as its changed a lot lately and gained a lot of new features, GitHub branch protection can really help reduce risk and put your mind more at ease when giving others write access; it's pretty powerful these days and heavily customizable. In fact, on the core Python org, we were able to use to to develop a way to safely give core triagers the missing GitHub-level permissions of the
Yup, that definitely seems reasonable to me. However, to my surprise, despite the project having several hundred stars, it doesn't look like you have any here—whereas QtPy only has a few times more and we've had many dozens of community contributors step up to help (including you!), many with multiple complex PRs over a sustained period of time. I figure maybe because QtPy occupies a somewhat different niche in the space, and is generally used directly mostly by the comparatively more experienced FOSS folks, as opposed to individual users, but its still a bit surprising. You could consider putting out a public call to your community, and maybe also reaching out to other trusted maintainers from the Qt-related Conda-Forge projects, PySide, QtPy, and Qt dev channels. However, ultimately this is really on the shoulders of your user community who benefits from your hard and thankless work here.
Surprisingly, there doesn't appear to be any fork at all that has had any independent activity on it... :|
Yeah, I'd of course defer to your expertise on that; the upstream pinning is only the proximate cause of the immediate (and I presume necessary for stable operation).
It seems like a saw some successes browsing the many Stack Overflow threads about this, but I only skimmed them and its unclear how much they had to modify things. |
|
Alrighty, I might start an exploration. Anyone interested could follow along. Plan is one last release covering Qt5. Only latest of each of Qt 5 and 6. Python 3.8+. This is a complete change of approach from trying to help everyone on any version. Just trying to reduce scope and build times so there's at least a little less resistance against maintenance. Not sure about macOS M1/ARM since I remember something about needing another PyPI exception since the universal wheel was even bigger, but I haven't found notes about that yet. I'll start with just version update level stuff. If that goes well then maybe we'll worry about some of the reported issues. Don't get too excited, this may quickly become nothing more than a refresher on why I didn't finish this effort last time I started. But, any questions from people interested in learning are welcome. altendky/qt-applications#23 |
|
Oh yeah, and not having runners for macOS ARM. Though I could check on those services that do or maybe cross build. Both of which introduce additional complexity. |
|
In case anyone has experience with this. Consistently with Qt 6.5 on Windows. Qt 5.15.2 on Windows and other platforms with Qt 6 are fine. Sometimes it's the directory and sometimes a file inside. Maybe running out of disk space and getting a silly message for it? |
|
I'm copying a directory. Not sure why only here or only a problem here, but... |
|
So, another useful note is that I would guess that a lot of people would be ok with just the qt-tools layer (which depends on and uses qt-applications). That gets you Designer etc. What it doesn't get is the plugins built in pyqt-plugins. That's fine if you aren't making custom Python widgets that you want to use inside Designer. Or... some QML plugin stuff too. Anyways, this is relevant since it removes the dependence on a specific PyQt version. I mean yeah, you want a vaguely related version of Designer, but it doesn't have to match exactly with the Qt version used by the PyQt you are using. But hopefully... we get lucky and future maintenance of these projects is a bit easier and it just keeps up. |
Not an expert, but one reason you'll see this is if Other reasons for this are Windows' very aggressive and sometimes unpredictable file locking whenever anything holds a handle to a file (or didn't close it properly), or something went wrong with setting the permissions for the file. But it seems the first is the most likely from what I can see...not sure though. |
|
Thanks for the pointers. I'm thinking that the reporting from |
|
Looks like two issues at this point. On macOS with Python 3.8-3.10 (3.11 passes) there's some library search path issue. On Windows 3.11 Qt 5 (not 3.8-3.10 nor Qt 6 on 3.11) there's some goofy thing with the QML paths or something. Or, there are at least those two things. |
|
macOS fixed. I was using |
|
PyQt5 5.15.9 // Register the plugin's types.
void PyQt5QmlPlugin::registerTypes(const char *uri)
{
// Construct the import path that the engine will have used by default.
// Unfortunately we won't know about any directories added by qmlscene's -I
// flag.
QStringList import_path;
import_path << QCoreApplication::applicationDirPath();
const char *env_path = getenv("QML2_IMPORT_PATH");
if (env_path)
{
#if defined(Q_OS_WIN)
QLatin1Char sep(';');
#else
QLatin1Char sep(':');
#endif
QStringList env_dirs = QString::fromLatin1(env_path).split(sep, QString::SkipEmptyParts);
foreach (QString dir_name, env_dirs)
import_path << QDir(dir_name).canonicalPath();
}
import_path << QLibraryInfo::location(QLibraryInfo::Qml2ImportsPath);PyQt6 6.4.2 // Register the plugin's types.
void PyQt6QmlPlugin::registerTypes(const char *uri)
{
<snip>
QStringList env_dirs = QString::fromLatin1(env_path).split(sep, Qt::SkipEmptyParts);
<snip>
import_path << QLibraryInfo::path(QLibraryInfo::Qml2ImportsPath); |
FWIW, at least in our QtPy CI matrix, PyQt5 5.15, PySide2 5.15 (conda only), PyQt6 6.4 and PySide6 6.4 all work on Python 3.11, while the PySide2 5.15 PyPI wheel isn't built for it (the PyQt5 wheel is built with the CPython stable ABI), and PyQt5/PySide2 5.12 and PySide6 <= 6.2-6.3, and I believe PyQt6 <= 6.2, are confirmed not to work. |
|
Thanks for sharing that reference. I posted to the maillist just in case I get lucky and someone has time and knowledge to help out. https://www.riverbankcomputing.com/pipermail/pyqt/2023-March/045225.html For the sake of not getting stuck, I'll just xfail the failing QML tests for PyQt5 on Windows for Python 3.11. If we resolve this even after release, it is easy enough to push another release at this level. qt-applications is the one I don't want to roll lots of releases of given the size and the three layers above it that need tested against it. |
|
One of the reasons to not move this out of my user org is that I have a pro account and so have double the concurrency (last I checked) of orgs. I could make the python-qt-tools org pro, but that would cost a lost more since that's per user cost in the org. Not saying it can't be moved, just noting one of the factors. With the smaller matrix right now it's not such a big deal. Though with things working I might expand it a bit again if more versions also work. |
|
One of the issues around (not) pinning is that we have both the underlying Qt/PyQt library version and the 'tools' version. Or, well, three if you take Qt and PyQt separately. Or four if you count PyQtX-QtX separately... Anyways, how would I describe a dependency on pyqt-plugins for use with PyQt6 6.3.1 and Qt 6.3.2 that is at least version 2.2 of all these wrapper projects here? Anyone have any commentary or thoughts about handling this better? One option is to just leave the regular dependencies unpinned, allowing people to get entirely the wrong combinations if they want, and then provide an extra so if you ask for |
|
Hmm, this is indeed a tricky situation. After some followup investigation, seems the fundamental issue is that the source distribution of the package can (at least theoretically) be built with a range of supported versions of PyQt, and don't necessarily need the PyQt version pinned, while the binary wheels are necessarily built with a specific Qt/PyQt version and thus are specific to that version, and (I presume) must be used with PyQt also pinned to that specific version. See this active Packaging Discourse thread for discussion of this specific use case. As a further complicating factor, as a result of this, two separately-varying version sequences are encoded into the package version, which then results in the above situation where it comes very complicated to impose separate constraints on both the actual It's a complex issue without an obvious an easy solution, for sure, with two separate though related parts: handling the differing dependency specs between the sdists and wheels, and handling the two different versions between your wrapper package and the underlying bindings. For the dependency specs, since you're already accepting the cost of using a dynamic For the versions, ultimately you probably want to decouple the two distinct versions (PyQt and your wrappers) in some form. Probably the easiest immediate solution is to only encode the version of your bindings in the public version identifier. This is the easiest immediate solution to implement, and allows flexibly or precisely specifying the compatible desired "tools" version separately from the PyQt version, as either you or the user specify them separately for There are no real downsides to this with the sdist, though it means for the wheels, each To avoid the latter issue, you could potentially split out the binary requirement to a separate package that only gets installed with the wheels, but I'm not sure there's an easy way to do that—I'd have to think about it more. I recommend asking about this further in the Packaging Discourse category to get some expert feedback and possible better ideas. |
|
Hum, so I don't think I'm actually going to take on the version scheme change this time. I don't want to hazard getting stuck and not releasing. Also, I don't think that another release with the existing scheme is likely to make the situation much worse. The biggest opportunity is that there exists no release for the higher 6 versions. But, I would probably Oh well. I don't think I've thought about this as much as you in the general context, seemingly, but I think that publishing an sdist with the present versioning scheme would probably be more confusing than helpful. You Lines 34 to 46 in 2583303 And yes, this dynamic code in As to separating out the binary packages, that was part of the point of the four layers. The two binary layers are pyqt-plugins and qt-applications. The pyqt-tools and qt-tools are the pure python wrappers around those. The primary use case I had imagined in terms of version constraints was that a user was working with PyQt 5.14.2 and wanted the latest tools that matched. Presently they can |
|
And... releases are out. Now to see if they work. Thanks for your interest, time, and insights around this @CAM-Gerlach. Let's hope I can keep the workload lower and still more responsive in the future. And maybe a user will jump in and give the next needed update a try. FYI, I'll be closing out most of the install related issues with the note below. I also entered several new tickets about issues I ignored or noticed in the past few days.
|
Hi @altendky ,
It appears this project remains very popular and useful to users thanks to your work, but it appears to not been maintained in some time. Due to its dependencies being hard-pinned down to specific PyQt versions that don't have pre-built binary wheels available for modern Pythons, users have now become unable to easily build and install it, and in PyQt6's case, given only <= 6.1.0 is supported, are versions not widely supported by current Qt-based software.
The status quo, with mass user confusion and frustration trying and failing to install the package, clearly isn't good for anyone involved, and I might suggest a number of options to deal with it, depending on what you prefer to do as maintainer:
Thanks!
The text was updated successfully, but these errors were encountered: