-
Notifications
You must be signed in to change notification settings - Fork 139
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Linux and macOS releases are missing -- static linking version preferred (Low Priority) #987
Comments
The current github action build of avrdude may not be suitable as a Linux release, lower/older version of Linux [eg: Ubuntu 18.04 or Debian 8) may be better. And different Linux distro may have different dependancies. So I am not sure how practical this is. For example, we only provide Windows binary for libusb projects. |
I just managed to build avrdude on a virtual machine running Debian Jessie. It did require a recent version of cmake (3.21.4) that I had to install from source as well.
For distribution, I checked the dependencies using
For maximum compatibility with different Linux distros, I am tempted to link these libraries statically:
Would you suggest any other libraries for static linking? For example, libudev or libreadline? |
OTOH, if you have an old reference system already available, could you just give the nightly builds a try? If they work even on your relatively old system, I'd just pick those. I don't think the APIs of the libraries you mentioned have been changed much lately. The bigger issue is probably that the "modern" build might require a certain LIBC_foo version symbol, for whatever reason. |
@ygramoel
|
Linking the four libraries mentioned above statically works for me. The others, including libudev, are linked dynamically:
This is the link command I used:
I had to It is most probably not a good idea to link libdl or libc statically. I am less certain about the others. |
I just tried it on an elderly Ubuntu 16.04LTS:
As you can see, it's not the libusb or libftdi stuff that is causing any trouble. It's that "minimum version" symbol in libc, and libreadline. |
I just tried the nightly build on this old Debian Virtual Machine - johan@debian:~/avrdude_nightly$ ./avrdude
./avrdude: error while loading shared libraries: libftdi1.so.2: cannot open shared object file: No such file or directory johan@debian:~/avrdude_nightly$ ldd avrdude
./avrdude: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./avrdude)
linux-vdso.so.1 (0x00007ffdfcef8000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbe8139f000)
libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007fbe81189000)
libusb-0.1.so.4 => /lib/x86_64-linux-gnu/libusb-0.1.so.4 (0x00007fbe80f80000)
libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007fbe80d68000)
libhidapi-libusb.so.0 => /usr/lib/x86_64-linux-gnu/libhidapi-libusb.so.0 (0x00007fbe80b61000)
libftdi1.so.2 => not found
libreadline.so.8 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbe807b6000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbe816a0000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fbe805a7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbe8038a000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbe80182000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbe7ff7e000) |
OK, so your system's libftdi1 is even older than Ubuntu 16, it seems. ;-) The biggest issue is libc. Everything else, you can just ship the .so that has been used to build along with the binary, and set |
Try adding librreadline to your above list of static libraries, and see whether the result will run on a current Linux distribution. |
Linking readline statically works; I had to add -ltinfo to the link command.
It runs on my Linux Mint 19 Tara, but not on a more recent Ubuntu 21.10. The problem is libtinfo:
|
The original version, with both libreadline and libtinfo linked dynamically, doesn't work on a recent Ubuntu either. However, linking both libreadline and libtinfo statically works. |
I'm Johan's (@ygramoel) colleague. Here is the binary he just made: https://new.embeetle.com/downloads/beetle_tools/linux/avrdude_7.0.0_feaa1c6_64b.7z It's a 7z compressed folder containing two files:
Can you please give it a try on your Linux machine(s)? |
Works here, customized Ubuntu 20.04. |
@kristofmulier Just to add some history and reasoning to this discussion, even if it may be redundant: The current CMake setup and Github Actions do not support building Linux binaries that are suitable for download, which is why they do not show up in the release artifacts. You can tell from the discussion that your requirement is not just a matter of including the build artifacts in the release, but actually quite a bit of work to get it right. Technically speaking, the whole build pipeline consists of three things: a) the CMake build system b) the GitHub build actions c) the GitHub release actions Regarding a) The current CMake setup is designed to support the two most common scenarios: 1) Building Un*x binaries on a defined target system using dynamic linking, suitable for package managers etc., and 2) Building Windows binaries suitable for download using static linking, supporting Windows 7 and up. See also: #960 (comment) Regarding b) The GitHub build action for Linux is merely a build check, with the intention that the build works on a current Linux system and toolset, throwing no errors and warnings, etc, for the scenario outlined in a). With that reasoning in mind, I picked the latest Debian distro (i.e. the VM 'ubuntu-latest') with the latest GCC toolset and the latest libraries. Regarding c) As the Linux binaries are not meant for downloading, only the Windows release artifacts (the MSVC builds with the custom libraries) are included in the release. Now, we have a few new requirements here, such as the Arduino team wishing for statically linked binaries that they can ship with the Arduino IDE (#946 (reply in thread)), which is probably what you expect from a Linux binary offered on the release page, or others finding value in building dynamically linked binaries for Windows (#954). All these requests are related, because they all require changes in a), b), and c). You are welcome to submit a PR that addresses your needs, however, I would like to point out that the requirements for building Linux binaries suitable for download conflict with the requirements that I outlined above (because you want static linking, support for older distros, etc.). So when you submit a PR, please keep the existing build and CI actions intact, and add a extra jobs that builds the required outputs. |
Works on Ubuntu 16.04 as well. |
Interesting from here: From the above comments, Arduino has just come out with their version of avrdude 7.0 test release for Linux/macOS and Windows. Their patches on top of the official avrdude 7.0 release. |
Hi @facchinm , |
Hello guys, @arduino maintainer here, I'm the guy responsible for https://github.com/arduino/avrdude-packing.
Yes, as I explained here the new
As I wrote here these are the dynamic dependencies required at runtime by the binary produced by our CI:
we are statically linking against:
In order to do so I have prepared a docker container able to crosscompile. Inside it, there are all the toolchains and dependencies required to produce that binary. As of now, the container is not yet public. But I'm planning to open it shortly. So that you guys can look how it works and, if you want, even use it in your CI.
Definitely yes. Our main goal is to have a binary that runs on almost everything. I used the following toolchains for the linux binaries:
By the way, the work I've done regarding the build is here if you are curious. Also, the release are there for testing purposes. So feel free to test it out.
Regarding this, feel free to integrate the work I already did here into your system. Removing the parts that you don't need. When the container will be made public, it will be possible to do so. |
What is the version used? Hopefully you are using the release here and not the old one from the signal11 repo.
Can you upgrade to libftdi1-1.5?
Wow, that libusb-1.0.20 is really old. It may be okay under Linux but I tend to think you may encounter users under macOS and Windows. 1.0.23 or 1.0.26 would be better (you can probably skip 1.0.24 and 1.0.25). |
I was using 0.11.2 I'm currently trying to update libusb to 1.0.26, libusb-compat to 0.1.7, libftdi to 1-1.5 and hidapi to 0.12.0 PS since libusb-compat is a mess to compile for win32 I'm using a precompiled version from libusb-win32 project (version 1.2.6.0) |
That is great.
Yes I agree that this is good. |
In the end I managed to:
libftdi1-1.5 was a pain to update and I gave up. They changed something in the build process and it does not build correctly with mingw and osxcross. So I gave up. |
Thanks for updating libusb-compat, libusb-1.0 and hidapi. Just wondering why you prefer to use cross-compiler. I also hear the same feedback by OpenOCD side that cross-compiling of libftdi-1.5 is problematic. If you use MSY2 under Windows (But there is no issue to build with MSYS mingw32/mingw64 -- they actually provide the latest dependancies. It is also used by the avrdude github action. Version of MSYS2 packages: @umbynos |
@umbynos |
Because we cannot build natively on GitHub actions CI, or at least we could, but the VMs GitHub offers are pretty updated (unfortunately) and do not allow us to build binaries that will work even on older OSes. Furthermore, there is no other way regarding arm and aarch64 architectures. |
Ps I just opened the repo we used to build https://github.com/arduino/crossbuild, if you want to give it an eye |
And also thanks @mcuee for the useful pointers and links. As of now, I'm a bit swamped work-wise. But when I'll have more bandwidth, I'll definitely give libftdi-1.5 a try. |
Just tried out the Arduino release of avrdude under macOS (mac Mini M1) and it seems to work fine. |
Thanks a lot @umbynos for the latest release: Is there a reason why there's no 64-bit version for Windows? I suppose the 32-bit version works just fine on Windows, though. |
git main may add more complexity due to the inclusion of readline. @umbynos |
Currently, our built version of avrdude is in an internal testing phase.
I'm the only one working on this, and I'm doing my best to keep up with all the issues and stuff. I have to find a solution to this, for sure. I don't know if really like the approach of bundling the
actually, we already have readline inside the container we use to build: https://github.com/arduino/crossbuild/tree/main/deps/readline-8.0 |
The solution is also simple -- to use libusb-compat-0.1, if you do not like to bundle libusb0.dll. I am not so sure if you like it or not. I think it is the right way to go.
readline may be a problem for Linux and macOS as mentioned here. I think the fix may not be so complicated though as you have figured out the way for avrdude 7.0 release and the fix to the git main may be the same. Please take a look at the following issue. Thanks. readline is not a real problem for avrdude mingw build but latest version of MSYS2 got problems with termcap/readline conflict. Since you are not using MSYS2 but cross compiling, you should have no issues. |
On the other hand, I think it is good to test static link version for Linux and macOS in the github action. Right now MinGW and MSVC use static linking (we should keep that) and Linux and macOS use dynamic linking (we should keep that as well but add static linking testing as well). |
For those who want to build their own Linux static linking binary, Arduino cross-build container and the Arduino-packing project build script are a good one to use. I have no issues to build 32bit Linux and macOS binary with avrdude git main using the container. You can see you do not need to use very complicated script. And only one patch is needed for Linux and macOS. I have some issues under mingw32 but hopefully @umbynos can help out (he is able to build under mingw32). @kristofmulier @MCUdude and @SpenceKonde, |
I will close this issue as #1162 will fix this issue and it is more generic (Linux, Windows and macOS static link binaries). |
Similar discussion here. |
As of now, avrdude project will only release Windows binary for avrdude and not Linux/macOS binary. If you need Linux (or macOS) static link avrdude binary for production purpose, it is good to use the release For testing purpose, you can try my snapshot release here. |
Re-open the issue but I make it more generic. |
Potential solutions but need more work on CMake.
Related discussions: |
Even though #1162 can be a potential solution, right now at least two patches used by @umbynos are not really acceptable for the avrdude project. Patch 06 may need some work. Patch 07 is just a hack. https://github.com/arduino/avrdude-packing/tree/main/patches (Patch 08 is just Arduino branding) |
After some thoughts, I have closed #1162. I think we should get #1159 done first and then this one. This is not a high priority either since there is the Arduino avrdude-packing project out there which provides the infrastructrure to build the static link version for Linux/macOS and Windows. Majority of the Linux users will still get avrdude through the Linux distro or build from source (typically with dynamic linking to dependacies). Majority of the macOS users will get avrdude through Homebrew or macPorts. |
You can use this one from Arduino avrdude-packing project. |
I will close this as not planned as the project has no intention to publish official Linux and macOS binaries. |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
I've very happy to find the latest releases here:
https://github.com/avrdudes/avrdude/releases
Thanks a lot @mariusgreuel and @dl8dtl for your awesome work! Unfortunately, I can't see any Linux releases. The Linux releases are available in the nightly builds though:
https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE
It would be great if they would also be available on the release page. I know that usually the Linux distros package their own binaries to fit into their package systems (RPM, dpkg, ...). But some people prefer to get the builds directly from GitHub. It would also be helpful for our usecase: we'd like to download
AVRDUDE
directly to the computer of our users (the users of Embeetle IDE, a new microcontroller IDE) without going through a package manager.Thanks a lot ^_^
The text was updated successfully, but these errors were encountered: