Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The source artifacts provided with GitHub releases are not buildable. #11505

Open
1 of 3 tasks
AnatolyPopov opened this issue Nov 9, 2024 · 2 comments
Open
1 of 3 tasks
Labels
bug Something isn't working

Comments

@AnatolyPopov
Copy link

Apache Iceberg version

1.7.0 (latest release)

Query engine

None

Please describe the bug 🐞

The artifacts that are published with GitHub releases are not buildable because the is no .git directory so there is no information that is required by Gradle plugin com.gorylenko.gradle-git-properties. So the build fails when it tries to execute generateGitProperties.

Ideally would be good to either have a pre-built downloadable artifacts or at least an artifact with the sources that is possible to build from.

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
@AnatolyPopov AnatolyPopov added the bug Something isn't working label Nov 9, 2024
@AnatolyPopov AnatolyPopov changed the title The artifacts provided with GitHub releases are not buildable. The source artifacts provided with GitHub releases are not buildable. Nov 9, 2024
@Fokko
Copy link
Contributor

Fokko commented Nov 9, 2024

Hey @AnatolyPopov Thanks for raising this 🙏

The official channel is through Apache: https://dlcdn.apache.org/iceberg/apache-iceberg-1.7.0/apache-iceberg-1.7.0.tar.gz

This .tar.gz contains a version.txt file, that's being picked up by Gradle:

iceberg/build.gradle

Lines 1054 to 1073 in 1c576c5

String getProjectVersion() {
if (versionFileExists()) {
return getVersionFromFile()
}
try {
// we're fetching the version from the latest tag (prefixed with 'apache-iceberg-'),
// which can look like this: '0.13.0-2-g805400f0.dirty' but we're only interested in the MAJOR.MINOR.PATCH part
String version = gitVersion(prefix: 'apache-iceberg-')
Pattern pattern = Pattern.compile("^([0-9]+)\\.([0-9]+)\\.([0-9]+)(.*)?\$")
Matcher matcher = pattern.matcher(version)
if (matcher.matches()) {
// bump the MINOR version and always set the PATCH version to 0
return matcher.group(1) + "." + (Integer.valueOf(matcher.group(2)) + 1) + ".0-SNAPSHOT"
}
return version
} catch (Exception e) {
throw new Exception("Neither version.txt nor git version exists: " + e.getMessage(), e)
}
}

When that file is not there, it will try to determine the version through git. This way we don't have to bump the version after each release.

@AnatolyPopov
Copy link
Author

Thanks for a quick reply @Fokko!

First of all, thanks, building from the tarball you linked above works as expected.
However I have several points that I think still should be addressed.

I've been taking the sources from the tarball attached to the release here https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.7.0. I would assume that it should as well be buildable, otherwise IMO it does not make too much sense to have it published.

The version.txt was the first thing I've tried to add after looking into exactly the same piece of code you linked above but it does not help because there is still no iceberg-build.properties so the build crashes with
> Task :generateGitProperties FAILED
as expected.
I would expect both version.txt and iceberg-build.properties to be included in release tarball attached to GitHub release as well.

Also I can see that iceberg-build.properties from the official tarball you linked above is different from the one that is generated from sources and is lacking the tag apache-iceberg-1.7.0 in git.tags property. This seems to be a little misleading when build properties from official release tarball include only release candidate tag apache-iceberg-1.7.0-rc1.

And one more thing is that even if I clone the repo, checkout the tag, and without manually creating version.txt will try to run assembleDist I will get 1.8.0-SNAPSHOT version. As mentioned in #11506 I think people would expect to be able to build an artifact with the release version from a release commit without manual modifications. Am I missing something at this point?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants