Reproducible Builds

[IzzySoft]

At IzzyOnDroid we support Reproducible Builds (see: Reproducible Builds, special client support and more at IzzyOnDroid). I finally managed to get Kiwix RB, but only with a little trick – because you calculate versionCode based on build time, which can never be RB:

kiwix-android/buildSrc/src/main/kotlin/VersionCodeGenerator.kt

Lines 22 to 38 in 116180e

	fun String.getVersionCode(): Int {
	// the date when the automatic version code generation started
	val lastDate = LocalDate.of(2024, 7, 17)
	// Calculate the number of days between the lastDate and today's date.
	// This gives us the total number of days since the last version code was set.
	val daysDifference = ChronoUnit.DAYS.between(lastDate, LocalDate.now()).toInt()
	// Base version code. This is the version code of the last release uploaded to the Play Store.
	// We use this as the starting point for generating new version codes automatically.
	val baseVersionCode = 231101
	// Generate and return the new version code.
	// The new version code is calculated by adding the number of days since lastDate
	// to the base version code. This creates a unique version code for each day.
	return baseVersionCode + daysDifference
	}

So what I did to check if that's the only show-stopper there:

- GRADLE_VERSION="$(sed -rn 's/distributionUrl.*gradle-([0-9.]+).*\.zip/\1/p' gradle/wrapper/gradle-wrapper.properties)"
- sed -r 's/return baseVersionCode \+ daysDifference/return 231235/' -i buildSrc/src/main/kotlin/VersionCodeGenerator.kt
- git clone https://github.com/obfusk/gradlew.py.git
- gradlew.py/gradlew.py --version $GRADLE_VERSION -v assembleRelease -PdisableSigning
- mv app/build/outputs/apk/release/kiwix-armeabi-v7a-release-unsigned.apk /outputs/unsigned.apk

And it did the trick. As that would require to manually update at each release after a failed build gives us the correct number, may I kindly ask you to re-think that dynamic creation of your versionCode? If you really need to calculate it, maybe base it on commit information? Like:

git rev-list --all --count

would give you the number of commits at that point, which you then can use to add to your baseVersionCode.

We'd appreciate if you could help making your build reproducible. We've also prepared some hints on reproducible builds for that.

Looking forward to your reply!

[kelson42]

@IzzySoft Thank you, I agree there is a weakness here. One thing is that I consider a bad practice to make the source code compilation depending formally from the revision code system.

@MohitMaliFtechiz Can you please propose a solution where the date comes from outside the piece of code to compute the APK (version). Obviously this should still be easy to compile, that means it should be trivial for the developer to retrieve automatically the date from the git HEAD or if not found to automatically fallback to current date. Maybe we have already a configuration part which can do that, or we can use an environnement variable which could be set via a document one liner?

[MohitMaliFtechiz]

@kelson42 For the RB(Reproducible Builds) of the latest release. We can do the following things:

• We can use the env variable for this, e.g. RELEASE_DATE, it can be set via a document one liner. @IzzySoft please confirm if you can set the env variable in your script or not. You can get the latest release date via git log -1 --format=%cd --date=format:%Y-%m-%d $(git describe --tags --abbrev=0) this command it will give you the date like this 2024-12-18, and set this to env variable.
• If this env variable is set we will build the application via this release date, if not set the fallback will return to the current date(Like it is currently generating).

In this approach, we can build the APK with the same version code(As the release APK has). Since building with the git HEAD can have a different date, if commits are added at later dates. So to generate with exact versioncode it is better to pass the date via env.

Here @IzzySoft already has the date I guess. Since they starts the new build based on the GitHub release.

[IzzySoft]

@kelson42

I consider a bad practice to make the source code compilation depending formally from the revision code system.

Well, maybe not ideal – but still better than making it dependent on the time a build is run 😉

@MohitMaliFtechiz

please confirm if you can set the env variable in your script or no

Confirmed. Especially if it can be done automatically as you described (export RELEASE_DATE="$(git log -1 --format=%cd --date=format:%Y-%m-%d $(git describe --tags --abbrev=0))".

If this env variable is set we will build the application via this release date, if not set the fallback will return to the current date(Like it is currently generating).

Sounds reasonable, thanks! Shall we give that a try with the next release?

Since they starts the new build based on the GitHub release.

Exactly: we build from the commit the tag points to – and compare the resulting APK against the one attached to the very same tag.

[MohitMaliFtechiz]

Confirmed. Especially if it can be done automatically as you described (export RELEASE_DATE="$(git log -1 --format=%cd --date=format:%Y-%m-%d $(git describe --tags --abbrev=0))".

@IzzySoft Thanks for the confirmation.

Sounds reasonable, thanks! Shall we give that a try with the next release?

Yeah sure, I am making the changes in code for this.

[IzzySoft]

Yeah sure, I am making the changes in code for this.

Wonderful! If you give me a ping when that release is up, I can adjust things here before the regular daily run, processing it manually in advance (instead of delaying the batch).

[kelson42]

Yeah sure, I am making the changes in code for this.

Wonderful! If you give me a ping when that release is up, I can adjust things here before the regular daily run, processing it manually in advance (instead of delaying the batch).

@IzzySoft You can have an idea if issues are planned or not and when by monitoring the milestone attached to the issue. Here this is clearly included in the current milestone = next release.

[IzzySoft]

@kelson42 forgive me, but I cannot monitor hundreds of projects. So a ping would be really appreciated. Otherwise it will simply take longer, and might include a failed build 🤷‍♂

[MohitMaliFtechiz]

@IzzySoft I will ping you once we release the new version.

Confirmed. Especially if it can be done automatically as you described (export RELEASE_DATE="$(git log -1 --format=%cd --date=format:%Y-%m-%d $(git describe --tags --abbrev=0))".

I think after making these changes, the release will be automatically done(No need for ping)?

[IzzySoft]

We need to adjust the recipe once manually to establish that change. Next releases then should work automatically, as the recipe will be "copied & adjusted" automatically. So yes, a one-time ping – then all should work fine automatically (until some other change breaks things 🙈).