Weekly Digest (13 January, 2019 - 20 January, 2019)

[weekly-digest]

Here's the Weekly Digest for git-for-windows/git:

---

ISSUES

Last week 8 issues were created.
Of these, 2 issues have been closed and 6 issues are still open.

OPEN ISSUES

💚 #2031 git gui and atom still asks me a ssh passphrase, by devcline
💚 #2030 Git Bash: prioritize cygwin binaries in PATH, by mcandre
💚 #2029 error: Use of uninitialized value $hash in chomp, by fgc0109
💚 #2028 Unable to update to 2.20.1 (process system closing required), by the78mole
💚 #2027 Regression: 2.20.1 git-upload-pack SPNEGO transaction breakage, by kkm000
💚 #2025 Inconsistent Authentication Behavior, by Eddie-Hartman

CLOSED ISSUES

❤️ #2032 Git install problems Windows 2008 R2 (solved - multiply mapped drive letters), by PhilipOakley
❤️ #2026 diff: ensure correct lifetime of external_diff_cmd, by kgybels

NOISY ISSUE

🔈 #2030 Git Bash: prioritize cygwin binaries in PATH, by mcandre
It received 3 comments.

---

PULL REQUESTS

Last week, 3 pull requests were created, updated or merged.

UPDATED PULL REQUEST

Last week, 1 pull request was updated.
💛 #2019 WIP: Address getenv() issues, by dscho

MERGED PULL REQUEST

Last week, 2 pull requests were merged.
💜 #2026 diff: ensure correct lifetime of external_diff_cmd, by kgybels
💜 #1991 Improve Azure Pipelines support, by dscho

---

COMMITS

Last week there were 19 commits.
🛠️ Merge pull request #1991 from dscho/azure-pipelines-g4w Improve Azure Pipelines support by dscho
🛠️ ci: parallelize testing on Windows The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the test-tool. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! travis: fix skipping tagged releases Add the comment that we added to the patch series intended for upstream git.git. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! tests: include detailed trace logs with --write-junit-xml upon failure Let's use the new test-tool path-utils file-size command in the new JUnit XML feature. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ tests: avoid calling Perl just to determine file sizes It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is. So let's just use the test-tool to do that job instead. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! tests: optionally write results as JUnit-style .xml Make sure to write the .xml in UTF-8 encoding. We also need to make sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! mingw: special-case arguments to sh While working on parallelizing the tests in Azure Pipelines, an issue was discovered with the is_msys2_sh() function: it expects the path components to be separated by exactly one dir separator. That does not need to be the case, though, e.g. when the components in the PATH variable have trailing slashes. Let's make the code much more robust in this respect. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: be more generous when wrapping up the setitimer() emulation Every once in a while, the Azure Pipeline fails with some semi-random error: timer thread did not terminate timely This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds. The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel. So let's give it ten seconds instead of only one. That should be enough. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ ci: use git-sdk-64-minimal build artifact Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here. In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call usr\bin\bash.exe directly, as git-cmd.exe is not included in the minimal SDK. That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m. Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge pull request #2026 from gitster/kg/external-diff-save-env diff: ensure correct lifetime of external_diff_cmd by dscho
🛠️ ci: move the Windows job to the top The Windows job currently takes a whopping ~1h20m to complete. Which is far longer than the next-longest job takes (linux-gcc, ~35m). As such, it makes sense to start the Windows job first, to minimize the overall run time (which is now pretty safely the run time of the Windows job). This affects only the Azure Pipelines configuration, not the Travis one, of course, as Travis cannot run our Windows job: 1h20m is distinctly longer than the 50 minute timeout of Travis' free tier. This commit is best viewed with --color-moved. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps It is not strictly necessary to start a command in PowerShell with &, but it is safer. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps We do not need build-extra. So let's not clone it. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps Some improvements (suggested via review on the Git mailing list) have not been incorporated into Git for Windows' master yet, other changes were left-overs from debugging that need reverting. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps Let's use the new YAML schema. See https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps Remove a trailing empty line. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps At some point we switched from Hosted Linux Preview to Ubuntu 16.04, at which point it became unnecessary to install the docker package for the Linux-32 job. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! Add a build definition for Azure DevOps For some reason, the indentation of the PowerShell tasks got all messed up. Fix that. While at it, do not define c() for only one user, and do not initialize /etc/passwd (it does not seem to have a noticeable effect). Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! ci/lib.sh: add support for Azure Pipelines TASKDEFINITIONSURI is not set in parallel tasks... And COLLECTIONURI is not set in the Linux32 job... Signed-off-by: Johannes Schindelin [email protected] by dscho

---

CONTRIBUTORS

Last week there was 1 contributor.
👤 dscho

---

STARGAZERS

Last week there were 9 stagazers.
⭐ chaojunke
⭐ armujahid
⭐ HS-Z
⭐ ba0m
⭐ chanekelsey
⭐ rory77
⭐ 77ZaRR77
⭐ AluBhorta
⭐ csabatini
You all are the stars! 🌟

---

RELEASES

Last week there were no releases.

---

That's all for last week, please 👀 Watch and ⭐ Star the repository git-for-windows/git to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆