Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Docker Compose version detection to predictably pick v2 only if v1 is not available #16793

Merged
merged 1 commit into from
Dec 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -105,16 +105,15 @@
Result lastResult = null;
Version version = null;
boolean isVersionHighEnough = false;
boolean isComposeAvailable = false;
boolean isComposeV2Available = false;
DockerComposeAvailability dockerComposeAvailability = null;

Check warning on line 108 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L108

Added line #L108 was not covered by tests

// Check if the Docker binary exists
final Optional<String> dockerBinary = getDockerPath();
if (isExcludedOs() == false && dockerBinary.isPresent()) {
dockerPath = dockerBinary.get();

// Since we use a multi-stage Docker build, check the Docker version meets minimum requirement
lastResult = runCommand(dockerPath, "version", "--format", "{{.Server.Version}}");
lastResult = runCommand(execOperations, dockerPath, "version", "--format", "{{.Server.Version}}");

Check warning on line 116 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L116

Added line #L116 was not covered by tests

if (lastResult.isSuccess()) {
version = Version.fromString(lastResult.stdout.trim(), Version.Mode.RELAXED);
Expand All @@ -123,15 +122,11 @@

if (isVersionHighEnough) {
// Check that we can execute a privileged command
lastResult = runCommand(dockerPath, "images");

lastResult = runCommand(execOperations, dockerPath, "images");

Check warning on line 125 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L125

Added line #L125 was not covered by tests
// If docker all checks out, see if docker-compose is available and working
Optional<String> composePath = getDockerComposePath();
if (lastResult.isSuccess() && composePath.isPresent()) {
isComposeAvailable = runCommand(composePath.get(), "version").isSuccess();
if (lastResult.isSuccess()) {
dockerComposeAvailability = DockerComposeAvailability.detect(execOperations, dockerPath).orElse(null);

Check warning on line 128 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L128

Added line #L128 was not covered by tests
reta marked this conversation as resolved.
Show resolved Hide resolved
}

isComposeV2Available = runCommand(dockerPath, "compose", "version").isSuccess();
}
}
}
Expand All @@ -140,8 +135,7 @@

this.dockerAvailability = new DockerAvailability(
isAvailable,
isComposeAvailable,
isComposeV2Available,
dockerComposeAvailability,
isVersionHighEnough,
dockerPath,
version,
Expand Down Expand Up @@ -291,17 +285,6 @@
return Arrays.asList(DOCKER_BINARIES).stream().filter(path -> new File(path).exists()).findFirst();
}

/**
* Searches the entries in {@link #DOCKER_COMPOSE_BINARIES} for the Docker Compose CLI. This method does
* not check whether the installation appears usable, see {@link #getDockerAvailability()} instead.
*
* @return the path to a CLI, if available.
*/
private Optional<String> getDockerComposePath() {
// Check if the Docker binary exists
return Arrays.asList(DOCKER_COMPOSE_BINARIES).stream().filter(path -> new File(path).exists()).findFirst();
}

private void throwDockerRequiredException(final String message) {
throwDockerRequiredException(message, null);
}
Expand All @@ -321,7 +304,7 @@
* while running the command, or the process was killed after reaching the 10s timeout,
* then the exit code will be -1.
*/
private Result runCommand(String... args) {
private static Result runCommand(ExecOperations execOperations, String... args) {
if (args.length == 0) {
throw new IllegalArgumentException("Cannot execute with no command");
}
Expand Down Expand Up @@ -356,14 +339,9 @@
public final boolean isAvailable;

/**
* True if docker-compose is available.
* Non-null if docker-compose v1 or v2 is available.
*/
public final boolean isComposeAvailable;

/**
* True if docker compose is available.
*/
public final boolean isComposeV2Available;
public final DockerComposeAvailability dockerComposeAvailability;

/**
* True if the installed Docker version is &gt;= 17.05
Expand All @@ -387,23 +365,70 @@

DockerAvailability(
boolean isAvailable,
boolean isComposeAvailable,
boolean isComposeV2Available,
DockerComposeAvailability dockerComposeAvailability,
boolean isVersionHighEnough,
String path,
Version version,
Result lastCommand
) {
this.isAvailable = isAvailable;
this.isComposeAvailable = isComposeAvailable;
this.isComposeV2Available = isComposeV2Available;
this.dockerComposeAvailability = dockerComposeAvailability;

Check warning on line 375 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L375

Added line #L375 was not covered by tests
this.isVersionHighEnough = isVersionHighEnough;
this.path = path;
this.version = version;
this.lastCommand = lastCommand;
}

public boolean isDockerComposeAvailable() {
return dockerComposeAvailability != null;
}
}

/**
* Marker interface for Docker Compose availability
*/
private interface DockerComposeAvailability {
/**
* Detects Docker Compose V1/V2 availability
*/
private static Optional<DockerComposeAvailability> detect(ExecOperations execOperations, String dockerPath) {
Optional<String> composePath = getDockerComposePath();

Check warning on line 395 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L395

Added line #L395 was not covered by tests
if (composePath.isPresent()) {
if (runCommand(execOperations, composePath.get(), "version").isSuccess()) {
return Optional.of(new DockerComposeV1Availability());

Check warning on line 398 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L398

Added line #L398 was not covered by tests
}
}

if (runCommand(execOperations, dockerPath, "compose", "version").isSuccess()) {
return Optional.of(new DockerComposeV2Availability());

Check warning on line 403 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L403

Added line #L403 was not covered by tests
}

return Optional.empty();

Check warning on line 406 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L406

Added line #L406 was not covered by tests
}

/**
* Searches the entries in {@link #DOCKER_COMPOSE_BINARIES} for the Docker Compose CLI. This method does
* not check whether the installation appears usable, see {@link #getDockerAvailability()} instead.
*
* @return the path to a CLI, if available.
*/
private static Optional<String> getDockerComposePath() {
// Check if the Docker binary exists
return Arrays.asList(DOCKER_COMPOSE_BINARIES).stream().filter(path -> new File(path).exists()).findFirst();

Check warning on line 417 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L417

Added line #L417 was not covered by tests
}

}

/**
* Docker Compose V1 availability
*/
public static class DockerComposeV1Availability implements DockerComposeAvailability {}

Check warning on line 425 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L425

Added line #L425 was not covered by tests

/**
* Docker Compose V2 availability
*/
public static class DockerComposeV2Availability implements DockerComposeAvailability {}

Check warning on line 430 in buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java#L430

Added line #L430 was not covered by tests

/**
* This class models the result of running a command. It captures the exit code, standard output and standard error.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
import org.opensearch.gradle.SystemPropertyCommandLineArgumentProvider;
import org.opensearch.gradle.docker.DockerSupportPlugin;
import org.opensearch.gradle.docker.DockerSupportService;
import org.opensearch.gradle.docker.DockerSupportService.DockerComposeV2Availability;
import org.opensearch.gradle.info.BuildParams;
import org.opensearch.gradle.precommit.TestingConventionsTasks;
import org.opensearch.gradle.util.GradleUtils;
Expand Down Expand Up @@ -171,11 +172,8 @@
.findFirst();

composeExtension.getExecutable().set(dockerCompose.isPresent() ? dockerCompose.get() : "/usr/bin/docker");
reta marked this conversation as resolved.
Show resolved Hide resolved
if (dockerSupport.get().getDockerAvailability().isComposeV2Available) {
composeExtension.getUseDockerComposeV2().set(true);
} else if (dockerSupport.get().getDockerAvailability().isComposeAvailable) {
composeExtension.getUseDockerComposeV2().set(false);
}
composeExtension.getUseDockerComposeV2()
.set(dockerSupport.get().getDockerAvailability().dockerComposeAvailability instanceof DockerComposeV2Availability);

Check warning on line 176 in buildSrc/src/main/java/org/opensearch/gradle/testfixtures/TestFixturesPlugin.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/testfixtures/TestFixturesPlugin.java#L175-L176

Added lines #L175 - L176 were not covered by tests

tasks.named("composeUp").configure(t -> {
// Avoid running docker-compose tasks in parallel in CI due to some issues on certain Linux distributions
Expand Down Expand Up @@ -232,8 +230,7 @@

private void maybeSkipTask(Provider<DockerSupportService> dockerSupport, Task task) {
task.onlyIf(spec -> {
boolean isComposeAvailable = dockerSupport.get().getDockerAvailability().isComposeV2Available
|| dockerSupport.get().getDockerAvailability().isComposeAvailable;
boolean isComposeAvailable = dockerSupport.get().getDockerAvailability().isDockerComposeAvailable();

Check warning on line 233 in buildSrc/src/main/java/org/opensearch/gradle/testfixtures/TestFixturesPlugin.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/testfixtures/TestFixturesPlugin.java#L233

Added line #L233 was not covered by tests
if (isComposeAvailable == false) {
LOGGER.info("Task {} requires docker-compose but it is unavailable. Task will be skipped.", task.getPath());
}
Expand Down
Loading