A problem of waiting for Tool process to do something

[pistotniks]

I have the following problem; I have a target that contains:

[PathExecutable(name: "docker-compose")] readonly Tool DockerCompose;
[PathExecutable(name: "docker")] readonly Tool Docker;

Target CreateDatabase => _ => _
  .Executes(() =>
  {
  DockerCompose($"-f {file} up -d");
  //NOW I HAVE TO WAIT FOR THIS PROCESS TO FINISH ITS THING; in this particular case the process is creating a database and running its sql scripts (I USUALLY SEE THIS WITH LOOKING AT THE LOGS)
  DockerCompose($"-f {file} stop");
  Docker($"container commit bla bla bla");
  });

Now I have noticed on one of your sample projects that someone used: Thread.Sleep(ONE HOUR)

DockerCompose($"-f {file} up -d");
Thread.Sleep(ONE HOUR)
DockerCompose($"-f {file} stop");
Docker($"container commit bla bla bla");

And it is ok, and it would solve my problem. But isn't there a better way for this? Cause I would like somehow to check the logs (Retry) (whether or not some particular log statement appeared) or something similar or smarter?
Any ideas or suggestions? I must be doing something wrong or I need to change some approach?

[matkoch]

What sample project?

[pistotniks]

On the website there is a Resources section in Getting started and then there is an Example section and I went thru each one of them and found https://github.com/avivasolutionsnl/sitecore-docker/blob/601f158cdbc69622b4c11ae5125ab19cdfdf4326/build/Build.cs#L78

But I have an information now that the console command has an output and that I can use it so I am good to go to write a Polly Retry code. I believe it is an approach I like. I can post my solution after I make it work:)

[pistotniks]

So here is my possible I believe better solution than a sleepy thingy:) written in 30 minutes and you can even TDD it. Maybe it helps someone:

 Target CreateDatabase => _ => _
    .DependsOn(LoginToAzure)
    .Executes(async () => 
    {
      string file = RootDirectory / "build/mydb/docker-compose.database.yml";
      DockerCompose($"-f {file} up -d");
      IList<string> requiredLogs = new List<string>()
      {
        "DATABASE IS READY TO USE!",
        "00_myscript1.sql",
        "01_myscript2_TracedMessagesQueue.sql",
        "DONE: Executing user defined scripts"
      };
      try
      {
        // Wait for requiredLogs
        await Retry.Until(async () => await ReadLogs(), requiredLogs);
        Docker($"container commit database FooBar");
        Docker($"push FooBar");
      }
      catch (Exception e)
      {
        ControlFlow.Fail(e.Message);
      }
      finally
      {
        DockerCompose($"-f {file} stop");
        Docker($"rm database"); // Comment out when you want to troubleshoot the logs of a stopped container
      }
    });

  private Task<string> ReadLogs()
  {
    IReadOnlyCollection<Output> outputLines = Docker("logs --since 59m database");
    var nonErrorLines = outputLines.Where(line => line.Type == OutputType.Std);
    var builder = new StringBuilder();
    foreach (var nonErrorLine in nonErrorLines)
    {
      builder.Append(nonErrorLine.Text);
    }
    return Task.FromResult(builder.ToString());
  }

internal class Retry
  {
    private const int MaxNumberOfRetries = 50;

    public static async Task<string> Until(Func<Task<string>> action, IList<string> requiredLogs)
    {
      bool RetryPolicy(string result)
      {
        return string.IsNullOrEmpty(result) || ShouldRetrySinceNotAllRequiredLogsAreThere(result, requiredLogs);
      }

      return await Until<string, Exception>
      (
        action,
        MaxNumberOfRetries,
        RetryPolicy
      );
    }

    private static bool ShouldRetrySinceNotAllRequiredLogsAreThere(string result, IList<string> requiredLogs) 
      => requiredLogs.Any(requiredLog => !result.Contains(requiredLog));

    private static Task<T> Until<T, TU>(Func<Task<T>> action, int numberOfRetries, Func<T, bool> retryPolicy) where TU : Exception
    {
      TimeSpan SleepDurationProvider(int retryAttempt) => TimeSpan.FromSeconds(new Random().Next(40, 59));
      Task<T> task = Policy
        .HandleResult(retryPolicy)
        .Or<TU>()
        .WaitAndRetryAsync(numberOfRetries, (Func<int, TimeSpan>)SleepDurationProvider,
          (result, timeSpan, retryCount, context) =>
          {
            Console.WriteLine(result.Exception != null
              ? $"Retry {retryCount} of {numberOfRetries} ({timeSpan.TotalSeconds}s.) - Reason: Exception was thrown: {result.Exception.Message}"
              : $"Retry {retryCount} of {numberOfRetries} ({timeSpan.TotalSeconds}s.) - Reason: Retry policy not fulfilled.");
            if (retryCount == MaxNumberOfRetries)
            {
              throw new Exception("Retries exceeded.");
            }
          })
        .ExecuteAsync(action);
      return task;
    }
  }