[CONTINT-4460] fix ecs fargate to use env specified agent image

[ewoodthomas]

What does this PR do?

Sets the agent image for Fargate tasks to the specified environment image tag rather than using public.ecr.aws/datadog/agent:latest

Which scenarios this will impact?

TestECSSuite should now properly test agent changes on Fargate. Prior to these changes, the latest published image tag was used on Fargate.

Motivation

Discovered this issue when adding an aws_account tag to ECS Fargate tasks. I added an expected tag to ecs_test.go, but TestECSSuite failed since the tag did not exist . The CI used the image pushed in the PR when defining the tests, but the Fargate tasks used the latest agent image tag published to the official Datadog ECR.

Additional Notes

Testing

Prior to changes; fargate task definition:

{
    "taskDefinitionArn": "arn:aws:ecs:us-east-1:376334461865:task-definition/ewoodthomas-ecs-cluster-nginx-fg:22",
    "containerDefinitions": [
        {
            "name": "datadog-agent",
            "image": "public.ecr.aws/datadog/agent:latest",
	    ...
	}
}

After changes; fargate task definition:

{
    "taskDefinitionArn": "arn:aws:ecs:us-east-1:376334461865:task-definition/ewoodthomas-ecs-cluster-nginx-fg:26",
    "containerDefinitions": [
        {
            "name": "datadog-agent",
            "image": "376334461865.dkr.ecr.us-east-1.amazonaws.com/agent-e2e-tests:ethan.woodthomas-agent",
            ...
        }
}

When no agent tag is specified, it still uses public.ecr.aws/datadog/agent:latest in the task definition.

[pducolin]

🔨 warning
Given dockerAgentFullImagePath implementation

func dockerAgentFullImagePath(e config.Env, repositoryPath, imageTag string, otel bool) string {
	if e.AgentFullImagePath() != "" {
		return e.AgentFullImagePath()
	}

	// if agent pipeline id and commit sha are defined, use the image from the pipeline pushed on agent QA registry
	if e.PipelineID() != "" && e.CommitSHA() != "" && imageTag == "" {
		tag := fmt.Sprintf("%s-%s", e.PipelineID(), e.CommitSHA())
		if otel {
			tag = fmt.Sprintf("%s-7-ot-beta", tag)
		}

		exists, err := e.InternalRegistryImageTagExists(fmt.Sprintf("%s/agent", e.InternalRegistry()), tag)
		if err != nil || !exists {
			panic(fmt.Sprintf("image %s/agent:%s not found in the internal registry", e.InternalRegistry(), tag))
		}
		return utils.BuildDockerImagePath(fmt.Sprintf("%s/agent", e.InternalRegistry()), tag)
	}

	...

and given we pass pipeline id and commit sha from the CI config in datadog-agent, I expect this to still install the latest agent version from "public.ecr.aws/datadog/agent" as imageTag is not empty.

In order to install the pipeline version and fallback to latest by default, I expect this to be

Suggested change

	Image: pulumi.String(dockerAgentFullImagePath(e, "public.ecr.aws/datadog/agent", "latest", false)),
	Image: pulumi.String(dockerAgentFullImagePath(e, "", "", false)),

[pducolin]

I see how dockerAgentFullImagePath can be confusing for the caller. Probably worth renaming it and add more context in the function description, will track this on our side so that we can follow up

[pducolin]

🔨 warning
See my previous comment for the context of this suggestion:

Suggested change

	Image: pulumi.String(dockerAgentFullImagePath(e, "public.ecr.aws/datadog/agent", "latest", false)),
	Image: pulumi.String(dockerAgentFullImagePath(e, "", "", false)),

[pducolin]

Nice catch, I still expect this to use latest, more details at #1171 (comment)