Skip to content

Commit

Permalink
Add installation & subscription options for Otel log ingestion lambda (
Browse files Browse the repository at this point in the history
…#294)

* otel subscription
  • Loading branch information
chaudharysaket authored Aug 19, 2024
1 parent e2895d5 commit d84fb54
Show file tree
Hide file tree
Showing 12 changed files with 1,095 additions and 59 deletions.
69 changes: 69 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ A CLI to install the New Relic AWS Lambda integration and layers.
* [AWS Lambda Layers](#aws-lambda-layers)
* [AWS Lambda Functions](#aws-lambda-functions)
* [NewRelic Log Subscription](#newRelic-log-subscription)
* [NewRelic Otel Log Ingestions](#newRelic-otel-ingestions-install)
* [NewRelic Otel Log Subscription](#newRelic-otel-log-subscription)

* **[Docker](#docker)**
* **[Contributing](#contributing)**
* **[Code Style](#code-style)**
Expand Down Expand Up @@ -255,6 +258,72 @@ newrelic-lambda subscriptions uninstall --function <name or arn>
| `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
| `--aws-region` or `-r` | No | The AWS region this function is located. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |

### NewRelic Otel Ingestions Install

#### Install Otel Log Ingestion

```bash
newrelic-lambda otel-ingestions install \
--nr-account-id <account id> \
--nr-api-key <api key>
```

| Option | Required? | Description |
|--------|-----------|-------------|
| `--nr-account-id` or `-a` | Yes | The [New Relic Account ID](https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/account-id) for this integration. Can also use the `NEW_RELIC_ACCOUNT_ID` environment variable. |
| `--nr-api-key` or `-k` | Yes | Your [New Relic User API Key](https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#user-api-key). Can also use the `NEW_RELIC_API_KEY` environment variable. |
| `--memory-size` or `-m` | No | Memory size (in MiB) for the New Relic log ingestion function. Default to 128MB. |
| `--nr-region` | No | The New Relic region to use for the integration. Can use the `NEW_RELIC_REGION` environment variable. Can be either `eu` or `us`. Defaults to `us`. |
| `--timeout` or `-t` | No | Timeout (in seconds) for the New Relic log ingestion function. Defaults to 30 seconds. |
| `--role-name` | No | Role name for the ingestion function. If you prefer to create and manage an IAM role for the function to assume out of band, do so and specify that role's name here. This avoids needing CAPABILITY_IAM. |
| `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
| `--aws-region` or `-r` | No | The AWS region for the integration. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |
| `--aws-role-policy` | No | Specify an alternative IAM role policy ARN for this integration. |
| `--tag <key> <value>` | No | Sets tags on the CloudFormation Stacks this CLI creates. Can be used multiple times, example: `--tag key1 value1 --tag key2 value2`. |
| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-aws-otel-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicOtelLogIngestion stack |


#### Uninstall Otel Log Ingestion

```bash
newrelic-lambda otel-ingestions uninstall
```

| Option | Required? | Description |
|--------|-----------|-------------|
| `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
| `--aws-region` or `-r` | No | The AWS region for the integration. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |
| `--force` or `-f` | No | Forces uninstall non-interactively |
| `--nr-account-id` or `-a` | No | The [New Relic Account ID](https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/account-id) for the integration. Only required if also uninstalling the New Relic AWS Lambda integration. Can also use the `NEW_RELIC_ACCOUNT_ID` environment variable. |
| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-aws-otel-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicOtelLogIngestion stack |

### NewRelic Otel Log Subscription

#### Install Otel Log Subscription

```bash
newrelic-lambda subscriptions install --function <name or arn> --otel
```

| Option | Required? | Description |
|--------|-----------|-------------|
| `--function` or `-f` | Yes | The AWS Lambda function name or ARN in which to remove a log subscription. Can provide multiple `--function` arguments. Will also accept `all`, `installed` and `not-installed` similar to `newrelic-lambda functions list`. |
| `--otel` or `-o` | Yes | Use this flag to install subscription filters for Lambdas that are instrumented with OpenTelemetry (Otel) |
| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-aws-otel-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicOtelLogIngestion stack |


#### Uninstall Otel Log Subscription

```bash
newrelic-lambda subscriptions uninstall --function <name or arn> --otel
```

| Option | Required? | Description |
|--------|-----------|-------------|
| `--function` or `-f` | Yes | The AWS Lambda function name or ARN in which to remove a log subscription. Can provide multiple `--function` arguments. Will also accept `all`, `installed` and `not-installed` similar to `newrelic-lambda functions list`. |
| `--otel` or `-o` | Yes | Use this flag to install subscription filters for Lambdas that are instrumented with OpenTelemetry (Otel) |
| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-aws-otel-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicOtelLogIngestion stack |

## Docker

Now, you can run newrelic-lambda-cli as a container.
Expand Down
15 changes: 14 additions & 1 deletion newrelic_lambda_cli/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@
IntegrationInstall,
IntegrationUpdate,
LayerInstall,
OtelIngestionInstall,
OtelIngestionUninstall,
OtelIngestionUpdate,
)
from newrelic_lambda_cli.utils import parse_arn

Expand Down Expand Up @@ -333,7 +336,17 @@ def disable_integration(self, linked_account_id, provider_slug, service_slug):


def validate_gql_credentials(input):
assert isinstance(input, (IntegrationInstall, IntegrationUpdate, LayerInstall))

assert isinstance(
input,
(
IntegrationInstall,
IntegrationUpdate,
LayerInstall,
OtelIngestionInstall,
OtelIngestionUpdate,
),
)

try:
return NewRelicGQL(input.nr_account_id, input.nr_api_key, input.nr_region)
Expand Down
9 changes: 8 additions & 1 deletion newrelic_lambda_cli/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@

import click

from newrelic_lambda_cli.cli import functions, integrations, layers, subscriptions
from newrelic_lambda_cli.cli import (
functions,
integrations,
layers,
otel_ingestions,
subscriptions,
)


@click.group()
Expand All @@ -17,6 +23,7 @@ def cli(ctx, verbose):
def register_groups(group):
functions.register(group)
integrations.register(group)
otel_ingestions.register(group)
layers.register(group)
subscriptions.register(group)

Expand Down
219 changes: 219 additions & 0 deletions newrelic_lambda_cli/cli/otel_ingestions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
# -*- coding: utf-8 -*-

import boto3
import click

from newrelic_lambda_cli import api, otel_ingestions, permissions, integrations
from newrelic_lambda_cli.types import (
OtelIngestionInstall,
OtelIngestionUninstall,
OtelIngestionUpdate,
)
from newrelic_lambda_cli.cli.decorators import add_options, AWS_OPTIONS, NR_OPTIONS
from newrelic_lambda_cli.cliutils import done, failure


@click.group(name="otel-ingestions")
def ingestion_group():
"""Manage New Relic AWS Lambda Otel Log Ingestion lambda"""
pass


def register(group):
group.add_command(ingestion_group)
ingestion_group.add_command(install)
ingestion_group.add_command(uninstall)
ingestion_group.add_command(update)


@click.command(name="install")
@add_options(AWS_OPTIONS)
@click.option(
"--aws-role-policy",
help="Alternative AWS role policy to use for integration",
metavar="<arn>",
)
@click.option(
"--stackname",
default=otel_ingestions.OTEL_INGEST_STACK_NAME,
help=f"The AWS Cloudformation stack name which contains the {otel_ingestions.OTEL_INGEST_LAMBDA_NAME} lambda function",
metavar="<arn>",
show_default=False,
required=False,
)
@click.option(
"--memory-size",
"-m",
default=128,
help="Memory size (in MiB) for the log ingestion function",
metavar="<size>",
show_default=True,
type=click.INT,
)
@add_options(NR_OPTIONS)
@click.option(
"--timeout",
"-t",
default=30,
help="Timeout (in seconds) for the New Relic log ingestion function",
metavar="<secs>",
show_default=True,
type=click.INT,
)
@click.option(
"--role-name",
default=None,
help="The name of a pre-created execution role for the log ingest function",
metavar="<role_name>",
show_default=False,
)
@click.option(
"--tag",
"tags",
default=[],
help="A tag to be added to the CloudFormation Stack (can be used multiple times)",
metavar="<key> <value>",
multiple=True,
nargs=2,
)
@click.pass_context
def install(ctx, **kwargs):
"""Install New Relic AWS OTEL Ingestion Lambda"""
input = OtelIngestionInstall(session=None, verbose=ctx.obj["VERBOSE"], **kwargs)

input = input._replace(
session=boto3.Session(
profile_name=input.aws_profile, region_name=input.aws_region
)
)

if input.aws_permissions_check:
permissions.ensure_integration_install_permissions(input)

click.echo("Validating New Relic credentials")
gql_client = api.validate_gql_credentials(input)

click.echo("Retrieving integration license key")
nr_license_key = api.retrieve_license_key(gql_client)

install_success = True

click.echo(
f"Creating {otel_ingestions.OTEL_INGEST_LAMBDA_NAME} Lambda function in AWS account"
)
res = otel_ingestions.install_otel_log_ingestion(input, nr_license_key)
install_success = res and install_success


@click.command(name="uninstall")
@add_options(AWS_OPTIONS)
@click.option(
"--nr-account-id",
"-a",
envvar="NEW_RELIC_ACCOUNT_ID",
help="New Relic Account ID",
metavar="<id>",
required=False,
type=click.INT,
)
@click.option(
"--stackname",
default=otel_ingestions.OTEL_INGEST_STACK_NAME,
help=f"The AWS Cloudformation stack name which contains the {otel_ingestions.OTEL_INGEST_LAMBDA_NAME} lambda function",
metavar="<arn>",
show_default=False,
required=False,
)
@click.option("--force", "-f", help="Force uninstall non-interactively", is_flag=True)
def uninstall(**kwargs):
"""Uninstall New Relic AWS OTEL Ingestion Lambda"""
input = OtelIngestionUninstall(session=None, **kwargs)

input = input._replace(
session=boto3.Session(
profile_name=input.aws_profile, region_name=input.aws_region
)
)

if input.aws_permissions_check:
permissions.ensure_integration_uninstall_permissions(input)

integrations.remove_log_ingestion_function(input, otel=True)

done("Uninstall Complete")


@click.command(name="update")
@add_options(AWS_OPTIONS)
@click.option(
"--stackname",
default=otel_ingestions.OTEL_INGEST_STACK_NAME,
help=f"The AWS Cloudformation stack name which contains the {otel_ingestions.OTEL_INGEST_LAMBDA_NAME} lambda function",
metavar="<arn>",
show_default=False,
required=False,
)
@click.option(
"--memory-size",
"-m",
help="Memory size (in MiB) for the log ingestion function",
metavar="<size>",
type=click.INT,
)
@add_options(NR_OPTIONS)
@click.option(
"--timeout",
"-t",
help="Timeout (in seconds) for the New Relic log ingestion function",
metavar="<secs>",
type=click.INT,
)
@click.option(
"--role-name",
default=None,
help="The name of a new pre-created execution role for the log ingest function",
metavar="<role_name>",
show_default=False,
)
@click.option(
"--stackname",
default="NewRelicOtelLogIngestion",
help="The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function",
metavar="<arn>",
show_default=False,
required=False,
)
@click.option(
"--tag",
"tags",
default=[],
help="A tag to be added to the CloudFormation Stack (can be used multiple times)",
metavar="<key> <value>",
multiple=True,
nargs=2,
)
def update(**kwargs):
"""UpdateNew New Relic AWS OTEL Ingestion Lambda"""
input = OtelIngestionUpdate(session=None, **kwargs)

input = input._replace(
session=boto3.Session(
profile_name=input.aws_profile, region_name=input.aws_region
)
)

if input.aws_permissions_check:
permissions.ensure_integration_install_permissions(input)

update_success = True

click.echo(
f"Updating {otel_ingestions.OTEL_INGEST_LAMBDA_NAME} Lambda function in AWS account"
)
res = otel_ingestions.update_otel_log_ingestion(input)
update_success = res and update_success

if update_success:
done("Update Complete")
else:
failure("Update Incomplete. See messages above for details.", exit=True)
Loading

0 comments on commit d84fb54

Please sign in to comment.