Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lambda-otel4s module #459

Open
wants to merge 66 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
1170ca8
Extract lambda-natchez module
alexcardell Feb 22, 2024
09d4127
Remove natchez-core dependency from lambda module
alexcardell Feb 22, 2024
375b814
Add lambda-otel4s module
alexcardell Feb 23, 2024
7ebde29
Use SqsEvent for attributes, not SqsRecord
alexcardell Feb 23, 2024
a5506a4
Add otel4s-sdk example
alexcardell Feb 23, 2024
a17882b
re-run prePR
alexcardell Feb 23, 2024
c596060
Remove resolvers
alexcardell Feb 23, 2024
88fa15a
Add otel4s TracedHandler tests
alexcardell Feb 27, 2024
11be471
Rework package structure
alexcardell Feb 27, 2024
08122de
prePR
alexcardell Feb 27, 2024
5bee0a3
Swap tests from SDK to oteljava
alexcardell Mar 12, 2024
6b80150
Run prePR
alexcardell Mar 12, 2024
5489668
Add back extra JVM test
alexcardell Mar 12, 2024
4ae91a8
Add OtelJava JVM example
alexcardell Mar 12, 2024
29f8248
Update otel4s to 0.6.0
alexcardell Apr 18, 2024
db56620
Simplify for-comp
alexcardell Apr 18, 2024
a737654
Fix headers
alexcardell Apr 18, 2024
5d0fb2b
Update scalaJS plugin
alexcardell Apr 18, 2024
9abdd5e
Remove semconv-experimental
alexcardell Apr 19, 2024
a961320
Add attributes for other events
alexcardell Apr 19, 2024
96ce0a9
Remove unused imports
alexcardell Apr 19, 2024
68ea923
Version bumps (to appease sbt 1.10)
alexcardell May 8, 2024
b2fcbf2
Add http4s-otel4s-middleware to example
alexcardell May 8, 2024
8f06d74
Fix import order
alexcardell May 8, 2024
4c1cbb3
use otel4s-sdk-trace-testkit to run tests across all platforms
iRevive May 10, 2024
2e0937b
Merge pull request #2 from iRevive/lambda-otel4s
alexcardell Sep 15, 2024
18768c2
Merge branch 'main' into lambda-otel4s
alexcardell Sep 15, 2024
51aad4a
Move TracedHandlerSuite to JVM
alexcardell Sep 15, 2024
a208070
Update workflows
alexcardell Sep 15, 2024
2d435fd
Swap module order
alexcardell Sep 15, 2024
ad3dbae
Fix TracedHandlerSuite class name
alexcardell Sep 15, 2024
fe22625
Add JS TracedHandler suite
alexcardell Sep 15, 2024
c8f2913
Remove unused imports
alexcardell Sep 15, 2024
e8fbf9d
Update base version
alexcardell Sep 15, 2024
501e76a
Update otel4s to 0.8
alexcardell Sep 15, 2024
5e05972
Lambda otel4s: test span attributes
iRevive Sep 16, 2024
a47540b
Merge pull request #3 from iRevive/lambda-otel4s
alexcardell Sep 16, 2024
81ce08d
Format
alexcardell Sep 16, 2024
84d303b
Swap List[Attribute[_]] to Attributes
alexcardell Sep 16, 2024
73ed59c
Remove unused import
alexcardell Sep 16, 2024
eefbc7d
Remove unused import
alexcardell Sep 16, 2024
b87ae6c
Add LambdaContextAttributes test
alexcardell Sep 16, 2024
1010972
Add LambdaMessageAttributesTest
alexcardell Sep 16, 2024
92dded7
Deduplicate examples module settings keys
alexcardell Sep 16, 2024
7687287
Format build.sbt
alexcardell Sep 16, 2024
d35d875
Make otel4s attribute utilities package private
alexcardell Sep 16, 2024
a6f8e6c
Remove _root_ from example imports
alexcardell Sep 16, 2024
684cb50
Fix example
alexcardell Sep 16, 2024
353ba48
Remove http4s-otel4s-middleware
alexcardell Sep 16, 2024
07d2bbe
Bump otel to 0.9
alexcardell Sep 16, 2024
20d7b5f
Fix 0.9 bump
alexcardell Sep 16, 2024
c52d901
Bump otel4s to 0.10
alexcardell Sep 25, 2024
c062551
Move KernelSources to KernelSource companion
alexcardell Sep 26, 2024
8ebf71c
Bump skunk again
alexcardell Sep 29, 2024
e538c3f
Remove Event parameter from TracedHandler
alexcardell Sep 29, 2024
4231bca
Use shared trace handler suite
alexcardell Sep 29, 2024
29edc99
Add v0.4.0 rewrites for natchez module
alexcardell Sep 29, 2024
9bcbb13
Fix formatting
alexcardell Sep 29, 2024
ef7d883
Add scalafix v0.4.0 rewrite input/output
alexcardell Sep 29, 2024
bb2970f
Split 0.3.0 and 0.4.0 rewrite rules into subprojects
alexcardell Sep 30, 2024
a99a656
Update workflows
alexcardell Sep 30, 2024
9b5d7d6
Fix scalafix 0.4.0 rewrites test
alexcardell Sep 30, 2024
ea168f5
Bump to scala 3.3.4
alexcardell Sep 30, 2024
9cca0f0
Remove 0.3.0 migrations
alexcardell Sep 30, 2024
c46d6d3
Bump sbt-lambda scalaVersion
alexcardell Sep 30, 2024
15d8feb
Remove unused examples imports
alexcardell Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,11 @@ jobs:

- name: Make target directories
if: github.event_name != 'pull_request' && (startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main')
run: mkdir -p lambda-cloudformation-custom-resource/.js/target lambda-http4s/.jvm/target unidocs/target lambda-http4s/.js/target lambda/js/target scalafix/rules/target lambda/jvm/target sbt-lambda/target google-cloud-http4s/jvm/target lambda-cloudformation-custom-resource/.jvm/target google-cloud-http4s/js/target project/target
run: mkdir -p lambda-natchez/jvm/target lambda-natchez/js/target lambda-cloudformation-custom-resource/.js/target lambda-http4s/.jvm/target unidocs/target lambda-http4s/.js/target lambda/js/target scalafix/rules/target lambda/jvm/target lambda-otel4s/jvm/target sbt-lambda/target lambda-otel4s/js/target google-cloud-http4s/jvm/target lambda-cloudformation-custom-resource/.jvm/target google-cloud-http4s/js/target project/target

- name: Compress target directories
if: github.event_name != 'pull_request' && (startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main')
run: tar cf targets.tar lambda-cloudformation-custom-resource/.js/target lambda-http4s/.jvm/target unidocs/target lambda-http4s/.js/target lambda/js/target scalafix/rules/target lambda/jvm/target sbt-lambda/target google-cloud-http4s/jvm/target lambda-cloudformation-custom-resource/.jvm/target google-cloud-http4s/js/target project/target
run: tar cf targets.tar lambda-natchez/jvm/target lambda-natchez/js/target lambda-cloudformation-custom-resource/.js/target lambda-http4s/.jvm/target unidocs/target lambda-http4s/.js/target lambda/js/target scalafix/rules/target lambda/jvm/target lambda-otel4s/jvm/target sbt-lambda/target lambda-otel4s/js/target google-cloud-http4s/jvm/target lambda-cloudformation-custom-resource/.jvm/target google-cloud-http4s/js/target project/target

- name: Upload target directories
if: github.event_name != 'pull_request' && (startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main')
Expand Down
56 changes: 44 additions & 12 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import com.typesafe.tools.mima.core._

name := "feral"

ThisBuild / tlBaseVersion := "0.3"
ThisBuild / tlBaseVersion := "0.4"
ThisBuild / startYear := Some(2021)

ThisBuild / developers := List(
Expand Down Expand Up @@ -46,8 +46,8 @@ ThisBuild / githubWorkflowBuildPreamble +=
)

val Scala212 = "2.12.19"
val Scala213 = "2.13.14"
val Scala3 = "3.3.3"
val Scala213 = "2.13.15"
val Scala3 = "3.3.4"
ThisBuild / crossScalaVersions := Seq(Scala212, Scala3, Scala213)

val catsEffectVersion = "3.5.4"
Expand All @@ -58,6 +58,7 @@ val natchezVersion = "0.3.6"
val munitVersion = "0.7.29"
val munitCEVersion = "1.0.7"
val scalacheckEffectVersion = "1.0.4"
val otel4sVersion = "0.10.0"

lazy val commonSettings = Seq(
crossScalaVersions := Seq(Scala3, Scala213)
Expand All @@ -69,6 +70,8 @@ lazy val root =
lambda,
lambdaHttp4s,
lambdaCloudFormationCustomResource,
lambdaNatchez,
lambdaOtel4s,
googleCloudHttp4s,
examples,
unidocs
Expand All @@ -89,7 +92,7 @@ lazy val lambda = crossProject(JSPlatform, JVMPlatform)
name := "feral-lambda",
libraryDependencies ++= Seq(
"org.typelevel" %%% "cats-effect" % catsEffectVersion,
"org.tpolecat" %%% "natchez-core" % natchezVersion,
"org.typelevel" %%% "case-insensitive" % "1.4.0",
"io.circe" %%% "circe-scodec" % circeVersion,
"io.circe" %%% "circe-jawn" % circeVersion,
"com.comcast" %%% "ip4s-core" % "3.6.0",
Expand Down Expand Up @@ -171,6 +174,34 @@ lazy val lambdaCloudFormationCustomResource = crossProject(JSPlatform, JVMPlatfo
.settings(commonSettings)
.dependsOn(lambda)

lazy val lambdaNatchez = crossProject(JSPlatform, JVMPlatform)
.in(file("lambda-natchez"))
.settings(
name := "feral-lambda-natchez",
libraryDependencies ++= Seq(
"org.tpolecat" %%% "natchez-core" % natchezVersion,
"org.scalameta" %%% "munit-scalacheck" % munitVersion % Test,
"org.typelevel" %%% "munit-cats-effect-3" % munitCEVersion % Test
)
)
.settings(commonSettings)
.dependsOn(lambda)

lazy val lambdaOtel4s = crossProject(JSPlatform, JVMPlatform)
.in(file("lambda-otel4s"))
.settings(
name := "feral-lambda-otel4s",
libraryDependencies ++= Seq(
"org.typelevel" %%% "otel4s-core-trace" % otel4sVersion,
"org.typelevel" %%% "otel4s-sdk-trace-testkit" % otel4sVersion % Test,
"org.typelevel" %%% "otel4s-semconv-experimental" % otel4sVersion % Test,
Copy link
Member

@armanbilge armanbilge Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experimental (incubating) semantic conventions. Breaking changes expected. Library instrumentation SHOULD NOT depend on this.

@iRevive should we be concerned? Gah, it's Test, I'm sorry. It's all fine :)

"org.scalameta" %%% "munit-scalacheck" % munitVersion % Test,
"org.typelevel" %%% "munit-cats-effect-3" % munitCEVersion % Test
)
)
.settings(commonSettings)
.dependsOn(lambda)

lazy val examples = crossProject(JSPlatform, JVMPlatform)
.in(file("examples"))
.settings(
Expand All @@ -183,17 +214,17 @@ lazy val examples = crossProject(JSPlatform, JVMPlatform)
)
)
.settings(commonSettings)
.dependsOn(lambda, lambdaHttp4s, googleCloudHttp4s)
.dependsOn(lambda, lambdaHttp4s, lambdaNatchez, lambdaOtel4s, googleCloudHttp4s)
.jvmSettings(libraryDependencies ++= Seq(
"org.typelevel" %%% "otel4s-oteljava" % otel4sVersion,
"io.opentelemetry" % "opentelemetry-sdk-extension-autoconfigure" % "1.34.1",
"com.google.cloud.functions.invoker" % "java-function-invoker" % "1.3.1"
))
.jsSettings(
scalaJSUseMainModuleInitializer := true,
Compile / mainClass := Some("feral.examples.http4sGoogleCloudHandler"),
scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }
)
.jvmSettings(
libraryDependencies ++= Seq(
"com.google.cloud.functions.invoker" % "java-function-invoker" % "1.3.1"
)
)
.enablePlugins(NoPublishPlugin)

lazy val unidocs = project
Expand All @@ -214,22 +245,23 @@ lazy val unidocs = project
)

lazy val scalafix = tlScalafixProject
.in(file("scalafix"))
.rulesSettings(
name := "feral-scalafix",
startYear := Some(2023),
crossScalaVersions := Seq(Scala212)
)
.inputSettings(
crossScalaVersions := Seq(Scala213),
libraryDependencies += "org.typelevel" %%% "feral-lambda-http4s" % "0.2.4",
libraryDependencies += "org.typelevel" %%% "feral-lambda" % "0.3.1",
headerSources / excludeFilter := AllPassFilter
)
.inputConfigure(_.disablePlugins(ScalafixPlugin))
.outputSettings(
crossScalaVersions := Seq(Scala213),
headerSources / excludeFilter := AllPassFilter
)
.outputConfigure(_.dependsOn(lambdaHttp4s.jvm).disablePlugins(ScalafixPlugin))
.outputConfigure(_.dependsOn(lambdaNatchez.jvm).disablePlugins(ScalafixPlugin))
.testsSettings(
startYear := Some(2023),
crossScalaVersions := Seq(Scala212)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
* Copyright 2021 Typelevel
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package feral.examples

import cats.Monad
import cats.effect.IO
import cats.syntax.all._
import feral.lambda.INothing
import feral.lambda.IOLambda
import feral.lambda.Invocation
import feral.lambda.events.SqsEvent
import feral.lambda.events.SqsRecord
import feral.lambda.otel4s._
import org.http4s.client.Client
import org.http4s.ember.client.EmberClientBuilder
import org.typelevel.otel4s.oteljava.OtelJava
import org.typelevel.otel4s.trace.Tracer
import org.typelevel.scalaccompat.annotation.unused

object SqsOtelExample extends IOLambda[SqsEvent, INothing] {

// Could be http4s-otel4s-middleware ClientMiddleware for example
def clientMiddleware(client: Client[IO]): Client[IO] = client

def handler =
OtelJava.autoConfigured[IO]().map(_.tracerProvider).evalMap(_.get("tracer")).flatMap {
implicit tracer: Tracer[IO] =>
for {
client <- EmberClientBuilder.default[IO].build
tracedClient = clientMiddleware(client)
} yield { implicit inv: Invocation[IO, SqsEvent] =>
TracedHandler[IO, SqsEvent, INothing](
handleEvent[IO](tracedClient)
)
}
}

def handleEvent[F[_]: Monad: Tracer](
@unused client: Client[F]
)(implicit inv: Invocation[F, SqsEvent]): F[Option[INothing]] = inv.event.flatMap { event =>
event
.records
.traverse(record =>
Tracer[F].span("handle-record", SqsRecordAttributes(record)).surround {
handleRecord[F](record)
})
.as(None)
}

def handleRecord[F[_]: Monad: Tracer](@unused record: SqsRecord): F[Unit] = {
Tracer[F].span("some-operation").surround {
Monad[F].unit
}
}

}
13 changes: 7 additions & 6 deletions examples/shared/src/main/scala/feral/examples/Http4sLambda.scala
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,15 @@

package feral.examples

import _root_.feral.lambda._
import _root_.feral.lambda.events._
import _root_.feral.lambda.http4s._
import _root_.feral.lambda.natchez._
import _root_.natchez.Trace
import _root_.natchez.http4s.NatchezMiddleware
import _root_.natchez.xray.XRay
import cats.effect._
import cats.effect.std.Random
import feral.lambda._
import feral.lambda.events._
import feral.lambda.http4s._
import natchez.Trace
import natchez.http4s.NatchezMiddleware
import natchez.xray.XRay
import org.http4s.HttpApp
import org.http4s.HttpRoutes
import org.http4s.client.Client
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@

package feral.examples

import _root_.feral.lambda._
import _root_.feral.lambda.events.SqsEvent
import _root_.natchez.Trace
import _root_.natchez.xray.XRay
import cats.effect._
import cats.effect.std.Random
import feral.lambda._
import feral.lambda.events.SqsEvent
import natchez.Trace
import natchez.xray.XRay
import skunk.Session

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package feral.lambda
package feral.lambda.natchez
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add Scalafix migrations for these renames?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay so I've added inputs and outputs but what's the common practice for adding the new v0.3.0 dependency, to run those tests? Do we require a 0.2->0.3 subproject and a 0.3->0.4 subproject?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the examples in cats have subprojects per version, I will work on that later

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of more subprojects, I wonder if we can just delete / replace the old migrations with the new ones since we are doing a version bump? I would prefer not to maintain them forever esp. if Scalafix/Scalameta may issue deprecations in the future 🤔

These migrations are already configured in Scala Steward with a hard-coded version.

https://github.com/scala-steward-org/scala-steward/blob/0765eec5afc158c2e30f5740b4aa23fc7ae0b7dd/modules/core/src/main/resources/scalafix-migrations.conf#L250-L255

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, well I added the second subproject (plus shuffling the old one into a nested directory) before reading that

If it can be deleted, great!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed


import natchez.TraceValue

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Copyright 2021 Typelevel
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package feral.lambda.natchez

import feral.lambda.events.ApiGatewayProxyEvent
import feral.lambda.events.ApiGatewayProxyEventV2
import feral.lambda.events.DynamoDbStreamEvent
import feral.lambda.events.KinesisStreamEvent
import feral.lambda.events.S3BatchEvent
import feral.lambda.events.SqsRecordAttributes
import natchez.Kernel
import org.typelevel.ci._

trait KernelSource[Event] {
def extract(event: Event): Kernel
}

object KernelSource {
@inline def apply[E](implicit ev: KernelSource[E]): ev.type = ev

def emptyKernelSource[E]: KernelSource[E] = _ => Kernel(Map.empty)

private[this] val `X-Amzn-Trace-Id` = ci"X-Amzn-Trace-Id"

implicit def apiGatewayProxyEvent: KernelSource[ApiGatewayProxyEvent] =
e => Kernel(e.headers.getOrElse(Map.empty))

implicit def apiGatewayProxyEventV2: KernelSource[ApiGatewayProxyEventV2] =
e => Kernel(e.headers)

implicit def sqsRecordAttributes: KernelSource[SqsRecordAttributes] =
a => Kernel(a.awsTraceHeader.map(`X-Amzn-Trace-Id` -> _).toMap)

implicit def s3BatchEvent: KernelSource[S3BatchEvent] = KernelSource.emptyKernelSource

@deprecated(
"See feral.lambda.events.KinesisStreamEvent deprecation",
since = "0.3.0"
)
implicit def kinesisStreamEvent: KernelSource[KinesisStreamEvent] =
KernelSource.emptyKernelSource

implicit def dynamoDbStreamEvent: KernelSource[DynamoDbStreamEvent] =
KernelSource.emptyKernelSource
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@
* limitations under the License.
*/

package feral.lambda
package feral.lambda.natchez

import cats.data.Kleisli
import cats.effect.IO
import cats.effect.kernel.MonadCancelThrow
import cats.syntax.all._
import feral.lambda.Invocation
import natchez.EntryPoint
import natchez.Span
import natchez.Trace
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,20 @@
* limitations under the License.
*/

package feral.lambda
package feral.lambda.natchez

import cats.data.Kleisli
import cats.effect.IO
import feral.lambda.INothing
import feral.lambda.Invocation
import feral.lambda.events.KinesisStreamEvent
import natchez.EntryPoint
import natchez.Span
import natchez.Trace

import scala.annotation.nowarn

class TracedLambdaSuite {
class TracedHandlerSuite {

@nowarn
def syntaxTest = { // Checking for compilation, nothing more
Expand Down
Loading