Add lambda-otel4s module #459

alexcardell · 2024-02-23T17:19:10Z

Supercedes #456

Adds lambda-natchez module
Adds lambda-otel4s module
- Traces lambdas with semantic conventions applied
Change examples crosstype to add OtelJava example
(WIP) Add scalafix migration to port natchez package imports

Discussion

There might be some extra utilities worth exposing to make dealing with batched events (like SQS events) easier -- most likely the context the user wants is on the Record, not the batched Event, but this is pretty equivalent to the existing Natchez module

This is targeting otel4s 0.10.0

Links

https://opentelemetry.io/docs/specs/semconv/faas/aws-lambda/
https://opentelemetry.io/docs/specs/semconv/faas/faas-spans/

alexcardell · 2024-02-27T18:59:29Z

Currently pointed at a local snapshot so CI will definitely fail, as otel4s main has NoPublish for everything involving the SDK

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/FaasAttributes.scala

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/TracedHandler.scala

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/package.scala

alexcardell · 2024-04-19T17:37:34Z

@iRevive Updated all the attributes and removed semconv-experimental

Would appreciate another look when you have time

iRevive · 2024-04-20T07:10:03Z

@alexcardell the changes look good.

We still need to wait for a release of https://github.com/http4s/http4s-otel4s-middleware, right?

alexcardell · 2024-04-22T08:49:10Z

The middleware would be useful for the example, but I don't believe it's necessary otherwise

This is to allow consumers the choice of natchez or otel4s No breaking changes except requiring a `feral.lambda.natchez._` import

Co-authored-by: Maksym Ochenashko <[email protected]>

alexcardell · 2024-09-16T18:37:14Z

Fixed the 0.9 issues, good to go

armanbilge · 2024-09-21T07:50:12Z

There are also some AWS goodies in the upcoming 0.10.0 otel4s release (for otel4s-sdk):

@iRevive is v0.10.0 coming soon?

armanbilge · 2024-09-21T07:51:57Z

lambda-natchez/shared/src/main/scala/feral/lambda/natchez/package.scala

+import feral.lambda.natchez.KernelSource
+import org.typelevel.ci._
+
+protected trait KernelSources {


Instead of putting these in the package object, we can put them in the companion object for KernelSource. Then they will automatically be in implicit scope without an import.

@alexcardell Can you look into this as well? 🙏

@armanbilge done :)

iRevive · 2024-09-22T08:19:10Z

There are also some AWS goodies in the upcoming 0.10.0 otel4s release (for otel4s-sdk):

@iRevive is v0.10.0 coming soon?

We are two PRs away from 0.10.0:

Both are implemented, some minor changes (docs, etc) may be required.

I think we will have 0.10.0 next week.

armanbilge · 2024-09-24T06:14:15Z

I am going hang on for otel4s v0.10.0 to avoid back-to-back breaking releases!

iRevive · 2024-09-24T17:01:13Z

0.10.0 is out. It should be a straightforward update.

armanbilge · 2024-09-24T17:06:31Z

so fast!!

alexcardell · 2024-09-25T09:31:28Z

Updated

armanbilge · 2024-09-27T15:43:06Z

build.sbt

+ libraryDependencies ++= Seq(
+ "org.typelevel" %%% "otel4s-core-trace" % otel4sVersion,
+ "org.typelevel" %%% "otel4s-sdk-trace-testkit" % otel4sVersion % Test,
+ "org.typelevel" %%% "otel4s-semconv-experimental" % otel4sVersion % Test,


Experimental (incubating) semantic conventions. Breaking changes expected. Library instrumentation SHOULD NOT depend on this.

~~@iRevive should we be concerned?~~ Gah, it's Test, I'm sorry. It's all fine :)

armanbilge · 2024-09-27T15:44:25Z

build.sbt

- "org.tpolecat" %%% "skunk-core" % "0.6.4"
+ "org.tpolecat" %%% "skunk-core" % "0.6.3"


Why is this rolled back?

The answer to this question is lost to time

If there was a reason I've forgotten, but it's probably a mistake

Bumped back

armanbilge · 2024-09-27T16:12:53Z

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/TracedHandler.scala

+object TracedHandler {
+
+ def apply[F[_]: Monad: Tracer, Event, Result](
+ handler: Event => F[Option[Result]]


Suggested change

handler: Event => F[Option[Result]]

handler: F[Option[Result]]

We don't need to pass the event explicitly, because the user obtains it from the Invocation.

armanbilge · 2024-09-27T16:13:22Z

lambda-natchez/shared/src/main/scala/feral/lambda/natchez/AwsTags.scala

-package feral.lambda
+package feral.lambda.natchez


Can we add Scalafix migrations for these renames?

Okay so I've added inputs and outputs but what's the common practice for adding the new v0.3.0 dependency, to run those tests? Do we require a 0.2->0.3 subproject and a 0.3->0.4 subproject?

I see the examples in cats have subprojects per version, I will work on that later

Instead of more subprojects, I wonder if we can just delete / replace the old migrations with the new ones since we are doing a version bump? I would prefer not to maintain them forever esp. if Scalafix/Scalameta may issue deprecations in the future 🤔

These migrations are already configured in Scala Steward with a hard-coded version.

https://github.com/scala-steward-org/scala-steward/blob/0765eec5afc158c2e30f5740b4aa23fc7ae0b7dd/modules/core/src/main/resources/scalafix-migrations.conf#L250-L255

Ah, well I added the second subproject (plus shuffling the old one into a nested directory) before reading that

If it can be deleted, great!

armanbilge · 2024-09-27T16:15:25Z

lambda-otel4s/jvm/src/test/scala/feral/lambda/otel4s/TracedHandlerSuite.scala

+import java.io.ByteArrayOutputStream
+import java.util.concurrent.atomic.AtomicInteger
+
+class TracedHandlerSuite extends CatsEffectSuite {


Can we avoid the substantial duplication/splitting of these suite? It looks like you based these on some tests that needed extra machinery because they were testing the platform-specific integration. But I think in your situation it should be possible to test TracedHandler using pure code.

Sure thing, makes sense

This was referenced Feb 23, 2024

Extract lambda-natchez module #456

Closed

Integrate feral-lambda with otel4s #340

Open

alexcardell force-pushed the lambda-otel4s branch from 6b6999d to 4242256 Compare February 27, 2024 18:32

alexcardell marked this pull request as ready for review March 12, 2024 19:21

alexcardell changed the title ~~WIP: Add lambda-otel4s module~~ Add lambda-otel4s module Mar 12, 2024

iRevive reviewed Mar 31, 2024

View reviewed changes

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/FaasAttributes.scala Outdated Show resolved Hide resolved

iRevive reviewed Mar 31, 2024

View reviewed changes

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/TracedHandler.scala Outdated Show resolved Hide resolved

iRevive reviewed Mar 31, 2024

View reviewed changes

lambda-otel4s/shared/src/main/scala/feral/lambda/otel4s/package.scala Outdated Show resolved Hide resolved

alexcardell and others added 18 commits May 8, 2024 10:25

Extract lambda-natchez module

1170ca8

This is to allow consumers the choice of natchez or otel4s No breaking changes except requiring a `feral.lambda.natchez._` import

Remove natchez-core dependency from lambda module

09d4127

Add lambda-otel4s module

375b814

Use SqsEvent for attributes, not SqsRecord

7ebde29

Add otel4s-sdk example

a5506a4

re-run prePR

a17882b

Remove resolvers

c596060

Add otel4s TracedHandler tests

88fa15a

Rework package structure

11be471

prePR

08122de

Swap tests from SDK to oteljava

5bee0a3

Run prePR

6b80150

Add back extra JVM test

5489668

Add OtelJava JVM example

4ae91a8

Update otel4s to 0.6.0

29f8248

Simplify for-comp

db56620

Co-authored-by: Maksym Ochenashko <[email protected]>

Fix headers

a737654

Update scalaJS plugin

5d0fb2b

Fix 0.9 bump

20d7b5f

armanbilge reviewed Sep 21, 2024

View reviewed changes

Bump otel4s to 0.10

c52d901

Move KernelSources to KernelSource companion

c062551

alexcardell requested a review from armanbilge September 26, 2024 15:27

armanbilge reviewed Sep 27, 2024

View reviewed changes

alexcardell added 5 commits September 29, 2024 11:31

Bump skunk again

8ebf71c

Remove Event parameter from TracedHandler

e538c3f

Use shared trace handler suite

4231bca

Add v0.4.0 rewrites for natchez module

29edc99

Fix formatting

9bcbb13

alexcardell requested a review from armanbilge September 29, 2024 15:25

alexcardell added 2 commits September 29, 2024 16:31

Add scalafix v0.4.0 rewrite input/output

ef7d883

Split 0.3.0 and 0.4.0 rewrite rules into subprojects

bb2970f

alexcardell force-pushed the lambda-otel4s branch from eb990d6 to bb2970f Compare September 30, 2024 16:19

alexcardell added 6 commits September 30, 2024 17:22

Update workflows

a99a656

Fix scalafix 0.4.0 rewrites test

9b5d7d6

Bump to scala 3.3.4

ea168f5

Remove 0.3.0 migrations

9cca0f0

Bump sbt-lambda scalaVersion

c46d6d3

Remove unused examples imports

15d8feb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add lambda-otel4s module #459

Add lambda-otel4s module #459

alexcardell commented Feb 23, 2024 •

edited

Loading

alexcardell commented Feb 27, 2024

alexcardell commented Apr 19, 2024

iRevive commented Apr 20, 2024

alexcardell commented Apr 22, 2024

alexcardell commented Sep 16, 2024

armanbilge commented Sep 21, 2024

armanbilge Sep 21, 2024

armanbilge Sep 25, 2024

alexcardell Sep 26, 2024

alexcardell Sep 26, 2024

iRevive commented Sep 22, 2024

armanbilge commented Sep 24, 2024

iRevive commented Sep 24, 2024

armanbilge commented Sep 24, 2024

alexcardell commented Sep 25, 2024

armanbilge Sep 27, 2024 •

edited

Loading

armanbilge Sep 27, 2024

alexcardell Sep 29, 2024 •

edited

Loading

armanbilge Sep 27, 2024

alexcardell Sep 29, 2024

armanbilge Sep 27, 2024

alexcardell Sep 29, 2024

alexcardell Sep 29, 2024

alexcardell Sep 30, 2024

armanbilge Sep 30, 2024

alexcardell Sep 30, 2024

alexcardell Sep 30, 2024

armanbilge Sep 27, 2024

alexcardell Sep 29, 2024

		"org.tpolecat" %%% "skunk-core" % "0.6.4"
		"org.tpolecat" %%% "skunk-core" % "0.6.3"

	handler: Event => F[Option[Result]]
	handler: F[Option[Result]]

Add lambda-otel4s module #459

Are you sure you want to change the base?

Add lambda-otel4s module #459

Conversation

alexcardell commented Feb 23, 2024 • edited Loading

Discussion

Links

alexcardell commented Feb 27, 2024

alexcardell commented Apr 19, 2024

iRevive commented Apr 20, 2024

alexcardell commented Apr 22, 2024

alexcardell commented Sep 16, 2024

armanbilge commented Sep 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iRevive commented Sep 22, 2024

armanbilge commented Sep 24, 2024

iRevive commented Sep 24, 2024

armanbilge commented Sep 24, 2024

alexcardell commented Sep 25, 2024

armanbilge Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcardell Sep 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcardell commented Feb 23, 2024 •

edited

Loading

armanbilge Sep 27, 2024 •

edited

Loading

alexcardell Sep 29, 2024 •

edited

Loading