Feature/refactor and support dashboard for baker (#1304)

* - Renamed BakeryExecutorJava to BakerResultToHttpResponseMapper - For BakerService and BakerResultToHttpResponseMapper, the 'callBaker' method is renamed to 'toBakerResultResponseIO' and 'toBakerResult' respectively. This more accurately describes what the method does. - Renamed BakerService to Http4sBakerServer - Split up bakery-state into bakery-state and baker-http-server. The idea is that baker-http-server can be used to expose baker endpoints (which can be then used for bakery but also for non-bakery APIs to use for the dashboard for example). - Moved baker-client, baker-server and dashboard from the bakery directory to http. * - Refactor BakerResultToResponseMapper into a separate InstanceResponseMapper. * Update dashboard project: - Static files are now added as resources in new jar artifact. - Added com.ing.baler.http.Dashboard object to safely access these static files. - Updated Http4sBakerServer to use this new way of accessing the dashboard. - Replaced dashboard-path property with enable-dashboard to reflect these changes. - No longer separately copy dashboard files into docker image. * - Cache npm build. * - Added documentation for the caching of the npmBuildTask - Added the distDirectory as something to be cleaned with clean command. * - Refactored Http4sBakerServer to have a routes method available seperately from the resource. - Created Htpp4sBakerServerSpec to test routes - Created a dashboard_config endpoint, since the settings.json is no longer available (that is now configured in the baker.dashboard property). - Enable publishing of the source code. * - Fixed structure of versionJson. - Fixed serving static files not working. * - Seperated Http4sBakerServer configuration loading into Http4sBakerServerConfiguration class. - Adding server specific configuration in reference.conf in server project. - Created a java-friendly constructor for the Http4sBakerServer. * - Reverted the field name to what it was, since it sometimes is accessed through reflection, even though it is private. - Instead made a new accessor so that this is not required anymore. * - Added some documentation to the 3 main subdirectories * - Updated packages in configuration. * - Fixed data model of recipe response. - Added some comments for fixing test issues. * - Add all routes to http4sbakerserver * - Make matching scala.2.12 compatible. * - Updated code based on review comments. - Added example code to main project, so they are run and compiled after changes. - Fixed example projects not working correctly. - Fixed test cases in baker-example not closing actor system correctly, causing them to only be able to run separately. * - Add noPublish to examples. * Fixed the BuildExampleDocker command. * - Try with parallel execution off. * Changed WebShop recipe test to use inmemory baker to have more stable tests. * - Remove event sink from test resource conf. Co-authored-by: Wessel W. Bakker <[email protected]> Co-authored-by: XK00LJ <[email protected]>
ing-bank · Aug 9, 2022 · 5bbcf00 · 5bbcf00
1 parent fad540e
commit 5bbcf00
Show file tree

Hide file tree

Showing 141 changed files with 1,075 additions and 624 deletions.
diff --git a/.gitignore b/.gitignore
@@ -76,4 +76,4 @@ testem.log
 # System Files
 .DS_Store
 Thumbs.db
-/bakery/dashboard/dashboard.zip
+/http/baker-http-dashboard/dashboard.zip
diff --git a/README.md b/README.md
@@ -74,16 +74,16 @@ Applying Baker will only be successful if you make sure that:
 To get started with SBT, simply add the following to your build.sbt file:
 
 ```
-libraryDependencies += "com.ing.baker" %% "baker-recipe-dsl" % "3.0.3"
-libraryDependencies += "com.ing.baker" %% "baker-runtime" % "3.0.3"
-libraryDependencies += "com.ing.baker" %% "baker-compiler" % "3.0.3"
+libraryDependencies += "com.ing.baker" %% "baker-recipe-dsl" % version
+libraryDependencies += "com.ing.baker" %% "baker-runtime" % version
+libraryDependencies += "com.ing.baker" %% "baker-compiler" % version
 ```
 
 From 1.3.x to 2.0.x we cross compile to both Scala 2.11 and 2.12. 
 
 Earlier releases are only available for Scala 2.11.
 
-From 3.0.x we support only Scala 2.12.
+From 3.x.x we support only Scala 2.12.
 
 # How to contribute?
 

diff --git a/bakery/README.MD b/bakery/README.MD
@@ -0,0 +1,9 @@
+The `bakery` directory contains modules for setting up a bakery cluster.
+
+Bakery is a way to host a baker service using akka-runtime and expose it using API endpoints. This allows you to separate
+the running of an akka cluster and managing of the persistence from using baker for your business process.
+
+A bakery team can provide baker clusters as a service (which requires specific knowledge about akka clusters).
+A client team can focus on creating recipes based on actual business processes.
+
+The `bakery-state` module is the entry point for bakery.
diff --git a/bakery/client/src/test/java/com/ing/bakery/client/JBakerClientSpec.java b/bakery/client/src/test/java/com/ing/bakery/client/JBakerClientSpec.java
diff --git a/bakery/dashboard/src/app/home/home.component.html b/bakery/dashboard/src/app/home/home.component.html
diff --git a/bakery/dashboard/src/assets/settings/settings.json b/bakery/dashboard/src/assets/settings/settings.json
diff --git a/bakery/dashboard/src/proxy.conf.json b/bakery/dashboard/src/proxy.conf.json
diff --git a/bakery/integration-tests/src/test/resources/kubernetes/webshop-baker.yaml b/bakery/integration-tests/src/test/resources/kubernetes/webshop-baker.yaml
@@ -114,7 +114,7 @@ data:
     baker {
       recipe-poll-interval: 5 seconds
       event-sink {
-        class: "com.ing.bakery.baker.KafkaEventSink",
+        class: "com.ing.bakery.components.KafkaEventSink",
         bootstrap-servers: "kafka-event-sink:9092",
         topic: "events"
       }

diff --git a/...ate-k8s/src/main/resources/reference.conf → ...manager/src/main/resources/reference.conf b/...ate-k8s/src/main/resources/reference.conf → ...manager/src/main/resources/reference.conf
diff --git a/...bakery/baker/KubernetesInteractions.scala → ...bakery/baker/KubernetesInteractions.scala b/...bakery/baker/KubernetesInteractions.scala → ...bakery/baker/KubernetesInteractions.scala
@@ -7,6 +7,7 @@ import akka.{Done, NotUsed}
 import cats.effect.{ContextShift, IO, Resource, Timer}
 import cats.implicits.catsSyntaxApplicativeError
 import com.ing.baker.runtime.akka.internal.DynamicInteractionManager
+import com.ing.bakery.components.RemoteInteractionDiscovery
 import com.ing.bakery.interaction.RemoteInteractionClient
 import com.typesafe.config.Config
 import com.typesafe.scalalogging.LazyLogging

diff --git a/...m/ing/bakery/baker/StateRuntimeSpec.scala → ...m/ing/bakery/baker/StateRuntimeSpec.scala b/...m/ing/bakery/baker/StateRuntimeSpec.scala → ...m/ing/bakery/baker/StateRuntimeSpec.scala
@@ -2,19 +2,23 @@ package com.ing.bakery.baker
 
 import akka.actor.ActorSystem
 import cats.effect.{IO, Resource}
+import com.ing.baker.http.DashboardConfiguration
+import com.ing.baker.http.client.scaladsl.BakerClient
+import com.ing.baker.http.server.common.RecipeLoader
+import com.ing.baker.http.server.scaladsl.Http4sBakerServer
 import com.ing.baker.il.CompiledRecipe
 import com.ing.baker.runtime.akka.{AkkaBaker, AkkaBakerConfig}
 import com.ing.baker.runtime.common.BakerException.NoSuchProcessException
 import com.ing.baker.runtime.common.{BakerException, SensoryEventStatus}
 import com.ing.baker.runtime.model.{InteractionInstance, InteractionManager}
 import com.ing.baker.runtime.scaladsl.{Baker, EventInstance, InteractionInstanceInput}
 import com.ing.baker.types._
-import com.ing.bakery.baker.mocks.KubeApiServer
+import com.ing.bakery.components.InteractionRegistry
 import com.ing.bakery.mocks.{EventListener, RemoteInteraction}
 import com.ing.bakery.recipe.Events.{ItemsReserved, OrderPlaced}
 import com.ing.bakery.recipe.Ingredients.{Item, OrderId, ReservedItems}
 import com.ing.bakery.recipe.{ItemReservationRecipe, SimpleRecipe, SimpleRecipe2}
-import com.ing.bakery.scaladsl.BakerClient
+import com.ing.bakery.baker.mocks.KubeApiServer
 import com.ing.bakery.testing.BakeryFunSpec
 import com.typesafe.config.ConfigFactory
 import org.mockserver.integration.ClientAndServer
@@ -474,7 +478,13 @@ class StateRuntimeSpec extends BakeryFunSpec with Matchers {
       _ <- Resource.eval(eventListener.eventSink.attach(baker))
       _ <- Resource.eval(RecipeLoader.loadRecipesIntoBaker(getResourceDirectoryPathSafe, baker))
 
-      server <- BakerService.resource(baker, executionContext, InetSocketAddress.createUnresolved("127.0.0.1", 0), "/api/bakery", "/opt/docker/dashboard", loggingEnabled = true)
+      server <- Http4sBakerServer.resource(
+        baker,
+        executionContext,
+        InetSocketAddress.createUnresolved("127.0.0.1", 0),
+        apiUrlPrefix = "/api/bakery",
+        dashboardConfiguration = DashboardConfiguration(enabled = true, applicationName = "StateRuntimeSpec", clusterInformation = Map.empty),
+        loggingEnabled = true)
       client <- BakerClient.resource(server.baseUri, "/api/bakery", executionContext)
 
     } yield Context(

diff --git a/...akery/baker/TestInteractionRegistry.scala → ...akery/baker/TestInteractionRegistry.scala b/...akery/baker/TestInteractionRegistry.scala → ...akery/baker/TestInteractionRegistry.scala
@@ -4,6 +4,7 @@ import akka.actor.ActorSystem
 import cats.effect.{IO, Resource}
 import com.ing.baker.runtime.model.InteractionManager
 import com.ing.bakery.baker.mocks.KubeApiServer
+import com.ing.bakery.components.{BaseInteractionRegistry, LocalhostInteractions}
 import com.ing.bakery.interaction.BaseRemoteInteractionClient
 import com.ing.bakery.mocks.RemoteInteraction
 import com.typesafe.config.{Config, ConfigValueFactory}

diff --git a/...ng/bakery/baker/mocks/KubeApiServer.scala → ...ng/bakery/baker/mocks/KubeApiServer.scala b/...ng/bakery/baker/mocks/KubeApiServer.scala → ...ng/bakery/baker/mocks/KubeApiServer.scala
diff --git a/...m/ing/bakery/baker/mocks/WatchEvent.scala → ...m/ing/bakery/baker/mocks/WatchEvent.scala b/...m/ing/bakery/baker/mocks/WatchEvent.scala → ...m/ing/bakery/baker/mocks/WatchEvent.scala
diff --git a/bakery/state/src/it/resources/application.conf b/bakery/state/src/it/resources/application.conf
@@ -43,10 +43,9 @@ baker {
     localhost.ports = [ 8081 ]
   }
   event-sink {
-    class: ""
+    class = ""
   }
   api-logging-enabled = false
-  dashboard-path = "/Users/hr29bv/work/baker/bakery/dashboard/dist"
 }
 
 inmemory-read-journal {

diff --git a/bakery/state/src/main/resources/reference.conf b/bakery/state/src/main/resources/reference.conf
@@ -9,9 +9,9 @@ baker {
   encryption.enabled = off
   metrics-port = 9095
 
+  api-host = "0.0.0.0"
   api-port = 8080
   api-url-prefix = "/api/bakery"
-  dashboard-path = "/opt/docker/dashboard"
   api-logging-enabled = false
 
   bake-timeout = 30 seconds
@@ -29,7 +29,7 @@ baker {
   }
 
   interactions {
-    class = "com.ing.bakery.baker.BaseInteractionRegistry"
+    class = "com.ing.bakery.components.BaseInteractionRegistry"
     localhost {
       port = 8081
       api-url-prefix = "/api/bakery/interactions"
@@ -155,10 +155,10 @@ akka {
       liveness-path = "health/alive"
       liveness-checks {
         cluster-health = "akka.sensors.ClusterHealthCheck"
-        name = "com.ing.bakery.baker.WatcherReadinessCheck"
+        name = "com.ing.bakery.components.WatcherReadinessCheck"
       }
       readiness-checks {
-        name = "com.ing.bakery.baker.BakerReadinessCheck"
+        name = "com.ing.bakery.components.BakerReadinessCheck"
       }
     }
   }

diff --git a/...n/scala/com/ing/bakery/baker/Bakery.scala → ...rc/main/scala/com/ing/bakery/Bakery.scala b/...n/scala/com/ing/bakery/baker/Bakery.scala → ...rc/main/scala/com/ing/bakery/Bakery.scala
@@ -1,4 +1,4 @@
-package com.ing.bakery.baker
+package com.ing.bakery
 
 import akka.actor.ActorSystem
 import akka.cluster.Cluster
@@ -7,6 +7,7 @@ import com.ing.baker.runtime.akka.{AkkaBaker, AkkaBakerConfig}
 import com.ing.baker.runtime.model.InteractionManager
 import com.ing.baker.runtime.recipe_manager.{ActorBasedRecipeManager, RecipeManager}
 import com.ing.baker.runtime.scaladsl.Baker
+import com.ing.bakery.components.{Cassandra, EventSink, InteractionRegistry, Watcher}
 import com.typesafe.config.{Config, ConfigFactory}
 import com.typesafe.scalalogging.LazyLogging
 import io.prometheus.client.CollectorRegistry
@@ -15,16 +16,16 @@ import org.http4s.metrics.prometheus.Prometheus
 import java.io.File
 import scala.concurrent.ExecutionContext
 
-case class Bakery(baker: Baker,
-                 executionContext: ExecutionContext,
-                 system: ActorSystem)
+case class AkkaBakery(baker: Baker, system: ActorSystem) {
+  def executionContext: ExecutionContext = system.dispatcher
+}
 
 object Bakery extends LazyLogging {
 
-  def resource(optionalConfig: Option[Config],
-               externalContext: Option[Any] = None,
-               interactionManager: Option[InteractionManager[IO]] = None,
-               recipeManager: Option[RecipeManager] = None) : Resource[IO, Bakery] = {
+  def akkaBakery(optionalConfig: Option[Config],
+                 externalContext: Option[Any] = None,
+                 interactionManager: Option[InteractionManager[IO]] = None,
+                 recipeManager: Option[RecipeManager] = None) : Resource[IO, AkkaBakery] = {
     val configPath = sys.env.getOrElse("CONFIG_DIRECTORY", "/opt/docker/conf")
     val config = optionalConfig.getOrElse(ConfigFactory.load(ConfigFactory.parseFile(new File(s"$configPath/application.conf"))))
     val bakerConfig = config.getConfig("baker")
@@ -55,9 +56,7 @@ object Bakery extends LazyLogging {
         bakerActorProvider = AkkaBakerConfig.bakerProviderFrom(config),
         timeouts = AkkaBakerConfig.Timeouts.apply(config),
         bakerValidationSettings = AkkaBakerConfig.BakerValidationSettings.from(config))(system))
-      _ <- Resource.make(IO {
-        baker
-      })(baker => IO.fromFuture(IO(baker.gracefulShutdown())))
+      _ <- Resource.make(IO {baker})(baker => IO.fromFuture(IO(baker.gracefulShutdown())))
       _ <- Resource.eval(eventSink.attach(baker))
       _ <- Resource.eval(IO.async[Unit] { callback =>
         //If using local Baker the registerOnMemberUp is never called, should onl be used during local testing.
@@ -70,7 +69,7 @@ object Bakery extends LazyLogging {
           }
       })
 
-    } yield Bakery(baker, system.dispatcher, system)
+    } yield AkkaBakery(baker, system)
   }
 }
 
diff --git a/...com/ing/bakery/baker/ClosableBakery.scala → ...scala/com/ing/bakery/ClosableBakery.scala b/...com/ing/bakery/baker/ClosableBakery.scala → ...scala/com/ing/bakery/ClosableBakery.scala
@@ -1,34 +1,31 @@
-package com.ing.bakery.baker
+package com.ing.bakery
 
-import java.io.Closeable
 import akka.actor.ActorSystem
 import cats.effect.IO
 import com.ing.baker.runtime.model.InteractionManager
 import com.ing.baker.runtime.recipe_manager.RecipeManager
 import com.ing.baker.runtime.scaladsl.Baker
-import com.ing.bakery.baker.Bakery.resource
+import com.ing.bakery.Bakery.akkaBakery
 import com.typesafe.config.Config
 
-import scala.concurrent.ExecutionContext
+import java.io.Closeable
 
 class ClosableBakery(baker: Baker,
-                     executionContext: ExecutionContext,
                      system: ActorSystem,
-                     close: IO[Unit]) extends Bakery(baker, executionContext, system) with Closeable {
+                     close: IO[Unit]) extends AkkaBakery(baker, system) with Closeable {
   override def close(): Unit = close.unsafeRunSync()
 }
 
 object ClosableBakery {
   /**
     * Create bakery instance as external context
     * @param externalContext optional external context in which Bakery is running, e.g. Spring context
-    * @return
     */
   def instance(optionalConfig: Option[Config],
                externalContext: Option[Any],
                interactionManager: Option[InteractionManager[IO]] = None,
                recipeManager: Option[RecipeManager] = None): ClosableBakery = {
-    val (baker: Bakery, close: IO[Unit]) = resource(optionalConfig, externalContext, interactionManager, recipeManager).allocated.unsafeRunSync()
-    new ClosableBakery(baker.baker, baker.executionContext, baker.system, close)
+    val (baker: AkkaBakery, close: IO[Unit]) = akkaBakery(optionalConfig, externalContext, interactionManager, recipeManager).allocated.unsafeRunSync()
+    new ClosableBakery(baker.baker, baker.system, close)
   }
-}
+}
diff --git a/...ain/scala/com/ing/bakery/baker/Main.scala → .../src/main/scala/com/ing/bakery/Main.scala b/...ain/scala/com/ing/bakery/baker/Main.scala → .../src/main/scala/com/ing/bakery/Main.scala
@@ -1,13 +1,16 @@
-package com.ing.bakery.baker
-
-import java.io.File
-import java.net.InetSocketAddress
+package com.ing.bakery
 
 import cats.effect.{ExitCode, IO, IOApp}
+import com.ing.baker.http.DashboardConfiguration
+import com.ing.baker.http.server.common.RecipeLoader
+import com.ing.baker.http.server.scaladsl.{Http4sBakerServer, Http4sBakerServerConfiguration}
+import com.ing.bakery.components.BakerReadinessCheck
 import com.ing.bakery.metrics.MetricService
 import com.typesafe.config.ConfigFactory
 import com.typesafe.scalalogging.LazyLogging
 
+import java.io.File
+import java.net.InetSocketAddress
 import scala.concurrent.duration.Duration
 
 object Main extends IOApp with LazyLogging {
@@ -16,31 +19,26 @@ object Main extends IOApp with LazyLogging {
 
     val configPath = sys.env.getOrElse("CONFIG_DIRECTORY", "/opt/docker/conf")
     val config = ConfigFactory.load(ConfigFactory.parseFile(new File(s"$configPath/application.conf")))
-    val bakerConfig = config.getConfig("baker")
-    val apiPort = bakerConfig.getInt("api-port")
-    val metricsPort = bakerConfig.getInt("metrics-port")
-    val apiUrlPrefix = bakerConfig.getString("api-url-prefix")
-    val dashboardPath = bakerConfig.getString("dashboard-path")
-    val loggingEnabled = bakerConfig.getBoolean("api-logging-enabled")
+    val metricsPort = config.getInt("baker.metrics-port")
 
     (for {
-      bakery <- Bakery.resource(Some(config))
+      bakery <- Bakery.akkaBakery(Some(config))
       _ <- MetricService.resource(InetSocketAddress.createUnresolved("0.0.0.0", metricsPort), bakery.executionContext)
 
-      bakerService <- BakerService.resource(
+      bakerService <- Http4sBakerServer.resource(
         bakery.baker,
-        bakery.executionContext,
-        InetSocketAddress.createUnresolved("0.0.0.0", apiPort),
-        apiUrlPrefix, dashboardPath, loggingEnabled)
+        Http4sBakerServerConfiguration.fromConfig(config),
+        DashboardConfiguration.fromConfig(config),
+        bakery.executionContext)
 
     } yield (bakery, bakerService))
       .use { case (bakery, bakerService) =>
         logger.info(s"Bakery started at ${bakerService.address}/${bakerService.baseUri}, enabling the readiness in Akka management")
         BakerReadinessCheck.enable()
-        RecipeLoader.pollRecipesUpdates(configPath, bakery,
+        RecipeLoader.pollRecipesUpdates(configPath, bakery.baker,
           Duration.fromNanos(config.getDuration("baker.recipe-poll-interval").toNanos))
 
-    }.as(ExitCode.Success)
+      }.as(ExitCode.Success)
   }
 
 }