Reworked mapping management and validation (#608)

* Reworked mapping management and validation

+ The collection of type mappings has moved to a new first class
  TypeMappings container type. There is an implicit conversion from
  List[TypeMapping] to TypeMappings, so this is a source compatible
  change (this will probably be deprecated in a later release).
  Most mapping validtion logic has been moved to TypeMappings, apart
  from mapping-specific rules (eg. for SqlMapping) which are delegated
  to the relevant Mapping subtype at the granularity of validity for
  individual type and field mappings.
+ Added MappingPredicate as an extensible mechanism for matching
  mappings to paths. TypeMatch replaces the existing Type linked
  association, PrefixedTypeMatch replaces the PrefixedMapping wrapper,
  and PathMatch corresponds to the the SwitchTypeMapping used by
  Gemini/Aura.
+ Mapping validation is now performed by default unless explicitly
  disabled. Validation is deferred until the mappings compiler is first
  referenced (to avoid init-order issues, object initialization errors
  and poor interatactions of the latter with munit-cats-effect) and is
  performed exactly once. Applications which have at least one guery
  unit test will trigger validation failures automatically at test time.
+ The validation algorithm has been reworked to start from the GraphQL
  schema and generate an exhaustive set of paths which are used to
  determine the relevant mappings to check against GraphQL and DB schema
  types. The support both traversal through mappings which are partly
  implicit (eg. the Circe and GenericMappings) and also accomodate
  mappings which are guarded by MappingPredicates.
+ The base mapping validator now also reports unused (ie. not reachable
  via any valid path in the GraphQL schema) type and field mappings.
+ The SQL mapping validator tests for,
  + Consistency of nullability and Scala type between GraphQL and SQL.
  + Leaf/ObjectMapping consistency with GraphQL leaf or non-leaf types.
  + Objects, interfaces and unions being nested within single a single
    DB table.
  + Associative fields also being keys.
  + Union field mappings must be hidden and leaf.
  + Embedded subobjects must be nested in their parent object table.
  + Joins must have at least one join condition.
  + Parallel joins must relate the same tables.
  + Serial joins must chain correctly.
+ Type mappings with non-trivial predicates are now indexed.
+ LeafMapping now supports MappingPrediates and are indexed along with
  ObjectMappings.
+ Built-in Scalar types now have explicit LeafMappings and can now be
  specialized for particular contexts via MappingPredicates.
+ Added a subtree Boolean attribute to FieldMapping to signal that a
  field value can represent structured result subtrees which are not
  explicitly mapped at all levels. This allows mapping validation to
  include the Circe and Generic mappings which implicitly map subtrees.
+ The ValueObjectMapping has a new on constructor and the withParent
  initializer method, which was present on all mappings has been removed
  and replaced by the ValueFieldMapping specific unwrap.
+ The IntrospectionMapping now uses the new mapping API.
+ All tests have been updated so validate under the new scheme.
+ The Doobie and Skunk test suites now have typed codecs which can
  capture the metadata needed for validation.
+ The tutorial and demo and profile projects have been updated to the
  new API.
+ Added schema validation checks to ensure interfaces are non-cyclic.
+ Added forUnderlyingNamed convenience method to Context.
+ ValidationFailure split out to a separate file.
+ The sql classifier has been moved from ValidationFailure to a
  SqlValidationFailure subtype.
+ GraphQL types menioned in ValidationFailure error messages are now
  correctly rendered uing GraphQL syntax.
+ ComposedMapping has been split out to a separate file.
+ Various signatures previously using List now use Seq.

* Factored factored out MappingPredicate logic

* Added missing implicit SourcePos argument

* Tone down the severity indicators

* Fixed typos

* fix pathmatch

* Typeref#withSchema

* Compute base Scala type name for opt Skunk Codecs

* Remove TypeRef#withSchema, use Schema#uncheckedRef instead

* Add support for narrowing interfaces and unions along Paths

* Added Path-based type mapping constructors

* Report ambiguous mappings

---------

Co-authored-by: Rob Norris <[email protected]>

build.sbt

@@ -39,6 +39,7 @@ ThisBuild / developers := List(
  Developer("tpolecat", "Rob Norris", "[email protected]", url("http://www.tpolecat.org")),
 )
 
+ThisBuild / tlFatalWarnings := true
 ThisBuild / tlCiScalafmtCheck := false
 ThisBuild / tlCiReleaseBranches := Seq("main")
 ThisBuild / tlSonatypeUseLegacyHost := false

demo/src/main/scala/demo/starwars/StarWarsMapping.scala

@@ -69,17 +69,13 @@ trait StarWarsMapping[F[_]] extends GenericMapping[F] { self: StarWarsData[F] =>
  val DroidType = schema.ref("Droid")
 
  val typeMappings =
- List(
+ TypeMappings(
  // #root
- ObjectMapping(
- tpe = QueryType,
- fieldMappings =
- List(
- GenericField("hero", characters),
- GenericField("character", characters),
- GenericField("human", characters.collect { case h: Human => h }),
- GenericField("droid", characters.collect { case d: Droid => d })
- )
+ ObjectMapping(QueryType)(
+ GenericField("hero", characters),
+ GenericField("character", characters),
+ GenericField("human", characters.collect { case h: Human => h }),
+ GenericField("droid", characters.collect { case d: Droid => d })
  )
  // #root
  )

demo/src/main/scala/demo/world/WorldMapping.scala

@@ -119,65 +119,53 @@ trait WorldMapping[F[_]] extends DoobieMapping[F] {
  val LanguageType = schema.ref("Language")
 
  val typeMappings =
- List(
+ TypeMappings(
  // #root
- ObjectMapping(
- tpe = QueryType,
- fieldMappings = List(
- SqlObject("cities"),
- SqlObject("city"),
- SqlObject("country"),
- SqlObject("countries"),
- SqlObject("language"),
- SqlObject("search"),
- SqlObject("search2")
- )
+ ObjectMapping(QueryType)(
+ SqlObject("cities"),
+ SqlObject("city"),
+ SqlObject("country"),
+ SqlObject("countries"),
+ SqlObject("language"),
+ SqlObject("search"),
+ SqlObject("search2")
  ),
  // #root
  // #type_mappings
- ObjectMapping(
- tpe = CountryType,
- fieldMappings = List(
- SqlField("code", country.code, key = true),
- SqlField("name", country.name),
- SqlField("continent", country.continent),
- SqlField("region", country.region),
- SqlField("surfacearea", country.surfacearea),
- SqlField("indepyear", country.indepyear),
- SqlField("population", country.population),
- SqlField("lifeexpectancy", country.lifeexpectancy),
- SqlField("gnp", country.gnp),
- SqlField("gnpold", country.gnpold),
- SqlField("localname", country.localname),
- SqlField("governmentform", country.governmentform),
- SqlField("headofstate", country.headofstate),
- SqlField("capitalId", country.capitalId),
- SqlField("code2", country.code2),
- SqlField("numCities", country.numCities),
- SqlObject("cities", Join(country.code, city.countrycode)),
- SqlObject("languages", Join(country.code, countrylanguage.countrycode))
- ),
+ ObjectMapping(CountryType)(
+ SqlField("code", country.code, key = true),
+ SqlField("name", country.name),
+ SqlField("continent", country.continent),
+ SqlField("region", country.region),
+ SqlField("surfacearea", country.surfacearea),
+ SqlField("indepyear", country.indepyear),
+ SqlField("population", country.population),
+ SqlField("lifeexpectancy", country.lifeexpectancy),
+ SqlField("gnp", country.gnp),
+ SqlField("gnpold", country.gnpold),
+ SqlField("localname", country.localname),
+ SqlField("governmentform", country.governmentform),
+ SqlField("headofstate", country.headofstate),
+ SqlField("capitalId", country.capitalId),
+ SqlField("code2", country.code2),
+ SqlField("numCities", country.numCities),
+ SqlObject("cities", Join(country.code, city.countrycode)),
+ SqlObject("languages", Join(country.code, countrylanguage.countrycode))
  ),
- ObjectMapping(
- tpe = CityType,
- fieldMappings = List(
- SqlField("id", city.id, key = true, hidden = true),
- SqlField("countrycode", city.countrycode, hidden = true),
- SqlField("name", city.name),
- SqlField("district", city.district),
- SqlField("population", city.population),
- SqlObject("country", Join(city.countrycode, country.code)),
- )
+ ObjectMapping(CityType)(
+ SqlField("id", city.id, key = true, hidden = true),
+ SqlField("countrycode", city.countrycode, hidden = true),
+ SqlField("name", city.name),
+ SqlField("district", city.district),
+ SqlField("population", city.population),
+ SqlObject("country", Join(city.countrycode, country.code)),
  ),
- ObjectMapping(
- tpe = LanguageType,
- fieldMappings = List(
- SqlField("language", countrylanguage.language, key = true, associative = true),
- SqlField("isOfficial", countrylanguage.isOfficial),
- SqlField("percentage", countrylanguage.percentage),
- SqlField("countrycode", countrylanguage.countrycode, hidden = true),
- SqlObject("countries", Join(countrylanguage.countrycode, country.code))
- )
+ ObjectMapping(LanguageType)(
+ SqlField("language", countrylanguage.language, key = true, associative = true),
+ SqlField("isOfficial", countrylanguage.isOfficial),
+ SqlField("percentage", countrylanguage.percentage),
+ SqlField("countrycode", countrylanguage.countrycode, hidden = true),
+ SqlObject("countries", Join(countrylanguage.countrycode, country.code))
  )
  // #type_mappings
  )

modules/circe/src/main/scala/circemapping.scala

@@ -59,7 +59,7 @@ trait CirceMappingLike[F[_]] extends Mapping[F] {
  override def mkCursorForField(parent: Cursor, fieldName: String, resultName: Option[String]): Result[Cursor] = {
  val context = parent.context
  val fieldContext = context.forFieldOrAttribute(fieldName, resultName)
- (fieldMapping(context, fieldName), parent.focus) match {
+ (typeMappings.fieldMapping(context, fieldName), parent.focus) match {
  case (Some(CirceField(_, json, _)), _) =>
  CirceCursor(fieldContext, json, Some(parent), parent.env).success
  case (Some(CursorFieldJson(_, f, _, _)), _) =>
@@ -70,7 +70,7 @@ trait CirceMappingLike[F[_]] extends Mapping[F] {
  }
 
  sealed trait CirceFieldMapping extends FieldMapping {
- def withParent(tpe: Type): FieldMapping = this
+ def subtree: Boolean = true
  }
 
  case class CirceField(fieldName: String, value: Json, hidden: Boolean = false)(implicit val pos: SourcePos) extends CirceFieldMapping
@@ -174,7 +174,7 @@ trait CirceMappingLike[F[_]] extends Mapping[F] {
 
  def hasField(fieldName: String): Boolean =
  tpe.hasField(fieldName) && focus.asObject.exists(_.contains(fieldName)) ||
- fieldMapping(context, fieldName).isDefined
+ typeMappings.fieldMapping(context, fieldName).isDefined
 
  def field(fieldName: String, resultName: Option[String]): Result[Cursor] = {
  val localField =

modules/circe/src/test/scala/CirceEffectData.scala

@@ -87,7 +87,7 @@ class TestCirceEffectMapping[F[_]: Sync](ref: SignallingRef[F, Int]) extends Cir
  // Compute a CirceCursor focussed on the root
  RootEffect.computeCursor("qux")((p, e) =>
  ref.update(_+1).as(
- Result(circeCursor(Path(p.rootTpe), e,
+ Result(circeCursor(Path.from(p.rootTpe), e,
  Json.obj(
  "qux" ->
  Json.obj(

modules/circe/src/test/scala/CircePrioritySuite.scala

@@ -44,7 +44,7 @@ object CircePriorityMapping extends CirceMapping[IO] {
  val BarrelType = schema.ref("Barrel")
  val FooType = schema.ref("Foo") 
 
- val typeMappings: List[TypeMapping] =
+ val typeMappings =
  List(
  ObjectMapping(
  tpe = QueryType,

modules/core/src/main/scala/compiler.scala

@@ -400,7 +400,7 @@ class QueryCompiler(parser: QueryParser, schema: Schema, phases: List[Phase]) {
  else {
  val hd = pendingFrags.head
  if (seen.contains(hd)) None
- else checkCycle(fragRefs(hd)._2 ++ pendingFrags.tail, seen + hd)
+ else checkCycle(fragRefs.get(hd).map(_._2).getOrElse(Set.empty) ++ pendingFrags.tail, seen + hd)
  }
  }
 
@@ -1231,7 +1231,7 @@ object QueryCompiler {
 
  case class ComponentMapping[F[_]](tpe: TypeRef, fieldName: String, mapping: Mapping[F], join: (Query, Cursor) => Result[Query] = TrivialJoin)
 
- def apply[F[_]](mappings: List[ComponentMapping[F]]): ComponentElaborator[F] =
+ def apply[F[_]](mappings: Seq[ComponentMapping[F]]): ComponentElaborator[F] =
  new ComponentElaborator(mappings.map(m => ((m.tpe, m.fieldName), (m.mapping, m.join))).toMap)
  }
 
@@ -1279,7 +1279,7 @@ object QueryCompiler {
  object EffectElaborator {
  case class EffectMapping[F[_]](tpe: TypeRef, fieldName: String, handler: EffectHandler[F])
 
- def apply[F[_]](mappings: List[EffectMapping[F]]): EffectElaborator[F] =
+ def apply[F[_]](mappings: Seq[EffectMapping[F]]): EffectElaborator[F] =
  new EffectElaborator(mappings.map(m => ((m.tpe, m.fieldName), m.handler)).toMap)
  }
 

modules/core/src/main/scala/composedmapping.scala

@@ -0,0 +1,47 @@
+// Copyright (c) 2016-2023 Association of Universities for Research in Astronomy, Inc. (AURA)
+// Copyright (c) 2016-2023 Grackle Contributors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package grackle
+
+import cats.MonadThrow
+
+import Cursor.AbstractCursor
+import syntax._
+
+abstract class ComposedMapping[F[_]](implicit val M: MonadThrow[F]) extends Mapping[F] {
+ override def mkCursorForField(parent: Cursor, fieldName: String, resultName: Option[String]): Result[Cursor] = {
+ val context = parent.context
+ val fieldContext = context.forFieldOrAttribute(fieldName, resultName)
+ typeMappings.fieldMapping(context, fieldName) match {
+ case Some(_) =>
+ ComposedCursor(fieldContext, parent.env).success
+ case _ =>
+ super.mkCursorForField(parent, fieldName, resultName)
+ }
+ }
+
+ case class ComposedCursor(context: Context, env: Env) extends AbstractCursor {
+ val focus = null
+ val parent = None
+
+ def withEnv(env0: Env): Cursor = copy(env = env.add(env0))
+
+ override def hasField(fieldName: String): Boolean =
+ typeMappings.fieldMapping(context, fieldName).isDefined
+
+ override def field(fieldName: String, resultName: Option[String]): Result[Cursor] =
+ mkCursorForField(this, fieldName, resultName)
+ }
+}

modules/core/src/main/scala/cursor.scala

@@ -453,6 +453,12 @@ case class Context(
  copy(path = fieldName :: path, resultPath = resultName.getOrElse(fieldName) :: resultPath, typePath = fieldTpe :: typePath)
  }
 
+ def forUnderlyingNamed: Context =
+ typePath match {
+ case Nil => copy(rootTpe.underlyingNamed.dealias)
+ case hd :: tl => copy(typePath = hd.underlyingNamed.dealias :: tl)
+ }
+
  override def equals(other: Any): Boolean =
  other match {
  case Context(oRootTpe, oPath, oResultPath, _) =>