Nested collection of objects

[arunma]

Newbie to Kotlin and Konf here. Is there an easier way to achieve mapping the following yaml to the Spec below:

Or does the solution involve using the jackson mapper with config.mapper? Thank you !

datasets:
  hive:
    - key: transactions
      uri: /user/somepath
      format: parquet
      database: transations_daily
      table: transx

    - key: second_transactions
      uri: /seconduser/somepath
      format: avro
      database: transations_monthly
      table: avro_table
  file:
    - key: users
      uri: s3://filestore
      format: parquet
      mode: overwrite

object DataSetsSpec: ConfigSpec("datasets"){
    val fileDataSets by optional<List<FileDS>>(default = emptyList())
    val hive by optional<List<HiveDS>>(default = emptyList())
}

object FileDS: ConfigSpec ("file"){
    val key by required<String>(description = "Name of the dataset")
    val uri by required<String>(description = "Target path of the file with the protocol qualifier")
    val format by optional<String>("parquet", "File format", "Format in which the file must be writte")
    val mode by optional<String>("append", "Save Mode", "Mode in which the file would be written - (overwrite, append)")
}

object HiveDS: ConfigSpec ("hive"){
    val key by required<String>()
    val uri by required<String>()
    val database by required<String>()
    val table by required<String>()
    val format by optional<String>("parquet", "File format", "Format in which the file must be writte")
    val mode by optional<String>("append", "Save Mode", "Mode in which the file would be written - (overwrite, append)")
}

[uchuhimo]

@arunma You can use Kotlin data class for nested objects in collections. Konf will handle all the serialization stuff. Here is a runnable solution for your case:

import com.uchuhimo.konf.Config
import com.uchuhimo.konf.ConfigSpec

val text = """
datasets:
  hive:
    - key: transactions
      uri: /user/somepath
      format: parquet
      database: transations_daily
      table: transx

    - key: second_transactions
      uri: /seconduser/somepath
      format: avro
      database: transations_monthly
      table: avro_table
  file:
    - key: users
      uri: s3://filestore
      format: parquet
      mode: overwrite
""".trimIndent()

object DatasetsSpec: ConfigSpec() {
    val file by optional<List<FileDS>>(default = emptyList())
    val hive by optional<List<HiveDS>>(default = emptyList())
}

data class FileDS(val key: String, val uri: String, val format: String = "parquet", val mode: String = "append")

data class HiveDS(val key: String, val uri: String, val database: String, val table: String,  val format: String = "parquet", val mode: String = "append")

fun main(args: Array<String>) {
    val config = Config { addSpec(DatasetsSpec) }.from.yaml.string(text)
    check(config[DatasetsSpec.file][0].key == "users")
    check(config[DatasetsSpec.hive][1].database == "transations_monthly")
}

[arunma]

@uchuhimo This is excellent ! Love the library. Thanks a lot !!

[yashar-sb-sb]

@uchuhimo
This way you can't have optional items?

[uchuhimo]

@yashar-sb-sb You can provide default value to Kotlin data class's field, which is equivalent to use optional items. Or can you provide an example in which Konf cannot work?

[drejc]

Is there a way to name properties in Kotlin data class?
For instance we have a YAML

servers:
  server:
    host: 
    time-out: 5

and a data class:

Server(val host: String, timeOut: Int)

How would one bind timeOut to "time-out" ?

[amit-handda]

@drejc or anyone who lands here, I would use @JsonProperty("time-out) val timeOut: Int instead.