-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API v2 #57
API v2 #57
Conversation
@arianvp I'll now take care of the first two items in the TODO list (adding comments and splitting code in modules), but otherwise all the items are up for grabs and any help would be amazing 😄 About the build, we have two possibilities here:
Option 2 is tempting so we keep up with all the changes in dhall-haskell, but on the other hand it would be really useful to build and generate on CI runs (to avoid breakage), so I think I'm leaning towards option 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just some remarks. not much so far.
Question @f-f and @ari-becker What format do we want Something that resembles https://github.com/coralogix/dhall-prometheus-operator/blob/master/ImportTypes.dhall ? It does seem like a nice paradigm. I was thinking, perhaps we can do this nested ala:
# types.dhall
{ io = ./io.dhall } # k8s.dhall
{ api = ./api.dhall } # api.dhall
{ apps = ./apps.dhall
, rbac = ./rbac.dhall
, storage = ./storage.dhall
...
} # apps.dhall
{ Deployment = ./Deployment.dhall
, DeploymentSpec = ./DeploymentSpec.dhall
} You can then do something like:
Similar to how that's currently done in I'm not sure if the nested directory structure is neccssery per se. Pro is that it allows people to just use
Not sure what is preferable. Thoughts? Note that this problem is also present in |
@arianvp the approach I took here comes from the fact that I consider the However, one might need objects that are not in these records (because they are on a different API version), so I propose we keep the current approach but also introduce a Some other notes:
|
I don't fully understand the second point. But I'm fine with not nesting the directories so let's do as you suggest |
@arianvp sorry I think I misunderstood your previous comment. |
Agree with this reasoning. Typing out the whole namespace for each record and type gets old fast. Would probably end in import style let-expressions (e.g.
Question : because the users of this library are presumably Kubernetes end-users, does the |
Lets keep everything in this Repo. Dhall langauge is slowly starting to stabilize, so it doesn't matter if we |
With this latest changes I'm pretty sure now #46 is properly implemented and working. This is how a simple deployment looks like with the new schemas: let types =
../types.dhall sha256:1ac84a230c0fdd003b4db05157b11329d273217cc6837083eb30c9288a67c87f
let defaults =
../defaults.dhall sha256:2e3d2b5d5fb52d08b078e803fa88ac1ed0f788c6a4d6342621a76eb77afe40f4
let deployment
: types.Deployment
= defaults.Deployment
⫽ { metadata =
defaults.ObjectMeta ⫽ { name = "nginx" }
, spec =
Some
( defaults.DeploymentSpec
⫽ { replicas =
Some 2
, template =
defaults.PodTemplateSpec
⫽ { metadata =
defaults.ObjectMeta ⫽ { name = "nginx" }
, spec =
Some
( defaults.PodSpec
⫽ { containers =
[ defaults.Container
⫽ { name =
"nginx"
, image =
Some "nginx:1.15.3"
, ports =
[ defaults.ContainerPort
⫽ { containerPort = 80 }
]
}
]
}
)
}
}
)
}
in deployment And the new $ dhall-to-yaml --omitEmpty <<< "./examples/deploymentSimple.dhall"
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- image: nginx:1.15.3
name: nginx
ports:
- containerPort: 80
metadata:
name: nginx
replicas: 2
metadata:
name: nginx |
@arianvp any idea why Hydra shows a passing CI? The CI should fail for this branch, as all the files in |
The CI generates the default folder as part of its build step. The result of the
So before the tests are run, the We should modify the Since we can't build our haskell script with Nix yet, we could hack out the types generation for now, so you can manually test stuff In the makefile https://github.com/dhall-lang/dhall-kubernetes/blob/master/Makefile change the
and in the derivation https://github.com/dhall-lang/dhall-kubernetes/blob/master/nix/dhall-kubernetes.nix that should cause those folders even if generated by hand to end up in the nix derivation and be available for the |
I'm not sure what it is; it for sure isn't commited
Tests currently fail; as expected
@f-f The build uses haskell now! 🎊 The tests now fail; as expected |
We need new examples anyway
dhall-kubernetes-generator.cabal
Outdated
@@ -18,7 +18,6 @@ executable dhall-kubernetes | |||
Dhall.Kubernetes.Convert | |||
Dhall.Kubernetes.Data | |||
Dhall.Kubernetes.Types | |||
Paths_dhall_kubernetes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TL;DR: you want it to refer to static data without using Template Haskell (it includes other stuff too)
But you're right that we're not using it here 😄
@arianvp @ari-becker I rewrote the readme with the new examples, and documented the pattern for #54, could you proofread before we merge? 😊 @arianvp CI is failing because hash mismatches; I generated all the hashes locally with dhall |
First release done 🎉, we are ready to merge this as soon as we get CI green 😄 |
@f-f did you manually We should adjust remember that the tests and the build run in a sandbox that exclude the |
@arianvp the |
Looking at the diff on the examples:
We forgot that not only we should omit |
@arianvp I did fix it some days ago 🙂 dhall-lang/dhall-haskell#872 But now that I think of it CI will try to compare the committed yamls (which I generated with the patch above) with the newly generated ones - which might include empty lists - so it will likely fail. Should we pin |
~~Let's just pin to that commit for now. ~~ Edit: perhaps we should wait til the next dhall release. because people will probably install |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arianvp @ari-becker I rewrote the readme with the new examples, and documented the pattern for #54, could you proofread before we merge? 😊
Jeez, sorry I missed this somehow... LGTM though.
If part of the documentation is to try and show cool things that can be done with dhall-kubernetes
, I'd just bring up a pattern I've found: sorry it's a little messy and hard to read:
λ(deployment : KubernetesTypes.apps.v1.Deployment)
→ Kubernetes.core.v1.Service
{ metadata = deployment.metadata }
⫽ { spec =
Some
( Kubernetes.core.v1.ServiceSpec
⫽ { ports =
Some
( Prelude.`Optional`.fold
KubernetesTypes.apps.v1.DeploymentSpec
deployment.spec
(List KubernetesTypes.core.v1.ServicePort)
( λ ( spec
: KubernetesTypes.apps.v1.DeploymentSpec
)
→ Prelude.`Optional`.fold
KubernetesTypes.core.v1.PodSpec
spec.template.spec
(List KubernetesTypes.core.v1.ServicePort)
( λ ( spec
: KubernetesTypes.core.v1.PodSpec
)
→ Prelude.`List`.concatMap
KubernetesTypes.core.v1.Container
KubernetesTypes.core.v1.ServicePort
( λ ( container
: KubernetesTypes.core.v1.Container
)
→ Prelude.`Optional`.fold
(List KubernetesTypes.core.v1.ContainerPort)
container.ports
(List KubernetesTypes.core.v1.ServicePort)
( λ ( portList
: List
KubernetesTypes.core.v1.ContainerPort
)
→ Prelude.`List`.map
KubernetesTypes.core.v1.ContainerPort
KubernetesTypes.core.v1.ServicePort
( λ ( containerPort
: KubernetesTypes.core.v1.ContainerPort
)
→ Kubernetes.core.v1.ServicePort
{ port =
containerPort.containerPort
}
⫽ { name =
containerPort.name
, protocol =
containerPort.protocol
, targetPort =
let IntOrString =
< Int :
Natural
| String :
Text
>
in if Prelude.`Optional`.null
Text
containerPort.name
then Some
( IntOrString.Int
containerPort.containerPort
)
else Prelude.`Optional`.map
Text
IntOrString
( λ ( name
: Text
)
→ IntOrString.String
name
)
containerPort.name
}
: KubernetesTypes.core.v1.ServicePort
)
portList
)
( [] : List
KubernetesTypes.core.v1.ServicePort
)
)
spec.containers
)
([] : List KubernetesTypes.core.v1.ServicePort)
)
([] : List KubernetesTypes.core.v1.ServicePort)
)
, selector =
deployment.metadata.labels
, type =
Some "ClusterIP"
}
: KubernetesTypes.core.v1.ServiceSpec
)
}
: KubernetesTypes.core.v1.Service
And what that does is basically generate a Service
from a Deployment
, mapping each of the ports of the Deployment
to be exposed through a Service
. If ports or added or changed in the Deployment
, then those changes will be reflected in the Service
.
It'd have to be updated for API v2 of course, and maybe it should be in some kind of function library for dhall-kubernetes
, but yeah, I thought it was pretty cool when I finally got it to work.
LGTM! |
@ari-becker looks great! I'd say more docs is better, so let's include useful patterns too. @arianvp was planning a documentation site after we merge this in, so this would be definitely welcome there (small sidenote: we now provide the |
I'm sorry if this is a huge diff, but we're basically swapping the contents of the repo 😄
This PR ports the conversion script from Python to Haskell (fix #7).
The main reasons for this are that:
dhall
library to generate always syntactically correct Dhall AST. It's also much easier to keep an eye on correctness, because types and pattern matching. It also forces us to deal with things like cyclic imports from the get goThings happening here:
default
todefaults
folder, as it is indhall-nethack
types.dhall
,defaults.dhall
, so that one can now easily "pin a version" by just importing these two records at a specific commit/tag. They also make it really easy to access objects, e.g.let types = https://raw.githubusercontent... sha256:... in types.Deployment
typesUnion.dhall
(fix Generate top-level union type for all apis #54), so one is able to send to Kubernetes different objects in the same stream\(a : Text) -> a
as a default, now we just don't generate a default (e.g. seeio.k8s.apimachinery.pkg.apis.meta.v1.Time
)Things left to do before we can merge:
package.yaml
and add version boundsconvert.py
scriptapi
folderJSONSchemaProps
object, fixing the cyclic imports (fix JSONSchemaProps.dhall references itself (Cyclic import) #47)