Using Typescript types/interfaces as input for Typebox Types

[sherzod-sync]

Hello,

I'd like to make a Typebox type based on a Typescript type or an interface. Is it possible?

For example, I have the following Typescript type:

type numberArrayT = number[]

I want to have something like:

const T = foo(numberArrayT) //Type.Array(Type.Number())

[sinclairzx81]

@sherzod-sync Hi,

Unfortunately this isn't possible in TypeScript as all type information is erased on compilation. Because type information is erased, there is nothing available to compute the Type.Array(Type.Number()) at runtime.

However, there are alternative solutions to this, but each requires custom compiler transformations / plugins. These kinds of solutions usually take the TypeScript Compiler API as a dependency and use it to intercept the compilation to read type information available gathered during a compilation pass. This type information can then be emitted to JavaScript (to be available at runtime) or used to build reflection lookups (which can also be used at runtime)

TypeBox doesn't offer this functionality by default, so you will need to type things with the Type.* API. However it does provide a small codegen example as part of the project that some users have used to convert TypeScript interfaces and types to TypeBox types (as a compilation pass you might run during your build step). To use it you will need to clone the project or simply copy paste the code into your project.

Codegen

https://github.com/sinclairzx81/typebox/blob/master/codegen/typebox.ts

Example

import { TypeBoxCodegen } from './path/to/typebox.ts'

const code = TypeBoxCodegen.Generate(`

  interface Vector {
     x: number,
     y: number,
     z: number
  }

  type Vectors = Vector[]
`)

console.log(code)

// outputs
// import { Type, Static } from '@sinclair/typebox'

// type Vector = Static<typeof Vector>
// const Vector = Type.Object({
//   x: Type.Number(),
//   y: Type.Number(),
//   z: Type.Number()
// })

// type Vectors = Static<typeof Vectors>
// const Vectors = Type.Array(Vector)

Hope this helps
S

[sherzod-sync]

Thanks for clarifying this!

[xddq]

Did something change for this? I tried to use it, but it has no export TypeBoxCodeGen. I tried (with Version 0.26.3)

import {TypeBoxCodeGen} from "@sinclair/typebox"

[sinclairzx81]

@xddq Hi. The code generation modules are not actually part of the published TypeBox package so can't be directly imported with @sinclair/typebox. The reason they are not included is due to them requiring the TypeScript compiler as a dependency. So to use the codegen modules, you will need to copy / paste them into your project.

Note that code generation is considered "out-of-scope" for the TypeBox library. The current codegen modules are provided as a reference for how one might approach codegen with TypeBox, as well as to provide a code transformation utility for a future TypeBox documentation website.

There hasn't been many changes to the codegen modules in 0.26.0, but I have renamed the modules to make it clear the X > Y transformation.

The current codegens are provided for reference.

• TypeScriptToTypeBox
• TypeScriptToJsonSchema
• TypeBoxToTypeScript
• TypeBoxToZod

[xddq]

@sinclairzx81 Alright, did update to your newest changes.
Is there anything else missing in the type generation?

I think I would continue with adding test cases to the repo and afterwards adding the ability to rename the generated types and schemas.

Btw. why did you decide to use UpperCamelCase naming for the schemas? I personally would prefer using lowerCamelCase and appending "Schema" to it (hence I want to add the ability to rename types and schemas ^^)

[sinclairzx81]

Alright, did update to your newest changes.

Nice one :)

Btw. why did you decide to use UpperCamelCase naming for the schemas? I personally would prefer using lowerCamelCase and appending "Schema" to it (hence I want to add the ability to rename types and schemas ^^)

TypeBox uses Pascal casing as type names are typically expressed using this convention. Also originally, TypeBox opted for this casing to be inline JavaScript's Object, Array, Number, Boolean, String primitive types, but this has since been extended to Pick, Omit, Partial TypeScript utility types (which also use Pascal casing).

Mostly, TypeBox tries to make a visual distinction between JavaScript values that are values, and JavaScript values that are types (noting everything is a value in JavaScript). Pascal casing helps make that distinction a little clearer.

Is there anything else missing in the type generation?

There are two main things missing from the current code generation (that I'm aware of):

• Mapped Types: type Mapped<T> = { [K in keyof T]: T[K] }
• Recursive Generics: type Node<T> = { id: string, nodes: Node<T>[] }

These two target representations need solving in some capacity, and it's not clear (at this stage) how best to solve. There is a little bit of HKT research / investigations to see if Type.Mapped() can be reasonably be added as a standard type, as well as working out how to reconcile recursive generic types such that the schematics will operate in both Ajv and TypeCompiler (as well as getting the inference working).

Other than that, most other types should be working ok, however just be a bit mindful the codegen side hasn't had much testing, so if you wanted to put up a couple of tests in ts2typebox, that might be good.

Cheers!
S

[xddq]

Mhh, I like the "Type.Number()" syntax but don't really agree with Type.Compile, Check, Error, etc.. syntax being correct in PascalCase.

So today I figured out that IndexedAccessTypes are not supported yet and took a look at the code generation. I had(still don't have :D) no idea of using typescript AST to do stuff before, if you got any advice on basic information I can look up, I would be happy to get a link. I did find https://ts-ast-viewer.com which helped a lot! Anyway, I was lucky and got it working for a minimal examle which should work for index access with "depth 1".

before:

// Arbitrary example types

export type Address = {
  street: string;
  number: number;
};

export type Address2 = {
  street2: Address["street"];
};

yarn start (in ts2typescript repo)

import { Type, Static } from '@sinclair/typebox'

export type Address = Static<typeof Address>
export const Address = Type.Object({
  street: Type.String(),
  number: Type.Number()
})

export type Address2 = Static<typeof Address2>
export const Address2 = Type.Object({
  street2: Address["street"]
})

on the WIP branch

// Arbitrary example types

export type Address = {
  street: string;
  number: number;
};

export type Address2 = {
  street2: Address["street"];
};

yarn start (in ts2typescript repo)

import { Type, Static } from '@sinclair/typebox'

export type Address = Static<typeof Address>
export const Address = Type.Object({
  street: Type.String(),
  number: Type.Number()
})

export type Address2 = Static<typeof Address2>
export const Address2 = Type.Object({
  street2: Type.String()
})

🎉

I have got some more questions where I would like to get your opinion.

1. IndexedAccessTypes. What do you think about the approach to:

• store all identifiers and their typeDefinitions in a map where export const Address = Type.Object({ street: Type.String(), number: Type.Number() }) will become an entry in the map with "Address" as key and "Type.Object({
  street: Type.String(),
  number: Type.Number()
  })" as value.
• Then whenever we find an AST node of kind "IndexedAccessType" we look in our map where based on the "Identifier" of the node and return the matching type based on the "StringLiteral" of the node.
• I feel like nested index access will need some love, but I guess I can get it working after putting some thought into it.

2. Testing (will do this after IndexedAccessTypes). I tried setting up testing but quickly realized it will not be the typical way of testing. What I thought of doing was to create tests for each scenario, e.g. number type gets generated correctly, array type, string type, union type etc. But even the simple cases seem somewhat hard.

• I came up with a "dream procedure" I would want to use which is:
  • write a string for the typescript code
  • run ts2typescript on that string and store the result
  • (now the harder part) I would love to create a third file which imports both the typescript type as well as the created typebox type which contains a utility type like "Equals". I would then try to compile this file(per test case) and if it errors, something with the type generation is wrong. The generated file would look something like this:

import TypescriptType from sampleFile
import TypeboxType from generatedFile

 type Equals<A, B> = A extends B ? (B extends A ? true : false) : false;
 type C = Equals<TypescripType, TypeboxType>;
 const x: C = true;

• I have no clue how to do this yet, but is this possible? Which approach would you suggest?

[sinclairzx81]

@xddq Hi.

There's been a little bit more work done on the TypeBox code gen modules. "Indexers Types", "Conditional Types" and "String Template Literal Types" compiler routines have been partially implemented on https://github.com/sinclairzx81/typebox/blob/master/example/codegen/typescript-to-typebox.ts

Additionally, I've setup a small auxiliary workbench project to help test some of this compiler work. See the links below which show the code generation output.

Workbench Project Here

• Indexer Types
• Conditional Types
• String Literal Types

Just keep in mind that "Indexer Types" and "String Literal Types" currently don't have any type inference capabilities but the schematics should validate against TypeCompiler and Ajv. Conditional Types are supported with type inference though, this functionality was added as standard on 0.26.0.

Cheers!
S

[xddq]

Oh, nice!

Could you add cli capabilites to the workbench project? This would be exactly what I wanted/needed with ts2typebox.

Do you want me to help you with any of this? I could

• continue with the indexed access types and send a pr against the workbench project
• work on adding cli capabilites to the workbench project and send a pr

I guess I will simply deprecate the ts2typescript one then, best to have it all in one place

[sinclairzx81]

@xddq Heya. The workbench is just a utility project for the time being (or supplementary development tooling built to find transformation issues (or at least have a way to reason about them)). I've moved the codegen modules over to the workbench for development, but am still syncing them back to this repository.

Do you want me to help you with any of this? I could

Sure! would be great to collaborate on some of this. This codegen stuff has been sitting there for a while, and I think it highlights some of the more complex disparities between TS and TB. Would be good to get another set of eyes across it (and your indexer update would be welcome there too). Just drop an issue / PR on the workbench repo and we can take a look at merge your updates across.

I guess I will simply deprecate the ts2typescript one then, best to have it all in one place

I'd hold off on deprecating. The workbench is not really a CLI project, it's intended to be more analogous to the TSP. I do think there's value in offering CLI tooling (particularly toolchain integration) so your project is absolutely worth keeping.

[xddq]

alright!

What do you think about creating a repository for the codegen itself(or perhaps even use the ts2typebox repo)? Syncinc back and forth across multiple repos will become time consuming.

Maybe something like "@sinclar/typebox-codegen-core" which contains more or less the codegen repos from here?

Like that we could simply import the codegen inside ts2typebox and the workbench and use it for different purposes.

Also, do you have some other means of communication? Or should we stick to github discussions?

[xddq]

@sinclairzx81 not sure if you have read this.

I guess I will then just send a PR today or tomorrow against the workbench project with the indexed access types changes.

[sinclairzx81]

@xddq Hi. Sorry I meant to reply sooner (had been caught up in some other work)

I've just setup a codegen project under private repository for the time being and sent you a collaborator invite. I've put some initial documentation in the project and moved the current code generation modules from the workbench project over to it. Things are a little bit in flux with the current setup, but the TypeScriptToTypeBox transform should ready to be updated with your indexer changes. I've also made a new type model system where it should be possible to transform in memory TB models out to some other IDL formats.

Let's move discussions to the new codegen project!
Cheers!

[burakakca]

Also, We may need to json to typebox for some cases

[gorbak25]

+1 on this. I need to use external json schema definitions(from OCI) and integrate them into an existing typebox project.

[xddq]

We are thinking about tackling this issue

But it would (similar to typebox) only support JSON schema draft 06.

Would this still be helpful to you?

@gorbak25 @burakakca

[gorbak25]

@xddq Yes, this would be enough. For now i copied the codegen from the examples and use it.

[xddq]

@gorbak25 from the example code?

The example code here is Typescript -> TypeBox, isnt it?
There are two projects that do this for you already, https://github.com/sinclairzx81/typebox-workbench inside the browser and https://github.com/xddq/ts2typebox via cli or programmatically.

I meant to ask for the JSON schema -> TypeBox case, but I guess I was in the wrong discussion : x

[gorbak25]

@xddq I need JSON schema -> TypeBox but currently I've used json2ts to generate typescript types from json schema and the example codegen code to get typebox definitions from the compiled json schema.
https://github.com/hocus-dev/hocus/blob/10a8fc1cd8d1eb830ded3f90d94b4694b8ebf272/app/agent/block-registry/oci-schema/Makefile#L13

[xddq]

@gorbak25 I just released schema2typebox, hope it can be helpful !