Pagination model

[MichalLytek]

I think it might be nice to have a pagination model that will allow to create universal resolvers, types definitions, etc. for collections.

For now I am going to expose two types of models:

Simple pagination

Like with standard rest apis - only offset (skip), limit (take), count and items.

@Resolver()
class SampleResolver {
  @Query(returns => [Type])
  paginatedQuery(
    @PaginationArgs() { offset, limit }: PaginationArgs,
  ): PaginationResponse<Type> {
    // ...
    return [data, count];
  }
}

By using @PaginationArgs() it will detect that the query is being paginated, so the returned tuple [TData, number] will be converted into an object with page info (hasNextPage, etc.), data and total properties.

Relay connection

Using cursor model, which is better for highly dynamic collections:
https://facebook.github.io/relay/graphql/connections.htm

API - TBD.

[erencay]

@19majkel94 Sorry if this is mostly irrelevant but recently I started using dedicated query and mutation resolvers which extends abstract resolver classes with an abstract resolve method. Under the hood they can hide a lot of boilerplate and provides a clean implementation. Each class represents one and only one operation with a naming strategy based on the class name. I have yet to implement this solution for pagination but it looks like this approach works the best for me, so far.

The thing is, this is something resembles may be what a "conventional framework" would do? I don't know if this kind of approach is beyond type-graphql's intention. Also typegraphql is successfully using the decorator pattern which you can simply plug-in/out features. Inheritance might not have it's place in it to solve such problems in the first place.

Just wanted to point out how I'm planning to solve the problem.

@InputType()
class LoginInput {
    @Field()
    public username: string;

    @Field()
    public password: string;
}

@MutationResolver(LoginInput, type => Login)
class Login extends AbstractMutationResolver<LoginInput> {
    public async resolve() {
        if (this.ctx.currentUser) {
            throw new Error("You are already logged in.")
        }

        return this.input.username == this.input.password;
    }
}

[MichalLytek]

I'm glad that you found a way to reduce a boilerplate for your case 😃
I would love to take a look at it when you finish it.

But I think that such constructs that you've presented is way too much complicated design to be the official API. Of course, it can reduce boilerplate but also introduces a huge amount of magic which is not intuitive.

TypeGraphQL design goal is to be simple yet powerful. The decorators to schema idea is understandable, features like authorization or validation are powerful yet simple. So do the pagination API has to be simple and easy to use, like the one in https://github.com/indigotech/graphql-schema-decorator.

[tonyxiao]

@19majkel94 what's the recommended way to do pagination until there's official support for pagination in type-graphql?

[tonyxiao]

Just tested - this seems to work for defining the graphql data models. Now going to experiment with how to actually use this efficiently with something like typeorm

import * as Relay from 'graphql-relay'
import { ArgsType, ClassType, Field, ObjectType } from 'type-graphql'

export type ConnectionCursor = Relay.ConnectionCursor

@ObjectType()
export class PageInfo implements Relay.PageInfo {
  @Field()
  hasNextPage!: boolean
  @Field()
  hasPreviousPage!: boolean
  @Field()
  startCursor?: ConnectionCursor
  @Field()
  endCursor?: ConnectionCursor
}

@ArgsType()
export class ConnectionArgs implements Relay.ConnectionArguments {
  @Field({ nullable: true, description: 'Paginate before opaque cursor' })
  before?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate after opaque cursor' })
  after?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate first' })
  first?: number
  @Field({ nullable: true, description: 'Paginate last' })
  last?: number
}

export function connectionTypes<T extends ClassType>(name: string, nodeType: T) {
  @ObjectType(`${name}Edge`)
  class Edge implements Relay.Edge<T> {
    @Field(() => nodeType)
    node!: T

    @Field({ description: 'Used in `before` and `after` args' })
    cursor!: ConnectionCursor
  }

  @ObjectType(`${name}Connection`)
  class Connection implements Relay.Connection<T> {
    @Field()
    pageInfo!: PageInfo

    @Field(() => [Edge])
    edges!: Edge[]
  }
  return {
    Connection,
    Edge,
  }
}

export {
  connectionFromArray,
  connectionFromPromisedArray,
  connectionFromArraySlice,
  connectionFromPromisedArraySlice,
} from 'graphql-relay'

[MichalLytek]

what's the recommended way to do pagination until there's official support for pagination in type-graphql?

Manually 😆

Just tested - this seems to work for defining the graphql data models.

That's right but without #180 you can't easily add some properties to edge or connection types.

I wonder if I should couple cursor pagination with graphql-relay making it a dependency or create my own helpers for returning paginated results 😕

[tonyxiao]

the implementation of pagination feels database dependent. I used this lib https://github.com/darthtrevino/relay-cursor-paging/ to interpret the ConnectionArgs into limit and offset, then used graphql-relay to turn result set into the correct form. non sql database may do it differently tho. that said no matter which one you choose I assume graphql-relay helpers can be applicable.

[cipriantarta]

Just tested - this seems to work for defining the graphql data models. Now going to experiment with how to actually use this efficiently with something like typeorm

How would I go about using that? 😬

[tonyxiao]

@cipriantarta
connectionPaging.ts

// Based on https://github.com/darthtrevino/relay-cursor-paging
import { ArgsType, Field } from 'type-graphql'

/**
 * TODO: Figure out how to validate this with class-validator
 * https://github.com/typestack/class-validator/issues/269
 */
@ArgsType()
export class ConnectionArgs implements ConnectionArguments {
  @Field({ nullable: true, description: 'Paginate before opaque cursor' })
  before?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate after opaque cursor' })
  after?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate first' })
  first?: number
  @Field({ nullable: true, description: 'Paginate last' })
  last?: number

  pagingParams() {
    return getPagingParameters(this)
  }
}

type PagingMeta =
  | { pagingType: 'forward'; after?: string; first: number }
  | { pagingType: 'backward'; before?: string; last: number }
  | { pagingType: 'none' }

function checkPagingSanity(args: ConnectionArgs): PagingMeta {
  const { first = 0, last = 0, after, before } = args
  const isForwardPaging = !!first || !!after
  const isBackwardPaging = !!last || !!before

  if (isForwardPaging && isBackwardPaging) {
    throw new Error('cursor-based pagination cannot be forwards AND backwards')
  }
  if ((isForwardPaging && before) || (isBackwardPaging && after)) {
    throw new Error('paging must use either first/after or last/before')
  }
  if ((isForwardPaging && first < 0) || (isBackwardPaging && last < 0)) {
    throw new Error('paging limit must be positive')
  }
  // This is a weird corner case. We'd have to invert the ordering of query to get the last few items then re-invert it when emitting the results.
  // We'll just ignore it for now.
  if (last && !before) {
    throw new Error("when paging backwards, a 'before' argument is required")
  }
  return isForwardPaging
    ? { pagingType: 'forward', after, first }
    : isBackwardPaging
      ? { pagingType: 'backward', before, last }
      : { pagingType: 'none' }
}

const getId = (cursor: ConnectionCursor) => parseInt(fromGlobalId(cursor).id, 10)
const nextId = (cursor: ConnectionCursor) => getId(cursor) + 1

/**
 * Create a 'paging parameters' object with 'limit' and 'offset' fields based on the incoming
 * cursor-paging arguments.
 *
 * TODO: Handle the case when a user uses 'last' alone.
 */
function getPagingParameters(args: ConnectionArgs) {
  const meta = checkPagingSanity(args)

  switch (meta.pagingType) {
    case 'forward': {
      return {
        limit: meta.first,
        offset: meta.after ? nextId(meta.after) : 0,
      }
    }
    case 'backward': {
      const { last, before } = meta
      let limit = last
      let offset = getId(before!) - last

      // Check to see if our before-page is underflowing past the 0th item
      if (offset < 0) {
        // Adjust the limit with the underflow value
        limit = Math.max(last + offset, 0)
        offset = 0
      }

      return { offset, limit }
    }
    default:
      return {}
  }
}

then in a custom repo.

  async findAndPaginate(conditions: FindConditions<T>, connArgs: ConnectionArgs) {
    const { limit, offset } = connArgs.pagingParams()
    const [entities, count] = await this.findAndCount({
      where: conditions,
      skip: offset,
      take: limit,
    })
    const res = connectionFromArraySlice(entities, connArgs, { arrayLength: count, sliceStart: offset || 0 })
    return extendPageInfo(res, {
      count,
      limit,
      offset,
    })
  }

[cipriantarta]

@tonyxiao Thanks for the details, I was mostly concerned about calling your method connectionTypes, but I think the signature should be export function connectionTypes<T extends ClassType>(name: string, nodeType: ClassType<T>), otherwise I get complains when trying to populate the edges.

[tonyxiao]

Actually, this is my latest version of that

import { TypeValue } from 'type-graphql/decorators/types'
export function connectionTypes<T extends TypeValue>(name: string, nodeType: T) {
  @ObjectType(`${name}Edge`)
  class Edge implements Relay.Edge<T> {
    @Field(() => nodeType)
    node!: T

    @Field({ description: 'Used in `before` and `after` args' })
    cursor!: ConnectionCursor

    @Field(() => GQLJSON)
    cursorDecoded() {
      return Relay.fromGlobalId(this.cursor)
    }
  }

  @ObjectType(`${name}Connection`)
  class Connection implements Relay.Connection<T> {
    @Field()
    pageInfo!: PageInfo

    @Field(() => [Edge])
    edges!: Edge[]
  }
  return {
    Connection,
    Edge,
  }
}

[cipriantarta]

Looks great. Thanks again Tony!

…

On Thu, 25 Oct 2018 at 19:58, Tony Xiao ***@***.***> wrote: Actually, this is my latest version of that import { TypeValue } from 'type-graphql/decorators/types'export function connectionTypes<T extends TypeValue>(name: string, nodeType: T) { @ObjectType(`${name}Edge`) class Edge implements Relay.Edge<T> { @field(() => nodeType) node!: T @field({ description: 'Used in `before` and `after` args' }) cursor!: ConnectionCursor @field(() => GQLJSON) cursorDecoded() { return Relay.fromGlobalId(this.cursor) } } — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#142 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA2skeM8TYtcrIyE53HDrAnE05Y6XOVOks5uoe23gaJpZM4WWrcs> .

[Veetaha]

@19majkel94 It would be great if we could get Class/FieldMetadata from classes decorated with @ObjectType, @Field and dynamically generate other @Object/InputType with statically unknown properties, so that implementing generic pagination with filtering higher order resolver would be possible. But now we can't inspect defined property names and types from the given @ObjectType class in generic context, which (I think) is not hard to expose. But there should be another way of defining @Object/InputType except via decorators. Could you please consider?

[MichalLytek]

@Veetaha
TypeGraphQL design goal is not to dynamically create derived types in runtime that can't be described in compile time (which means using any as an arg type).

I have in plans an API that would allow to transform types like the TS Partial, Pick, etc. For more advanced use cases, you may use a code generator that will parse the base model class and produce new derived types/classes.

[Veetaha]

@19majkel94 I understand that this was not your goal, though do you think this would be excessive? I am not a big fan of code-generation, because it only emphasizes that the given tool failed to reduce boilerplate. I don't see any obstacles for allowing dynamic creation of resolvers/types, as GrahQL has no generics and very restricted build-in tools to achieve it, so maintaining such a feature would lead to much less boilerplate. Please, don't refuse)

And why do you say

(which means using any as an arg type)

I try to avoid any type and use unknown if needed, moreover, when I stumble with any I just throw in some generics and get the strongest typing.

Once agian about dynamic resolvers/types. I suppose we can "add decorators dynamically" by manually invoking them with the proper target arguments e.g. Field(_type => Boolean)(ObjClass.prototype, 'propName'). But still, a disability to get ClassMetadata from the given constructor function limits metaprogramming (( Can you guarantee this feature in future or at least consider it?

[MichalLytek]

No. The goal is to have 1:1 mapping of TS types to GraphQL types. I will try to provide as much convenient tools for that, but I won't create a dynamic API for generating GraphQL types without TS types - you can use graphql-js for that. If you like that, I can provide an interop with it by passing type => GraphQLObjectType in decorator but it's a dirty and not recommended workaround.

[Qix-]

Having generators would be useful in certain cases. If I'm returning a somewhat large data set, I don't want to have to enumerate it all in memory before sending it to the client, but rather stream it. Generators would be wonderful here. I'm not sure if that's possible with Apollo, however.

[MichalLytek]

If you mean function * by generators, it's not possible, even by GraphQL spec (streaming chunks of data).

For large datasets, it may be better to just use pagination for manual paging.

[tonyxiao]

There is @defer directive though. Not quite pagination but maybe potentially relevant?

[Veetaha]

@19majkel94 Interesting to peek the definition of PaginationResponse<Type> as graphql doesn't support tuples, by the way, this a little bit annoying having to map typeorm's pagination tuple to { data: T[]; total: number; } object each time, but this is out of scope.

[subvertallchris]

@tonyxiao Any chance you have a full example of your approach in a gist or repo anywhere?

[ceefour]

@tonyxiao where did you import PageInfo from?

[tonyxiao]

@ceefour it's not imported, but part of the snippeet above. @subvertallchris we actually aren't using GraphQL anymore at the moment, so don't have easy access to a full example. :(

[Superd22]

@ceefour just fyi we've been using https://github.com/wemaintain/auto-relay in prod to great avail and are actively maintaining it.

[ceefour]

thanks @Superd22 , I've been writing the PageInfo etc. classes myself. not too bad actually although pretty repetitive. Your library seems very interesting and useful!

Definitely what I was looking for. (and what I expected at least core of it is in typegraphql)

[MichalLytek]

and what I expected at least core of it is in typegraphql

@ceefour
I'm a sole developer here with a full-time job. I would like to have everything in TypeGraphQL but I have to choose and give priorities to features 😞

[Superd22]

I 100% agree that this kind of "side" and somewhat opinionated (not everyone uses relay) features cannot be the priority when there's only one core contributor to an open source project.

The decision to create a separate package rather than submit a PR to this repo was made because we needed something working asap and it seemed to be better to wait until typegraphql ships in 1.0 and becomes a monorepo to implement this.

We 100% plan on helping porting some features of auto-relay into typegraphql if and when appropriate.

[MOTORIST]

@ceefour
Pagination has different implementations (offset, relay, ...) and not required to use ORM. What implementation should be in the typegraphql?

[ceefour]

@MOTORIST You're right. On second thought I'm already happy that the Relay choice is provided by @wemaintain's project.

Perhaps a better approach is to endorse that project from TypeGraphQL's documentation?

[lemiesz]

I think it would be better to have a TypeGQL-extensions (TypeGQL-plugins) repo. That could support custom utilities like this. While I would definitely use this pagination feature, I think its nicer to keep the core library minimal.

[MichalLytek]

Perhaps a better approach is to endorse that project from TypeGraphQL's documentation?

Sure we can make a section in docs for community libraries/plugins 😉
Feel free to make a PR if you find other extensions to TypeGraphQL.

[jnpco98]

I was trying to implement connections with @tonyxiao 's examples and this repo https://github.com/kazekyo/nestjs-graphql-relay, but I cant seem to work out how to use fromglobalid, toglobalid etc.

I was thinking of making an ID field resolver that converts { uuid/nanoid, created_at } ->json to a global id, then use fromglobalid to parse the data. But i just cant think of how I can use the getId, nextId func above to convert it to an offset.

Can someone fill me in on what i'm missing here. Thanks

[jnpco98]

I read through graphql-relay-js code and decided that the way i was implementing the whole thing is pretty incompatible with how the library expects the values. Decided to just write my own helper functions. Should've done this in the first place T_T

[theprobugmaker]

@jnpco98 I would be nice if you shared your solution. Thanks

[calmonr]

I needed to create a GraphQL server using TypeGraphQL that was relay-compliant. I found this issue (other issues), pull requests and even libraries that tried to solve the problem but most of them didn't (or not entirely).

I just created a repository with a relay-compliant GraphQL server using TypeGraphQL and TypeORM. It covers things such as global object identification and bi-directional relay cursor pagination (following the specification).

https://github.com/calmonr/typegraphql-relay

It's a boilerplate, not a library.

It has room for improvement (ordering, filtering, dataloader, more examples, etc) but most important parts are implemented and working properly.

I really liked the @Superd22 (auto-relay) implementation but it was inspired by @kazekyo and @9renpoto that has an issue when paginating. The user must specify before when paginating backwards (aka: a weird corner case). (good job tho)

The documentation is WIP but you can easily run it by using docker-compose up -d and yarn start:dev (http://127.0.0.1:8080/graphql)

Just updated the documentation.

Feel free to send suggestion, issues, pull requests.

Thank you.

cc @subvertallchris @ceefour @tonyxiao @zefexdeveloper @MOTORIST @jnpco98 @MichalLytek

[MaxAst]

It would be nice to have an API that makes it easy to implement a relay-based cursor pagination. Would be happy to contribute this, as I am implementing this right now following a similar approach to this. @MichalLytek what process shall we follow for this? Should I write an RFC with a simple example?

[calmonr]

Hi there @MaxAst. I agree with this. Unfortunately, I had to archive the repository as I'm not currently actively working with the language - JS/TS (or the technology - TypeGraphQL/TypeORM) anymore but I know it's an important topic and a common question. I hope this goes forward and I'm glad I could help somehow the community. 👍🏻

[MaxAst]

@calmonr no worries at all, you already helped the community a ton by providing this example implementation! :) I hope we can somehow bake it into type-graphql

[MichalLytek]

I plan to have such pagination helpers published as @typegraphql/pagination (or other name) separate package but all packed into a single monorepo.

@MaxAst yeah sure, go ahead and write about your concept, API, expectations. I've done some small helpers for an API with relay-like pagination so I'm familiar with this concept. Will take a look also at @calmonr work.

[johnsonjo4531]

I have created a relay pagination library that basically does all the work @tonyxiao did above. Here's the npm page, the github, and website.

It's basically just used like this:

import {
  Field,
  ObjectType,
} from "type-graphql";
import {
  ConnectionType,
  EdgeType
} from "typegraphql-relay-connections";

@ObjectType()
class Song {
  @Field()
  id!: string

  @Field()
  title!: string;
}

@ObjectType()
export class SongEdge extends EdgeType(Song) {}

@ObjectType()
export class SongConnection extends ConnectionType({
  edge: SongEdge,
  node: Song,
}) {}

Full Code Example

Also like was said above by tonyxiao the implementation of the paginate is dependent on your database. I already have something that returns a relay compatible return type for mongoose using a mongoose plugin which I call mongoose-relay-paginate.