Feature/new integrations script (#272)

* Move discord

* Use standard var

* Better error handling

* Delete discord

* Move Sendgrid to app router

* Fix export

* Upgrade routes to return 400 on missing params

* Add status codes

* Check status returned

* Test settings getter

* Feature/posthog frontend (#242)

* Create param validator

* Move discord

* Move user settings getter to app router

* Use standard var

* move getAllPublicUserData to app router

* Better error handling

* Delete getAllData.ts

* Delete discord

* Throw errors

* Handle error throwing

* Move updateSettings to app router

* Update confluence.svg

* Configure jest

* Create settings.test.ts

* Create updateSettings.test.ts

* Add getting test

* Use generalized validator

* Move Stripe to app router

* Move Sendgrid to app router

* Move vscode to airtable Analytics to app router

* Frontend analytics

* Change export

* Fix export

* Fix export

* Feature/posthog backend (#243)

* Create param validator

* Move discord

* Move user settings getter to app router

* Use standard var

* move getAllPublicUserData to app router

* Better error handling

* Delete getAllData.ts

* Delete discord

* Throw errors

* Handle error throwing

* Move updateSettings to app router

* Update confluence.svg

* Configure jest

* Create settings.test.ts

* Create updateSettings.test.ts

* Add getting test

* Use generalized validator

* Move Stripe to app router

* Move Sendgrid to app router

* Move vscode to airtable Analytics to app router

* Create posthog.ts

* Add types, move to utils

* Capture posthog event

* Change export

* Fix export

* Fix export

* Check dev env

* Extract response types

* Use responseTypes

* Fix settings to use data response

* Early return if no apiKey

* Add error events

* Remove event logging

* Use request URL to pass to posthog

* Extract tracking

* Use tracker object

* Move to route

* Fix typo

* Create route groups

* Fix imports

* Delete layout.tsx

* Create AsanaLoginLink.tsx

* Create guide to adding a new Oauth service

* Add asana

* Add asana

* Add asana

* Add SQL for new services

* Add env vars sections

* Update CONTRIBUTING.md

* Add lang descriptors

* Create script to allow quick integrations

* Update CONTRIBUTING.md

* Delete discord

* Fix tracking

* Fix import

* Fix import

* Roll back hover to dev

* Fix tracking

* Fix typo

* Extract service list

* Use correct prop

* Simplify code

* Data reorg

* Fix param for getting user data

* Update CONTRIBUTING.md

* Improve text

* Improve SQL section

* Update CONTRIBUTING.md

* Add last steps

CONTRIBUTING.md

@@ -12,29 +12,29 @@ Anyone is free to contribute changes to any file in this repository. You don't n
 
 To start developing, clone and:
 
-```
+```bash
 yarn
 yarn dev
 ```
 
 Or with npm
 
-```
+```bash
 npm i
 npm run dev
 ```
 
 (Check your node version, we recommend 18)
 
-We use a recent version of Next. You may refer to the documentation at https://nextjs.org/docs/.
+We use a recent version of Next. You may refer to the [documentation](https://nextjs.org/docs/).
 
 This repo is automatically deployed on vercel to [app.watermelontools.com](app.watermelontools.com) on merges to `main`.
 
 All the backend lives as serverless functions under `api`, with the route being the filename.
 
-We now use the new app router for some features.
+We now use the new app router for _all_ features.
 
-As we now use OAuth2.0, local development cannot be done on new integrations.
+As we use OAuth2.0, local development cannot be done on new integrations.
 
 All environment vars are on vercel, the committer is responsible for correct deployments.
 
@@ -52,7 +52,7 @@ We have a `utils` folder that includes all the business logic. We have an `api`
 
 The developer has to match the `utils` folder structure to the `api` route schema. This makes it easier to maintain.
 
-> As an example, we have `utils/user/getProfile.ts` that is imported in `pages/api/user/getProfile.ts` and returns a `types/UserProfile.ts`. In the database, you will find a _user_ table with all the data on the type.
+> As an example, we have `utils/user/getProfile.ts` that is imported in `app/api/user/getProfile.ts` and returns a `types/UserProfile.ts`. In the database, you will find a _user_ table with all the data on the type.
 
 We do all of this as a security measure. We don't want data exposed and we consider our backend safe.
 
@@ -64,6 +64,160 @@ First, we use oauth so you need to ensure that the service supports it.
 
 Remember that there are several procedures in our db to replicate.
 
+The steps to do so are:
+
+- Publicly announce the new integration
+  > serviceName refers to the shortname, like `github` or `gitlab`, lowercase
+  > ServiceReadableName refers to the name of the service and it's use in the UI, like `GitHubPRs` or `LinearTasks`
+- Set the necesary vercel env vars
+  > usually `SERVICE_CLIENT_SECRET` and `SERVICE_CLIENT_ID`
+- Create the table in our DB
+
+  > **This is sketch of a possible new table**
+  >
+  > _Add any other required columns as needed_
+
+```sql
+  CREATE TABLE serviceName (
+    id INT PRIMARY KEY,
+    name VARCHAR(255),
+    email VARCHAR(255),
+    updated_at DATETIME DEFAULT GETDATE() NOT NULL,
+    created_at DATETIME DEFAULT GETDATE() NOT NULL,
+    access_token VARCHAR(255),
+    refresh_token VARCHAR(255),
+    avatar_url VARCHAR(255),
+    workspace VARCHAR(255),
+    workspace_image VARCHAR(255),
+    watermelon_user VARCHAR(255),
+    deleted BIT DEFAULT 0 NULL,
+    deleted_at DATETIME DEFAULT GETDATE() NULL,
+    FOREIGN KEY (watermelon_user) REFERENCES watermelon.dbo.users(id)
+    );
+```
+
+- Edit the userSettings table
+
+  ```sql
+  ALTER TABLE userSettings ADD ServiceReadableName INT DEFAULT 3;`
+  ```
+
+  ```sql
+  UPDATE userSettings SET ServiceReadableName = 3 WHERE ServiceReadableName IS NULL;
+  ```
+
+- Create the necessary procedures
+- Setting the information for the user using `create_serviceName`
+
+  ```sql
+  CREATE PROCEDURE dbo.create_serviceName
+      @access_token varchar(255),
+      @id varchar(255),
+      @name varchar(255),
+      @displayName varchar(255),
+      @email varchar(255),
+      @avatarUrl varchar(255),
+      @team_id varchar(255),
+      @team_name varchar(255),
+      @watermelon_user varchar(255)
+  AS
+  DECLARE @insertTable TABLE (
+      access_token varchar(255),
+      id varchar(255),
+      name varchar(255),
+      displayName varchar(255),
+      email varchar(255),
+      avatarUrl varchar(255),
+      team_id varchar(255),
+      team_name varchar(255),
+      watermelon_user varchar(255)
+  )
+
+  DECLARE @wmid VARCHAR(255) = (
+  SELECT
+    id
+  FROM
+    dbo.users
+  WHERE
+    email = @watermelon_user
+  )
+
+  INSERT
+    INTO
+    dbo.serviceName
+      (
+      access_token,
+    id,
+    name,
+    displayName,
+    email,
+    avatarUrl,
+    team_id,
+    team_name,
+    watermelon_user
+      )
+  OUTPUT
+  inserted.access_token,
+    inserted.id,
+    inserted.name,
+    inserted.displayName,
+    inserted.email,
+    inserted.avatarUrl,
+    inserted.team_id,
+    inserted.team_name,
+    inserted.watermelon_user
+      INTO
+    @insertTable
+  VALUES
+      (
+      @access_token,
+      @id,
+      @name,
+      @displayName,
+      @email,
+      @avatarUrl,
+      @team_id,
+      @team_name,
+      @wmid
+      )
+
+  SELECT
+    *
+  FROM
+    @insertTable
+  FOR JSON PATH,
+  WITHOUT_ARRAY_WRAPPER
+
+  ```
+
+- Fetching the settings is unchanged as we use the same procedure for all services
+- Edit the settings getter
+- Fetching the tokens
+
+  - Edit the procedure `get_all_user_tokens` to match the service
+  - Edit the procedure `get_all_tokens_from_gh_username` to match the service
+
+- Create a `ServiceLoginLink.ts` component in the `components` folder
+- Add the service to the `loginArray.tsx` file
+- Add the service to the `loginGrid.tsx` file in the correct section
+- Add the service to `form.tsx` under _settings_
+
+- Run the `newIntegration.sh` script which will:
+
+  - Create the service folder under `(loggedIn)`
+  - Copy the `loading.tsx` from any other service
+  - Create the function under `/utils/db/service/saveUser` that you need to complete
+  - Populate the `page.tsx` file to be finished the correct service parameters
+  - Create an empty getter in `/utils/actions`
+
+- Now you need to edit the `page.tsx` file to match the service
+- Get the data in the `getService.tsx` file under actions
+- Add to the action log
+- Pass the data to the AI in `utils/actions/getOpenAISummary.ts` file
+- Return the data as Markdown in `api/actions/github/route.tsx`
+- Return one result in `api/hover/route.tsx`
+- Return the settings decided results in `api/extension/route.tsx`
+
 ## Issues
 
 If there's something you'd like to see please [open an issue](https://github.com/watermelontools/watermelon/issues/new).

app/(loggedIn)/atlassian/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/atlassian/page.tsx

@@ -23,7 +23,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = isConfluence ? "Confluence" : "Jira";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/bitbucket/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/bitbucket/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "Bitbucket";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/github/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/github/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "GitHub";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/gitlab/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/gitlab/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "GitLab";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/jira/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/jira/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "Jira";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/linear/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/linear/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "Linear";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/notion/loading.tsx

@@ -1,5 +1,5 @@
 import LoadingConnectedService from "../../../components/services/loading";
 
-export default function loadingConnetedService() {
+export default function loadingConnectedService() {
   return <LoadingConnectedService />;
 }

app/(loggedIn)/notion/page.tsx

@@ -21,7 +21,7 @@ export default async function ServicePage({
   // change service name
   const serviceName = "Notion";
   const [userData, serviceToken] = await Promise.all([
-    getAllPublicUserData({ userEmail }).catch((e) => {
+    getAllPublicUserData({ email: userEmail }).catch((e) => {
       console.error(e);
       return null;
     }),

app/(loggedIn)/page.tsx

@@ -13,10 +13,11 @@ async function HomePage({}) {
   const userEmail = session?.user?.email;
   const userName = session?.user?.name;
   // if not logged in, do not show anything
-  const data = await getAllPublicUserData({ userEmail }).catch((e) => {
+  const data = await getAllPublicUserData({ email: userEmail }).catch((e) => {
     console.error(e);
     return null;
   });
+  console.log(data);
   const comingSoon = [
     "PHPStorm",
     "IntelliJ",