refactor(server): Add ApiUrl + ServerUrl env + allow usage of https #8579

AMoreaux · 2024-11-19T10:47:33Z

Use SERVER_URL to define the NestJS instance URL
Add API_URL to set the public URL. Loadbalancer in production. In non-secure environments, it will be the same as the SERVER_URL
Allows setting an SSL certificate and using https.

Replaced individual environment-based server URL retrievals with a centralized ServerUrl utility. This change simplifies URL management and ensures a consistent approach across different services. Added validation for SSL configurations when using HTTPS.

Removed standalone ServerUrl and integrated with combined ServerUrl/ApiUrl module. Refactored codebase to use ApiUrl for public network accessibility and adjusted corresponding imports.

Renamed utils file for better readability and consistency. Updated all references to the new file name and added unit tests for ServerUrl and ApiUrl functionalities.

github-actions · 2024-11-19T10:48:27Z

	Warnings
⚠️	Changes were made to the environment variables, but not to the documentation - Please review your changes and check if a change needs to be documented!

Generated by 🚫 dangerJS against 79a8da2

greptile-apps

PR Summary

This PR adds HTTPS support and introduces separate URL management for server and API endpoints, with SSL certificate configuration and improved URL handling across the application.

Added SSL_KEY_PATH and SSL_CERT_PATH environment variables in /packages/twenty-server/.env.example for HTTPS support
Introduced ServerUrl and ApiUrl utilities in /packages/twenty-server/src/engine/utils/server-and-api-urls.ts for centralized URL management
Added validation in main.ts to ensure SSL certificates exist when using HTTPS protocol
Refactored services to use ApiUrl.get() instead of direct environment variable access for consistent URL handling
Added IPv6 support and protocol detection in URL handling with proper hostname normalization

_{9 file(s) reviewed, 15 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

packages/twenty-server/.env.example

packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts

...server/src/engine/core-modules/workspace-invitation/services/workspace-invitation.service.ts

packages/twenty-server/src/engine/utils/server-and-api-urls.ts

packages/twenty-server/src/main.ts

Move the reset of ServerUrl and ApiUrl to beforeEach in tests to ensure clean state before each test runs. Update example SSL_KEY_PATH and SSL_CERT_PATH to use relative paths for more portability.

Import the ApiUrl module and set the local URL in the setup function. This ensures the tests have the correct API endpoint configured.

Reordered import statements in `workspace-invitation.service.spec.ts` for improved code organization. This change ensures that dependencies are imported in a more logical sequence.

Included API_URL in self-hosting documentation for public endpoint configuration. This helps developers set up the correct URLs for API interactions in their self-hosted environments.

packages/twenty-server/.env.example

packages/twenty-server/src/main.ts

Removed protocol check logic from main.ts and enforced URL validation to require protocol directly in environment variables definition. This ensures consistent and secure URL formats throughout the application.

Add a new Bash script for generating self-signed SSL certificates, including a README with instructions. Supports customizable domain, root certificate name, and validity period, and integrates root certificate into macOS keychain.

charlesBochet · 2024-11-22T15:16:50Z

packages/twenty-server/src/engine/api/rest/rest-api.service.ts

@@ -6,8 +6,8 @@ import { AxiosResponse } from 'axios';

 import { Query } from 'src/engine/api/rest/core/types/query.type';
 import { getServerUrl } from 'src/utils/get-server-url';


add TODO + Deprecate flag

charlesBochet · 2024-11-22T15:32:35Z

packages/twenty-server/src/main.ts

+  // prevent ipv6 issue for redirectUri builder
+  url.hostname = url.hostname === '[::1]' ? 'localhost' : url.hostname;
+
+  ServerUrl.set(url.toString());


Make API_URL mandatory (maybe with default in environment.variable or in .env.example)

charlesBochet · 2024-11-22T15:33:11Z

packages/twenty-server/src/main.ts

@@ -68,7 +86,17 @@ const bootstrap = async () => {
    app.use(session(getSessionStorageOptions(environmentService)));
  }

-  await app.listen(process.env.PORT ?? 3000);
+  await app.listen(serverUrl.port, serverUrl.hostname);


this won't work for self-hosting configuration where both server and front are server by server + this will prevent internal curls

charlesBochet · 2024-11-22T15:36:34Z

packages/twenty-server/src/main.ts


 const bootstrap = async () => {
  const app = await NestFactory.create<NestExpressApplication>(AppModule, {
    cors: true,
    bufferLogs: process.env.LOGGER_IS_BUFFER_ENABLED === 'true',
    rawBody: true,
    snapshot: process.env.DEBUG_MODE === 'true',
+    ...(process.env.SERVER_URL &&
+    process.env.SERVER_URL.startsWith('https') &&
+    process.env.SSL_KEY_PATH &&


I would only make the check SSL_KEY_PATH and SSL_CERT_PATH

charlesBochet · 2024-11-22T15:36:59Z

packages/twenty-server/src/main.ts

+  ServerUrl.set(url.toString());
+  ApiUrl.set(environmentService.get('API_URL'));
+
+  logger.log(`Application is running on: ${url.toString()}`, 'Server Info');


API is running

charlesBochet · 2024-11-22T15:37:48Z

packages/twenty-website/src/content/developers/self-hosting/self-hosting-var.mdx

@@ -37,6 +37,7 @@ yarn command:prod cron:calendar:calendar-event-list-fetch
    ['REDIS_URL', 'redis://localhost:6379', 'Redis connection url'],
    ['FRONT_BASE_URL', 'http://localhost:3001', 'Url to the hosted frontend'],
    ['SERVER_URL', 'http://localhost:3000', 'Url to the hosted server'],
+    ['API_URL', 'http://my-load-balancer', 'Url to the public endpoint'],


URL to the API

charlesBochet · 2024-11-22T15:40:52Z

packages/twenty-website/src/content/developers/self-hosting/self-hosting-var.mdx

@@ -37,6 +37,7 @@ yarn command:prod cron:calendar:calendar-event-list-fetch
    ['REDIS_URL', 'redis://localhost:6379', 'Redis connection url'],
    ['FRONT_BASE_URL', 'http://localhost:3001', 'Url to the hosted frontend'],


this will be renamed FRONT_DOMAIN + default subDomain + protocol

redirect (workspace.subDomain . FRONT_DOMAIN)

Let's keep: FRONT_BASE_URL and extract defaultSubDomain + protocol + domain from there when we need it

in case we are not in multidomain, we should expect defaultSubDomain to be extratable from FRONT_BASE_URL

charlesBochet · 2024-11-22T15:58:50Z

packages/twenty-server/src/engine/utils/server-and-api-urls.ts

@@ -0,0 +1,37 @@
+// The url of the Server, should be exposed in a private network
+const ServerUrl = (() => {


remove these classes

charlesBochet · 2024-11-22T16:01:31Z

...server/src/engine/core-modules/workspace-invitation/services/workspace-invitation.service.ts

@@ -234,7 +235,7 @@ export class WorkspaceInvitationService {
          link: link.toString(),
          workspace: { name: workspace.displayName, logo: workspace.logo },
          sender: { email: sender.email, firstName: sender.firstName },
-          serverUrl: this.environmentService.get('SERVER_URL'),
+          serverUrl: ApiUrl.get(),


pass API_URL from environmentService

charlesBochet · 2024-11-22T16:01:51Z

...r/src/engine/core-modules/workspace-invitation/services/workspace-invitation.service.spec.ts

@@ -70,6 +71,7 @@ describe('WorkspaceInvitationService', () => {
    environmentService = module.get<EnvironmentService>(EnvironmentService);
    emailService = module.get<EmailService>(EmailService);
    onboardingService = module.get<OnboardingService>(OnboardingService);
+    ApiUrl.set('http://localhost:3000');


not need anymore

charlesBochet · 2024-11-22T16:02:02Z

packages/twenty-server/src/engine/core-modules/sso/services/sso.service.ts

@@ -199,7 +193,11 @@ export class SSOService {
  buildIssuerURL(
    identityProvider: Pick<WorkspaceSSOIdentityProvider, 'id' | 'type'>,
  ) {
-    return `${this.environmentService.get('SERVER_URL')}/auth/${identityProvider.type.toLowerCase()}/login/${identityProvider.id}`;
+    const authorizationUrl = new URL(ApiUrl.get());


pass API_URL from environmentService

charlesBochet · 2024-11-22T16:02:40Z

packages/twenty-server/src/engine/core-modules/open-api/open-api.service.ts

-      request,
-      this.environmentService.get('SERVER_URL'),
-    );
+    const baseUrl = getServerUrl(request, ApiUrl.get());


pass API_URL from environmentService

charlesBochet · 2024-11-22T16:03:30Z

packages/twenty-server/src/engine/api/rest/rest-api.service.ts

-      request,
-      this.environmentService.get('SERVER_URL'),
-    );
+    const baseUrl = getServerUrl(request, ApiUrl.get());


pass API_URL from environmentService ==> actually use the existing SERVER_URL

AMoreaux added 3 commits November 19, 2024 10:57

refactor: consolidate ServerUrl and ApiUrl handling

398364d

Removed standalone ServerUrl and integrated with combined ServerUrl/ApiUrl module. Refactored codebase to use ApiUrl for public network accessibility and adjusted corresponding imports.

refactor: rename serverAndApiUrl to server-and-api-urls

4fbaaac

Renamed utils file for better readability and consistency. Updated all references to the new file name and added unit tests for ServerUrl and ApiUrl functionalities.

AMoreaux mentioned this pull request Nov 19, 2024

feat(subdomain): use subdomain by workspace #8378

Closed

8 tasks

greptile-apps bot reviewed Nov 19, 2024

View reviewed changes

AMoreaux added 4 commits November 19, 2024 12:34

fix(server-setup): Adjust test hooks and SSL config paths

8a88f40

Move the reset of ServerUrl and ApiUrl to beforeEach in tests to ensure clean state before each test runs. Update example SSL_KEY_PATH and SSL_CERT_PATH to use relative paths for more portability.

test: Adjust WorkspaceInvitationService tests to set ApiUrl

886c6f2

Import the ApiUrl module and set the local URL in the setup function. This ensures the tests have the correct API endpoint configured.

[test] Reorder import statements

7ef1c73

Reordered import statements in `workspace-invitation.service.spec.ts` for improved code organization. This change ensures that dependencies are imported in a more logical sequence.

feat(self-hosting): Add API_URL to environment variables

a933d31

Included API_URL in self-hosting documentation for public endpoint configuration. This helps developers set up the correct URLs for API interactions in their self-hosted environments.

charlesBochet assigned Weiko Nov 19, 2024

charlesBochet added the -PR: awaiting review label Nov 19, 2024

Weiko reviewed Nov 20, 2024

View reviewed changes

packages/twenty-server/.env.example Outdated Show resolved Hide resolved

packages/twenty-server/src/main.ts Outdated Show resolved Hide resolved

AMoreaux added 2 commits November 20, 2024 15:36

fix(server): enforce protocol in environment URLs

8b40561

Removed protocol check logic from main.ts and enforced URL validation to require protocol directly in environment variables definition. This ensures consistent and secure URL formats throughout the application.

feat(ssl-generation): Add SSL generation script

8cf6502

Add a new Bash script for generating self-signed SSL certificates, including a README with instructions. Supports customizable domain, root certificate name, and validity period, and integrates root certificate into macOS keychain.

AMoreaux requested a review from Weiko November 20, 2024 14:41

chore(): remove comment

79a8da2

AMoreaux mentioned this pull request Nov 22, 2024

feat(*): add subdomain management for multiworkpsace #8680

Open

charlesBochet reviewed Nov 22, 2024

View reviewed changes

charlesBochet added -PR: awaiting author and removed -PR: awaiting review labels Nov 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(server): Add ApiUrl + ServerUrl env + allow usage of https #8579

refactor(server): Add ApiUrl + ServerUrl env + allow usage of https #8579

AMoreaux commented Nov 19, 2024 •

edited

Loading

github-actions bot commented Nov 19, 2024 •

edited

Loading

greptile-apps bot left a comment

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024 •

edited

Loading

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

charlesBochet Nov 22, 2024

		@@ -6,8 +6,8 @@ import { AxiosResponse } from 'axios';

		import { Query } from 'src/engine/api/rest/core/types/query.type';
		import { getServerUrl } from 'src/utils/get-server-url';

		@@ -37,6 +37,7 @@ yarn command:prod cron:calendar:calendar-event-list-fetch
		['REDIS_URL', 'redis://localhost:6379', 'Redis connection url'],
		['FRONT_BASE_URL', 'http://localhost:3001', 'Url to the hosted frontend'],

		@@ -0,0 +1,37 @@
		// The url of the Server, should be exposed in a private network
		const ServerUrl = (() => {

refactor(server): Add ApiUrl + ServerUrl env + allow usage of https #8579

Are you sure you want to change the base?

refactor(server): Add ApiUrl + ServerUrl env + allow usage of https #8579

Conversation

AMoreaux commented Nov 19, 2024 • edited Loading

github-actions bot commented Nov 19, 2024 • edited Loading

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charlesBochet Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AMoreaux commented Nov 19, 2024 •

edited

Loading

github-actions bot commented Nov 19, 2024 •

edited

Loading

charlesBochet Nov 22, 2024 •

edited

Loading