Url proxy endpoint featuring OpenAPI #141

kasperg · 2022-10-05T13:20:59Z

Link to issue

https://reload.atlassian.net/browse/DDFSOEG-245

Description

The purpose of this PR is twofold:

We need to expose an endpoint which supports transforming an "unproxied" url to a url which has been modified to support proxies used by a library agency for access control based on the configuration provided by the library. This endpoint can be used by the React components which render data containing unproxied urls. This is handled by the UrlProxyResource class.
We want a standardized approach to exposing functionality and data from Drupal to external parties. We have looked into our options and argue that we should use Drupals REST and OpenAPI modules.

This PR does the following:

Add and patch the required modules to provide a valid OpenAPI specification in composer.json
Replace the url proxy controller and routing with a resource and configuration and update tests accordingly
Add an initial OpenAPI 2.0 specification in openapi.json
Add a GitHub Actions job with tasks to ensure that our openapi.json specification stays valid and does not drift
Add documentation describing reasoning and future development process for API endpoints
Some other boy scout changes found when working with the codebase

Checklist

My complies with our coding guidelines.
My code passes our static analysis suite. If not then I have added a comment explaining why this change should be exempt from the code standards and process.
My code passes our continuous integration process. If not then I have added a comment explaining why this change should be exempt from the code standards and process.

and modified dpl_url_proxy setup/test

By using a ParamConverter and url encoding json. Following data works after /dpl-url-proxy/ : { "scheme": "http", "host": "john.com", "port": "", "path": "", "query": "kalle=1", "fragment": "" } It results in following url: %7B%0A%20%20%22scheme%22%3A%20%22http%22%2C%0A%20%20%22host%22%3A%20%22john.com%22%2C%0A%20%20%22port%22%3A%20%22%22%2C%0A%20%20%22path%22%3A%20%22%22%2C%0A%20%20%22query%22%3A%20%22kalle%3D1%22%2C%0A%20%20%22fragment%22%3A%20%22%22%0A%7D%0A PhpUnitTest breaks now though because: * The endpoint urls in tests needs to be updated with new url structure * The UrlParamConverter does not seem to be loaded in test context

It is nice to have a sensible order.

Otherwise we will not be able to use it with other programmatic tools.

This is necessary for us to be able to specify the API at a level where it is usable for us in client generation.

This way we avoid a lot of complexity caused by wonky encoding required by the url param converter. Patches to the OpenAPI REST module means that we will be able to specify query parameter structure anyway.

This is now handled by our rest resource instead.

There is no need for a double nested objects. An object with an url key should do fine. Specifying this is now supported through our patches to the OpenAPI REST module.

We need to explicitly specify the XDEBUG mode for this to work. - coverage: This gets us coverage needed for output - debug: This enables step debugging which can be helpful when writing unit tests

There is no need to keep it a secret and having it available is useful for automations.

This allows us to use the API with other tools without having a running site. Also it allows us to detect changes to the specification which may occur unintentionally as it partly autogenerated based on configuration. Our specification it somewhat handheld so ensuring that it is valid is also helpful. Node tools swagger-cli and jsome helps us with validation and formatting.

These are individual per site and should not be overwritten with future updates. By keeping the configuration a part of the ignored settings we ensure this.

This is in line with other modules

This allows us to return metadata about this response, links etc. Update the specification accordingly

They should not be enabled by default on production sites.

We have decided on using OpenAPI and REST for API development and specification. Thus we no longer need modules related to JSON:API. OpenAPI UI Redoc is not necessary for us to work with or expose our API so remove it entirely. This reduces our code surface.

Ddfsoeg 245 part 2

This is not needed for JavaScript code - only JSON.

The default 4x throttle is not relevant when running Lighthouse in containers or in GitHub Actions.

The cpu multiplier value must be wrapped in a throttling entry.

It has been replaced by the url proxy resource

Use query args as cache context. It is more specific to what we are doing. Also remember to use snake_case instead of camelCase for variables.

config/sync/user.role.anonymous.yml

config/sync/user.role.authenticated.yml

spaceo and others added 30 commits October 3, 2022 17:11

Adding experiment with openapi / openapi_rest

b40aaf6

and modified dpl_url_proxy setup/test

Correcting UrlParamConverter::convert return type

2a36181

Make sure url proxy endpoint is varying on url

d616550

Reorder patched projects alphabetically

de23403

It is nice to have a sensible order.

Patch OpenAPI module to ensure it generates a valid spec

35c3b5f

Otherwise we will not be able to use it with other programmatic tools.

Patch OpenAPI REST module to support parameter and response specs

648a053

This is necessary for us to be able to specify the API at a level where it is usable for us in client generation.

Implement parameter for url proxy resource as plain GET parameter

73024ce

This way we avoid a lot of complexity caused by wonky encoding required by the url param converter. Patches to the OpenAPI REST module means that we will be able to specify query parameter structure anyway.

Remove obsolete routing for url proxy module

21c9d1d

This is now handled by our rest resource instead.

Update url proxy response format and add specification

b5c9ef6

There is no need for a double nested objects. An object with an url key should do fine. Specifying this is now supported through our patches to the OpenAPI REST module.

Do not modify urls if no proxy expression is configured

284c931

Fix coverage in text format when running PHPUnit tests

3be9f4b

We need to explicitly specify the XDEBUG mode for this to work. - coverage: This gets us coverage needed for output - debug: This enables step debugging which can be helpful when writing unit tests

Allow anonymous users to access our API documentation

97adfd1

There is no need to keep it a secret and having it available is useful for automations.

Do not not(!) ignore url proxy settings

dee8752

These are individual per site and should not be overwritten with future updates. By keeping the configuration a part of the ignored settings we ensure this.

Use DPL prefix for url proxy module name

018ffbf

This is in line with other modules

Fix url proxy resource code style

6b45d95

Fix UrlProxyResource test code style

bbcd403

Rewrap response in data

a5cab4f

This allows us to return metadata about this response, links etc. Update the specification accordingly

Disable modules related to development of REST services

1b015ba

They should not be enabled by default on production sites.

Remove obsolete modules

0f9c6bf

We have decided on using OpenAPI and REST for API development and specification. Thus we no longer need modules related to JSON:API. OpenAPI UI Redoc is not necessary for us to work with or expose our API so remove it entirely. This reduces our code surface.

Merge pull request #2 from reload/DDFSOEG-245-part-2

8fefa5b

Ddfsoeg 245 part 2

Add initial ADR to document the usage of REST and OpenAPI

ab8c54e

Link to both Orval and GraphQL definitions in API spec ADR

31ec6cf

Do not quote keys in Lighthouse configuration

baee32e

This is not needed for JavaScript code - only JSON.

Reduce Lighthouse throttling

afbfdae

The default 4x throttle is not relevant when running Lighthouse in containers or in GitHub Actions.

Fix Lighthouse throttling

758f7e9

The cpu multiplier value must be wrapped in a throttling entry.

Fix link to Taskfile in ADR

5a3934c

Make sure that url is defaulting to url_param

a6752bd

Add developer documentation for implementing new APIs

d2bf748

kasperg added 2 commits October 5, 2022 15:12

Add proper description of UrlProxyResource

63164f1

Remove obsolete url proxy controller

e7a65ce

It has been replaced by the url proxy resource

github-actions bot temporarily deployed to pr-141 October 5, 2022 13:22 Inactive

kasperg marked this pull request as ready for review October 5, 2022 13:26

Update url proxy resource cache tag handling

8fb9c28

Use query args as cache context. It is more specific to what we are doing. Also remember to use snake_case instead of camelCase for variables.

github-actions bot temporarily deployed to pr-141 October 6, 2022 06:38 Inactive

cableman suggested changes Oct 6, 2022

View reviewed changes

config/sync/user.role.anonymous.yml Outdated Show resolved Hide resolved

config/sync/user.role.authenticated.yml Outdated Show resolved Hide resolved

Revert user role names to english

e4ec424

github-actions bot temporarily deployed to pr-141 October 6, 2022 13:48 Inactive

cableman approved these changes Oct 6, 2022

View reviewed changes

kasperg merged commit 4892d46 into release/4 Oct 6, 2022

kasperg deleted the opsaet-openapi-understottelse-og-ensretning-af-rest-api-opsaetning-DDFSOEG-245 branch October 6, 2022 13:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Url proxy endpoint featuring OpenAPI #141

Url proxy endpoint featuring OpenAPI #141

kasperg commented Oct 5, 2022 •

edited by Dresse

Loading

Url proxy endpoint featuring OpenAPI #141

Url proxy endpoint featuring OpenAPI #141

Conversation

kasperg commented Oct 5, 2022 • edited by Dresse Loading

Link to issue

Description

Checklist

kasperg commented Oct 5, 2022 •

edited by Dresse

Loading