Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SD 272: decs integration #110

Merged
merged 61 commits into from
Apr 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
c1812d2
Add sql-producer behaviour and instructions for setting up local stac…
Mar 2, 2021
caf52b6
Add behaviour to convert form complaints data to fit DECS schema
Mar 8, 2021
7fa9ecf
Add tests checking complaint data formatting is correct
Mar 15, 2021
dc59eb4
Update node version and sql queue config
MaxOSullivan Mar 29, 2021
4d753af
Update eslint version and fix linting errors
Apr 13, 2021
8bb2634
Add function to convert enums across complaint types to base complain…
Apr 14, 2021
bec1e53
inject validator as argument to validAgainstSchema, add validatorFactory
Apr 15, 2021
147c0f9
Add logging of send to queue response
Apr 15, 2021
adb6736
changed preprod namespace to new preprod namespace
Apr 16, 2021
ab0a748
Merge pull request #113 from UKHomeOffice/SD-710-new-preprod-env
James-Hovell Apr 16, 2021
7e94623
SD-720: Fixed eslint errors and upgraded eslint
SarahJaneLuff Apr 16, 2021
0f4f072
Merge pull request #114 from UKHomeOffice/feature/SD-720-Update-Linti…
joehod Apr 16, 2021
ceac847
Add tests for send-to-sqs
Apr 16, 2021
56f16ff
Add meaningful error messages to util functions
Apr 16, 2021
bd50ba9
readme
Apr 16, 2021
9332cd7
Merge pull request #115 from UKHomeOffice/SD-710-new-preprod-env
James-Hovell Apr 16, 2021
2dc5afa
Add sql-producer behaviour and instructions for setting up local stac…
Mar 2, 2021
ee3a483
Add behaviour to convert form complaints data to fit DECS schema
Mar 8, 2021
ef7868c
Add tests checking complaint data formatting is correct
Mar 15, 2021
97e0962
Update node version and sql queue config
MaxOSullivan Mar 29, 2021
aa751ef
Update eslint version and fix linting errors
Apr 13, 2021
38ced3b
Add function to convert enums across complaint types to base complain…
Apr 14, 2021
683244a
inject validator as argument to validAgainstSchema, add validatorFactory
Apr 15, 2021
d2d779a
Add logging of send to queue response
Apr 15, 2021
17b336c
Add tests for send-to-sqs
Apr 16, 2021
26f30e9
Add meaningful error messages to util functions
Apr 16, 2021
19a3e91
rebase from master
Apr 19, 2021
e5ece04
Update package lock
Apr 19, 2021
33cc247
SD-725: Upgrade Node Version to 14
SarahJaneLuff Apr 19, 2021
3302c4d
SD-725: Corrected Typo Error in .drone.yml file for feature branch ev…
SarahJaneLuff Apr 19, 2021
a2a6656
Merge pull request #119 from UKHomeOffice/feature/SD-725-upgrade-Node…
sulthan-ahmed Apr 19, 2021
4dc8ea8
formatting for kube change
Apr 19, 2021
8a7409b
Merge pull request #120 from UKHomeOffice/SD-710-new-preprod-env
James-Hovell Apr 19, 2021
29d5113
SD-725: Added tagging conditional of kube repo in drone.yml file, to …
SarahJaneLuff Apr 19, 2021
b92e1e3
Merge pull request #121 from UKHomeOffice/feature/SD-725-upgrade-Node…
James-Hovell Apr 19, 2021
79923b9
Refactor to json converters to functional approach
Apr 20, 2021
8abd40d
Add sql-producer behaviour and instructions for setting up local stac…
Mar 2, 2021
861cfff
Add behaviour to convert form complaints data to fit DECS schema
Mar 8, 2021
7bcaeb2
Add tests checking complaint data formatting is correct
Mar 15, 2021
f7e8dbe
Update node version and sql queue config
MaxOSullivan Mar 29, 2021
6b20185
Update eslint version and fix linting errors
Apr 13, 2021
3f33545
Add function to convert enums across complaint types to base complain…
Apr 14, 2021
fa6c93a
inject validator as argument to validAgainstSchema, add validatorFactory
Apr 15, 2021
97ca852
Add logging of send to queue response
Apr 15, 2021
56eef15
Add tests for send-to-sqs
Apr 16, 2021
5ceadf3
Add meaningful error messages to util functions
Apr 16, 2021
fd2b544
rebase from master
Apr 19, 2021
86c159b
Add sql-producer behaviour and instructions for setting up local stac…
Mar 2, 2021
3dba7d9
Add behaviour to convert form complaints data to fit DECS schema
Mar 8, 2021
4bc627f
Add tests checking complaint data formatting is correct
Mar 15, 2021
7410edf
Update node version and sql queue config
MaxOSullivan Mar 29, 2021
5738272
Update eslint version and fix linting errors
Apr 13, 2021
502e0c9
Add function to convert enums across complaint types to base complain…
Apr 14, 2021
d9d2f1c
inject validator as argument to validAgainstSchema, add validatorFactory
Apr 15, 2021
5b492db
Add logging of send to queue response
Apr 15, 2021
fd0d8f0
Add tests for send-to-sqs
Apr 16, 2021
5014a42
Add meaningful error messages to util functions
Apr 16, 2021
ec748b4
Refactor to json converters to functional approach
Apr 20, 2021
fdfbf89
Fix merge conflicts with remote
Apr 20, 2021
4445851
Fix sendToQueue config option and add test
Apr 20, 2021
f8bb068
Add fallback stub email to config
Apr 20, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions .drone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ type: kubernetes
steps:
- name: build
pull: if-not-exists
image: node:10
image: node:14
commands:
- npm --loglevel warn install
- npm run test
when:
branch: [ master, features/* ]
branch: [ master, feature/* ]
event: push

- name: build_ukvi-complaints
Expand All @@ -21,7 +21,7 @@ steps:
- n=0; while [ "$n" -lt 60 ] && [ ! docker stats --no-stream ]; do n=$(( n + 1 )); sleep 1; done
- docker build -t app-$${DRONE_COMMIT_SHA} .
when:
branch: [ master, features/* ]
branch: [ master, feature/* ]
event: push

- name: image_to_quay
Expand Down Expand Up @@ -50,8 +50,7 @@ steps:
commands:
- git clone https://$${DRONE_GIT_USERNAME}:$${DRONE_GIT_TOKEN}@github.com/UKHomeOffice/kube-ukvi-complaints.git
- cd kube-ukvi-complaints
- git tag dev-${DRONE_COMMIT_SHA}
- git push origin --tags
- if ! [ $(git tag -l dev-${DRONE_COMMIT_SHA}) ]; then git tag dev-${DRONE_COMMIT_SHA} && git push origin --tags; fi
when:
branch: master
event: push
Expand All @@ -67,8 +66,7 @@ steps:
commands:
- git clone https://$${DRONE_GIT_USERNAME}:$${DRONE_GIT_TOKEN}@github.com/UKHomeOffice/kube-ukvi-complaints.git
- cd kube-ukvi-complaints
- git tag ${DRONE_DEPLOY_TO}-${DRONE_COMMIT_SHA}
- git push origin --tags
- if ! [ $(git tag -l ${DRONE_DEPLOY_TO}-${DRONE_COMMIT_SHA}) ]; then git tag ${DRONE_DEPLOY_TO}-${DRONE_COMMIT_SHA} && git push origin --tags; fi
when:
event: promote
target: [ preprod, prod ]
Expand Down Expand Up @@ -101,7 +99,7 @@ steps:
from_secret: kube_server_dev
KUBE_TOKEN_DEV:
from_secret: kube_token_dev
KUBE_NAMESPACE: ukvi-complaints-preprod
KUBE_NAMESPACE: sas-ukvi-complaints-preprod
APP_IMAGE_TAG: ${DRONE_COMMIT_SHA}
when:
event: promote
Expand Down
2 changes: 2 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ apps/**/acceptance/**/*.js
LICENSE
README.md
CONTRIBUTING.md
test/executions
test/coverage
24 changes: 24 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,27 @@ extends:

env:
es6: true
mocha: true

overrides:
- files:
"test/**/*.js"
rules:
implicit-dependencies/no-implicit: [error, { dev: true }]
max-nested-callbacks: off
complexity: [error, 40]
globals:
chai: true
sinon: true
expect: true
proxyquire: true
browser: true
page: true
baseURL: true
submitPage: true
getErrorSummaries: true
getErrorMessages: true
- files:
"apps/ukvi-complaints/behaviours/*.js"
rules:
complexity: [error, 40]
11 changes: 8 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
.DS_Store
apps/*/translations/en
node_modules
node_modules/
public
npm-debug.log
apps/*/translations/*
!apps/*/translations/src
.idea
.DS_Store
apps/.DS_Store
.nyc_output
.env
test/executions
test/coverage
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:10-alpine
FROM node:14-alpine

RUN apk upgrade --no-cache
RUN addgroup -S app
Expand Down
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Getting started


Get the project from Github
```bash
$ git clone [email protected]:UKHomeOffice/UKVI-Complaints.git && cd UKVI-Complaints
Expand All @@ -19,6 +20,37 @@ Run the services locally with Docker Compose
$ docker-compose up
```


### Set up AWS SQS queue locally

Run AWS services locally using localstack.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be able to explain the technical architecture before hand, e.g. the data gets sent to a AWS Queue in DECs service.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay yeah I can do that. Max and I were gonna make some changes to the README anyway so we can add more about that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you'll have time for this, so maybe just add this as part of a tech debt ticket for the decs intergration


```bash
$ docker run --rm -p 4566:4566 -p 4571:4571 localstack/localstack
```

To create an SQS queue on the localstack instance.
```bash
aws \
sqs create-queue \
--queue-name local-queue \
--endpoint-url http://localhost:4566 \
--region eu-west-2
```

This will return a url, add this is the url of the localstack sqs queue:
```
http://localhost:4566/000000000000/local-queue
```

To view what is on the queue currently run:
```bash
aws --endpoint-url=http://localhost:4566 --region eu-west-2 sqs receive-message --queue-url http://localhost:4566/000000000000/local-queue --max-number-of-messages 10
```


### Running in dev

Getting your hands dirty (You'll need [Redis](http://redis.io/) for this)
```bash
$ npm run dev
Expand Down
6 changes: 5 additions & 1 deletion apps/ukvi-complaints/behaviours/caseworker-email.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ const Emailer = require('hof-behaviour-emailer');
const path = require('path');
const moment = require('moment');

// eslint-disable-next-line complexity
const getDataRows = (model, translate) => {
return [
{
Expand Down Expand Up @@ -216,8 +217,11 @@ module.exports = config => {
// eslint-disable-next-line no-console
console.warn('WARNING: Email `from` address must be provided. Falling back to stub email transport.');
}

const transport = config.emailCaseworker ? config.transport : 'stub';

return Emailer(Object.assign({}, config, {
transport: config.from ? config.transport : 'stub',
transport: config.from ? transport : 'stub',
recipient: config.caseworker,
subject: (model, translate) => translate('pages.email.caseworker.subject'),
template: path.resolve(__dirname, '../emails/caseworker.html'),
Expand Down
4 changes: 3 additions & 1 deletion apps/ukvi-complaints/behaviours/conditional-content.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
'use strict';

module.exports = superclass => class extends superclass {

locals (req, res, next) {
locals(req, res, next) {
res.locals.isBehaviourIssue = req.form.values.reason === 'staff-behaviour';
res.locals.isRefundIssue = req.form.values.reason === 'refund';
res.locals.isApplicationIssue = req.form.values.reason === 'immigration-application';
Expand Down
1 change: 1 addition & 0 deletions apps/ukvi-complaints/behaviours/customer-email.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ const Emailer = require('hof-behaviour-emailer');
const path = require('path');
const moment = require('moment');

// eslint-disable-next-line complexity
const getDataRows = (model, translate) => {
return [
{
Expand Down
41 changes: 41 additions & 0 deletions apps/ukvi-complaints/behaviours/send-to-sqs.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
'use strict';
joehod marked this conversation as resolved.
Show resolved Hide resolved
const Validator = require('jsonschema').Validator;
const config = require('../../../config');
const { validAgainstSchema, sendToQueue } = require('../lib/utils');
const formatComplaintData = require('../lib/format-complaint-data');

module.exports = superclass => class SendToSQS extends superclass {

// eslint-disable-next-line consistent-return
saveValues(req, res, next) {
try {
if (!config.sendToQueue) {
return next();
}

const complaintData = formatComplaintData(req.sessionModel.attributes);
joehod marked this conversation as resolved.
Show resolved Hide resolved

if (validAgainstSchema(complaintData, new Validator())) {
return sendToQueue(complaintData)
.then(() => {
next();
})
.catch(err => {
SendToSQS.handleError(next, err);
});
}
} catch (err) {
SendToSQS.handleError(next, err);
}
}


static handleError(next, err) {
err.formNotSubmitted = true;
// eslint-disable-next-line no-console
console.error(err);
return next(err);
}

};

Loading