Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat: Grant activity email trigger #3571 #3647

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Feat: Grant activity email trigger #3571 #3647

wants to merge 9 commits into from

Conversation

greg-adams
Copy link
Contributor

@greg-adams greg-adams commented Oct 15, 2024 •
edited
Loading

Ticket #3571

Description

  • Adds queries to gather email recipients and grant activity for each recipient of daily digest email
  • Adds new instance of terraform scheduled ecs module to schedule cron task

Screenshots / Demo Video

Testing

  • Tested using Mailpit
  • Testing can be manually triggered by running docker compose exec app yarn workspace server node ./src/scripts/sendGrantActivityDigestEmail.js. The time window (periodStart/periodEnd) can be adjusted in this file if needed.
    • The env variable ENABLE_GRANT_ACTIVITY_DIGEST_SCHEDULED_TASK must be enabled

Automated and Unit Tests

  • Added Unit tests

Manual tests for Reviewer

  • Added steps to test feature/functionality manually

Checklist

  • Provided ticket and description
  • Provided screenshots/demo
  • Provided testing information
  • Provided adequate test coverage for all new code
  • Added PR reviewers

@github-actions github-actions bot added enhancement New feature or request Infra Issues related to the infrastructure underlying all the tools. javascript Pull requests that update Javascript code terraform Pull requests that update Terraform code labels Oct 15, 2024
@greg-adams greg-adams self-assigned this Oct 15, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 15, 2024
View reviewed changes
packages/server/__tests__/email/email.test.js Fixed Show fixed Hide fixed
packages/server/src/lib/email.js
});

// TODO: add plain text version of the email
const emailPlain = emailHTML.replace(/<[^>]+>/g, '');

Check failure

Code scanning / CodeQL

Incomplete multi-character sanitization High

This string may still contain
<script
Loading
, which may cause an HTML element injection vulnerability.

Copilot Autofix AI about 1 month ago

To fix the problem, we should replace the current regular expression-based sanitization with a well-tested library that can handle HTML sanitization more effectively. One such library is sanitize-html, which is designed to remove unsafe HTML tags and attributes while preserving safe content.

Steps to fix:

  1. Install the sanitize-html library.
  2. Replace the current regular expression-based sanitization with a call to sanitize-html.
Suggested changeset 2
packages/server/src/lib/email.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/packages/server/src/lib/email.js b/packages/server/src/lib/email.js
--- a/packages/server/src/lib/email.js
+++ b/packages/server/src/lib/email.js
@@ -565,3 +565,7 @@
     // TODO: add plain text version of the email
-    const emailPlain = emailHTML.replace(/<[^>]+>/g, '');
+    const sanitizeHtml = require('sanitize-html');
+    const emailPlain = sanitizeHtml(emailHTML, {
+        allowedTags: [], // Remove all HTML tags
+        allowedAttributes: {} // Remove all attributes
+    });
 
EOF
@@ -565,3 +565,7 @@
// TODO: add plain text version of the email
const emailPlain = emailHTML.replace(/<[^>]+>/g, '');
const sanitizeHtml = require('sanitize-html');
const emailPlain = sanitizeHtml(emailHTML, {
allowedTags: [], // Remove all HTML tags
allowedAttributes: {} // Remove all attributes
});

packages/server/package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/packages/server/package.json b/packages/server/package.json
--- a/packages/server/package.json
+++ b/packages/server/package.json
@@ -87,3 +87,4 @@
     "uuid": "^8.3.2",
-    "xlsx": "^0.18.5"
+    "xlsx": "^0.18.5",
+    "sanitize-html": "^2.13.1"
   },
EOF
@@ -87,3 +87,4 @@
"uuid": "^8.3.2",
"xlsx": "^0.18.5"
"xlsx": "^0.18.5",
"sanitize-html": "^2.13.1"
},
This fix introduces these dependencies
Package Version Security advisories
sanitize-html (npm) 2.13.1 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 15, 2024 •
edited
Loading

QA Summary

QA Check Result
🌐 Client Tests ✅
🔗 Server Tests ✅
🤝 E2E Tests ✅
📏 ESLint ✅
🧹 TFLint ✅

Test Coverage

Coverage report for `packages/client`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 35.54 34.41 32.34 37.12
🔴  src 0 100 100 0
🔴   App.vue 0 100 100 0 2-9
🔴  src/arpa_reporter 0 100 100 0
🔴   App.vue 0 100 100 0 2-13
🟡  ...ter/components 58.58 48 46.8 59.78
🟡   AlertBox.vue 80 75 50 80 13
🔴   ...oadButton.vue 20 0 0 20 2-7,38-67
🟢   ...ileButton.vue 100 100 100 100
🟢   ...ttonSmall.vue 100 100 100 100
🟢   ...mplateBtn.vue 100 100 100 100
🟡   ...avigation.vue 67.74 63.63 52.63 67.74 ...13-219,228-235
🔴   StandardForm.vue 45 50 41.66 45.45 ...24-128,135-157
🟢  ...porter/helpers 84.61 79.48 87.5 84.61
🟢   form-helpers.js 84.21 79.48 85.71 84.21 7,16,25,81-83
🟢   short-uuid.js 100 100 100 100
🔴  ...eporter/router 0 0 0 0
🔴   index.js 0 0 0 0 20-131
🔴  ...reporter/store 3.92 0 2.17 4.12
🔴   index.js 3.92 0 2.17 4.12 11-14,32-261
🔴  ...reporter/views 40.98 25.13 41.37 42.91
🟢   AgenciesView.vue 100 0 100 100 16
🔴   AgencyView.vue 40.74 36.36 50 45.83 53-62,70-96
🔴   HomeView.vue 26.66 34.14 36.36 26.66 36-66,113,137-207
🔴   LoginView.vue 29.62 36.36 20 30.76 1,4,19-34,72-100
🔴   ...plateView.vue 27.02 35.71 45.45 25 ...2,30-37,69-113
🔴   ...ploadView.vue 24.24 16.66 33.33 26.66 1,30-31,116-144
🔴   ...eriodView.vue 48 18.18 75 52.17 64-90
🟡   ...riodsView.vue 57.57 28.57 60 59.37 101,132,149-171
🔴   ...pientView.vue 40 10.52 30.76 41.02 73-93,110-152
🔴   ...ientsView.vue 48.48 8.33 46.66 53.33 ...46,165-190,203
🟡   UploadView.vue 61.53 43.75 62.5 61.36 ...41-442,448-449
🔴   UploadsView.vue 16.66 0 0 18.18 59,110-287
🔴   UserView.vue 46.34 33.33 68.75 48.64 84,97-137
🟡   UsersView.vue 60 7.69 62.5 66.66 91-92,105-107,123
🔴   ...ationView.vue 37.03 18.18 26.66 41.66 ...19,238,246-270
🟡  src/components 53.6 44.2 55.31 54.31
🔴   ...vityTable.vue 18.75 0 28.57 19.35 115-185
🟡   BaseLayout.vue 69.56 53.84 60 69.56 172,219-231
🔴   CopyButton.vue 30 33.33 40 30 5,53-62
🟢   ...tActivity.vue 80.76 62.85 83.33 80.76 ...31-136,179,201
🟡   GrantNotes.vue 78.57 76.19 77.77 78.04 163-165,184-195
🔴   GrantsTable.vue 45.03 38.77 45 46.85 ...51-455,463-543
🔴   ...dUploader.vue 24.24 33.33 37.5 24.24 16,25,74-111
🔴   SearchFilter.vue 40.74 28 37.5 40 ...52,64,67,72-82
🔴   ShareGrant.vue 38.7 64.28 23.07 40 ...08-112,131-152
🟢   ...ivityItem.vue 96.15 90.9 100 96.15 64
🟡   UserAvatar.vue 70 75 100 62.5 35-37
🟢   ...eaderText.vue 100 100 100 100
🔴  ...ponents/Modals 29.21 44.34 33.1 32.94
🔴   ...anization.vue 17.39 65.62 30 22.22 1-14,149-178
🔴   AddTeam.vue 45.45 55.55 61.53 57.69 ...04,210,222-245
🔴   AddUser.vue 36.66 66.66 60 37.5 ...40,145,148-176
🔴   ...anization.vue 15.38 64.28 14.28 16.66 1-15,58-78
🔴   EditTeam.vue 18.18 26.43 33.33 20.83 ...29,208,216-301
🔴   EditUser.vue 21.05 66.66 25 22.22 1,101-128
🔴   ...ilsLegacy.vue 22.03 0 0 24.07 131,177,205-369
🟢   ...Followers.vue 83.33 88.88 57.14 83.33 10,114-115
🟡   ImportTeams.vue 50 41.17 50 53.33 28,64-69,81-82
🔴   ImportUsers.vue 42.85 60 40 46.15 29,65-80
🔴   ...archPanel.vue 27.9 15.78 23.52 28.57 ...68-178,211-255
🔴   SearchPanel.vue 21.62 63.26 32 27.58 ...77-380,386-458
🔴  src/helpers 18.12 17.92 18.6 18.88
🟢   constants.js 100 100 100 100
🟢   currency.js 100 100 100 100
🟡   dates.js 66.66 100 33.33 100
🔴   fetchApi.js 6 13.79 5.26 6.12 10-12,20-132
🔴   filters.js 4 0 0 4.54 19-51
🔴   form-helpers.js 0 0 0 0 5-82
🟡   gtag.js 77.77 90 75 77.77 12,51
🟢   testHelpers.js 100 100 100 100
🔴   ...patWarning.js 0 0 0 0 39-61
🟢  ...s/featureFlags 84.61 100 71.42 84.61
🟡   index.js 60 100 60 60 8,16
🟢   utils.js 100 100 100 100
🔴  src/mixin 20 0 28.57 20
🔴   ...zableTable.js 20 0 28.57 20 16-31,36-37,42
🔴  src/router 18.91 14.28 11.11 18.91
🔴   index.js 18.91 14.28 11.11 18.91 ...76-177,181-200
🟢  src/store 100 100 100 100
🟢   index.js 100 100 100 100
🔴  src/store/modules 2.89 0 4.51 3.01
🔴   agencies.js 5.26 100 8.33 5.55 13-70
🔴   alerts.js 20 100 20 20 10-24
🔴   grants.js 1.03 0 0.98 1.06 61-445
🔴   organization.js 33.33 100 33.33 33.33 21-25
🔴   roles.js 20 100 20 25 13-22
🔴   tenants.js 11.11 100 14.28 12.5 13-32
🔴   users.js 2.43 0 4.76 2.5 17-100
🔴  src/views 44.73 37.22 33.77 46.55
🔴   ...orterView.vue 25.58 51.85 18.18 26.82 ...,62,84,109-151
🟡   ...boardView.vue 50 17.64 50 52 89-98,114-125
🔴   ...tailsView.vue 32.97 14.28 17.07 33.69 ...96-435,441-462
🟢   GrantsView.vue 100 100 100 100
🟡   LoginView.vue 56 38.88 44.44 58.33 23,137-159
🟡   MyGrantsView.vue 77.77 66.66 66.66 77.77 1,69
🟡   ...ofileView.vue 78.26 80 42.85 78.26 1,32,63,136-140
🟢   NotFoundView.vue 100 100 100 100
🔴   ...tionsView.vue 47.05 57.14 41.66 53.33 ...97-100,114-118
🔴   ...ivityView.vue 46.42 23.8 43.75 46.42 ...01,114,120-134
🔴   TeamsView.vue 44.44 88.88 41.66 53.33 1,58,142,156-163
🟡   UsersView.vue 50 66.66 36.36 53.84 ...16-121,133-139
Coverage report for `packages/server`
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🟡 All files 60.44 53.38 56.55 60.49
🟢  src 81.63 33.33 60 81.63
🟢   configure.js 81.63 33.33 60 81.63 42,61-68,97-99
🟢  src/arpa_reporter 98.75 66.66 100 98.75
🟢   configure.js 97.36 40 100 97.36 36
🟢   environment.js 100 100 100 100
🟢   use-request.js 100 100 100 100
🟡  src/arpa_reporter/db 50.73 45.88 50 51.9
🟡   arpa-subrecipients.js 53.19 50 38.46 54.54 23-60,101,113-122
🔴   reporting-periods.js 37.2 46.87 40 38.09 46,77-156
🟢   settings.js 100 83.33 100 100 13
🟡   uploads.js 50 28.57 52.38 51.42 ...9,84,99-124,141-150
🔴  src/arpa_reporter/lib 29.57 33.08 34.56 28.46
🟢   arpa-ec-codes.js 100 100 100 100
🔴   audit-report.js 21.44 19.35 24.19 21.32 ...529,554-684,732-758
🟡   ensure-async-context.js 75 100 50 100
🟢   format.js 90.62 90 90 91.3 41-42
🟡   log.js 75 50 50 75 13,25
🟡   preconditions.js 66.66 33.33 100 66.66 3
🔴   spreadsheet.js 9.09 0 0 9.09 15-32
🟢   validation-error.js 85.71 100 50 85.71 16
🔴  src/arpa_reporter/routes 40 14.92 14.28 40.6
🔴   agencies.js 22.58 0 0 23.33 13-21,26-53
🟡   application_settings.js 75 100 0 75 10-11
🟡   audit-report.js 68.91 58.33 100 68.91 57-58,64-78,100-116
🟢   exports.js 81.42 83.33 100 81.42 61-75,98-99
🔴   reporting-periods.js 20 0 0 20.43 ...137,143-149,154-180
🔴   subrecipients.js 23.8 0 0 23.8 ...3,17-27,31-48,52-63
🔴   uploads.js 28.28 7.89 9.09 29.16 ...154,164-166,173-180
🔴   users.js 19.6 0 0 20 15-35,39-44,48-81
🔴  src/arpa_reporter/services 44.55 34.79 47.55 44.83
🔴   generate-arpa-report.js 36.86 2.77 50 37.24 ...5,984-997,1071-1138
🔴   get-template.js 21.62 0 0 21.62 18-79
🟡   persist-upload.js 69.66 90 73.07 69.76 ...206,227-241,279-301
🔴   records.js 20.75 0 11.11 21.15 38-204,221-276
🔴   revalidate-uploads.js 37.5 100 0 37.5 5-14
🔴   validate-upload.js 43.56 58.51 37.03 44.2 ...371,393,411-688,703
🟢   validation-rules.js 98.18 90 90.9 100 157,173
🟡  src/db 74.26 71.42 68.42 74.29
🟢   connection.js 100 50 100 100 6
🟢   constants.js 100 100 100 100
🟡   helpers.js 75 83.33 50 75 5,21-22
🟢   index.js 82.4 78.54 82.08 82.35 ...1422,1604-1605,1612
🟢   saved_search_migration.js 92 88.23 71.42 93.61 5,69,134
🔴   tenant_creation.js 10.58 2.7 0 11.11 15-40,48-210,[220](https://github.com/usdigitalresponse/usdr-gost/blob/ff63392289a870192f972eda9e1cd41b45c284e1/pa...*[Comment body truncated]*

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 15, 2024 •
edited
Loading

Terraform Summary

Step Result
🖌 Terraform Format & Style ✅
⚙️ Terraform Initialization ✅
🤖 Terraform Validation ✅
📖 Terraform Plan ✅

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output 
stdout:
Success! The configuration is valid.


-------------------------------------
stderr:
Plan Summary
CHANGE RESOURCE
add module.api.module.grant_activity_digest_scheduled_task.aws_iam_role.default[0]
module.api.module.grant_activity_digest_scheduled_task.aws_iam_role_policy.default[0]
module.api.module.grant_activity_digest_scheduled_task.aws_scheduler_schedule.default[0]
update module.api.aws_ecs_service.default[0]
module.api.module.grant_digest_scheduled_task.aws_iam_role_policy.default[0]
module.api.module.grant_digest_scheduled_task.aws_scheduler_schedule.default[0]
module.arpa_audit_report.aws_ecs_service.default
module.arpa_treasury_report.aws_ecs_service.default
module.consume_grants.aws_ecs_service.default
module.website.aws_s3_object.deploy-config[0]
module.website.aws_s3_object.origin_dist_artifact["assets/style-Lb7S2aAW.js.map"]
recreate module.api.aws_ecs_task_definition.default[0]
module.arpa_audit_report.aws_ecs_task_definition.consumer
module.arpa_treasury_report.aws_ecs_task_definition.consumer
module.consume_grants.aws_ecs_task_definition.consume_grants

Pusher: @greg-adams, Action: pull_request_target, Workflow: Continuous Integration

@greg-adams greg-adams force-pushed the feat/digest-trigger branch 9 times, most recently from b064b84 to 11900d8 Compare October 16, 2024 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TylerHendrickson TylerHendrickson Awaiting requested review from TylerHendrickson

At least 1 approving review is required to merge this pull request.

Assignees

@greg-adams greg-adams

Labels
enhancement New feature or request Infra Issues related to the infrastructure underlying all the tools. javascript Pull requests that update Javascript code terraform Pull requests that update Terraform code
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@greg-adams