Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Truncate Bodies Script #9669

Merged
merged 17 commits into from
Nov 1, 2024
Merged

Truncate Bodies Script #9669

merged 17 commits into from
Nov 1, 2024

Conversation

timolegros
Copy link
Collaborator

@timolegros timolegros commented Oct 24, 2024

Link to Issue

Closes: #8671

Description of Changes

  • Async migration to truncate bodies in the DB
  • Modified routes to truncate bodies before inserting into the DB

Test Plan

  • (Required) get a fresh DB dump: pnpm dump-db
  • Create a thread with more than 2k characters
  • Execute cd packages/commonwealth
    • pnpm ts-exec scripts/truncate-bodies.ts threads
    • pnpm ts-exec scripts/truncate-bodies.ts comments
    • pnpm ts-exec scripts/truncate-bodies.ts thread-versions
    • pnpm ts-exec scripts/truncate-bodies.ts comment-versions
  • View the thread you created before
  • CRUD new threads with more than 2k characters and check version histories

Deployment Plan

Execute the following after deployment:

NODE_ENV=production node --import=extensionless/register --max_old_space_size=$(./scripts/get-max-old-space-size.sh) --enable-source-maps ./build/scripts/truncate-bodies.js threads

NODE_ENV=production node --import=extensionless/register --max_old_space_size=$(./scripts/get-max-old-space-size.sh) --enable-source-maps ./build/scripts/truncate-bodies.js comments

NODE_ENV=production node --import=extensionless/register --max_old_space_size=$(./scripts/get-max-old-space-size.sh) --enable-source-maps ./build/scripts/truncate-bodies.js thread-versions

NODE_ENV=production node --import=extensionless/register --max_old_space_size=$(./scripts/get-max-old-space-size.sh) --enable-source-maps ./build/scripts/truncate-bodies.js comment-versions

Other Considerations

@timolegros timolegros marked this pull request as ready for review October 25, 2024 10:56
@timolegros timolegros added the deployment plan (PRs only) requires manual infrastructure changes on release label Oct 28, 2024
@Rotorsoft
Copy link
Contributor

Not sure if this is related, but getting this error:

[12:19:46.559] ERROR (setupErrorHandlers.ts): content_url must be defined if body length is greater than 2000 {"url":"/api/viewCount","method":"POST","headers":{"host":"localhost:3000","referer":"http://localhost:8080/sushi/discussion/23761-apply-to-become-a-sushi-high-kitchen-member"},"user":{"id":128620},"status":500}
    err: {
      "type": "Error",
      "message": "content_url must be defined if body length is greater than 2000",
      "stack":
          Error: content_url must be defined if body length is greater than 2000

@timolegros
Copy link
Collaborator Author

Not sure if this is related, but getting this error:

[12:19:46.559] ERROR (setupErrorHandlers.ts): content_url must be defined if body length is greater than 2000 {"url":"/api/viewCount","method":"POST","headers":{"host":"localhost:3000","referer":"http://localhost:8080/sushi/discussion/23761-apply-to-become-a-sushi-high-kitchen-member"},"user":{"id":128620},"status":500}
    err: {
      "type": "Error",
      "message": "content_url must be defined if body length is greater than 2000",
      "stack":
          Error: content_url must be defined if body length is greater than 2000

Is this when trying to post a comment?

@Rotorsoft
Copy link
Contributor

  • pnpm ts-exec scripts/truncate-bodies threads

no just refreshing the original sushi thread after the dump

Copy link
Contributor

@Rotorsoft Rotorsoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested and works as expected 🚀

@timolegros timolegros requested a review from Rotorsoft November 1, 2024 17:03
@timolegros timolegros merged commit 86c026b into master Nov 1, 2024
10 checks passed
@timolegros timolegros deleted the tim/truncate-bodies-async branch November 1, 2024 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployment plan (PRs only) requires manual infrastructure changes on release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migration to Blob Storage (part 4/4)
3 participants