Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate database from SQLite to PostgreSQL #148

Open
4 tasks done
johndpjr opened this issue Oct 3, 2023 · 1 comment
Open
4 tasks done

Migrate database from SQLite to PostgreSQL #148

johndpjr opened this issue Oct 3, 2023 · 1 comment
Assignees
Labels
backend Deals with the FastAPI and web-scraping backend enhancement Adds value to a previous feature waiting Waiting for something to happen

Comments

@johndpjr
Copy link
Owner

johndpjr commented Oct 3, 2023

Context

SQLite is good for a single-file database solution; it won't scale well for our needs since we'll be having multiple webscrapers working at once and we'll need this database to be hosted on some URL somewhere. PostgreSQL is another SQL-based database solution that handles concurrency and scales better than SQLite. We were using SQLAlchemy as a database abstraction layer which allows us to write SQLAlchemy code without having to worry about the underlying implementation (e.g. SQLite, PostgreSQL, etc.).

TODO

  • Check out backend/app/database/database.py for the migration entry point and some additional information
  • Install the database driver for PostgreSQL by adding psycopg2==2.9.8 to the requirements.txt file and running pip3 install -r requirements.txt
  • Follow the SQLAlchemy connection tutorial
  • Comment below on actionable steps that developers need to complete to setup their local PostgreSQL instance

Notes

You don't need to set up a remote PostgreSQL instance. A local one will work for now.

@johndpjr johndpjr added enhancement Adds value to a previous feature needs more detail An incomplete ticket backend Deals with the FastAPI and web-scraping backend labels Oct 3, 2023
@johndpjr johndpjr added this to the Sprint 1 milestone Oct 3, 2023
@johndpjr johndpjr added this to AgTern Oct 3, 2023
@github-project-automation github-project-automation bot moved this to Backlog in AgTern Oct 3, 2023
@johndpjr johndpjr moved this from Backlog to Todo in AgTern Oct 3, 2023
@johndpjr johndpjr removed the needs more detail An incomplete ticket label Oct 3, 2023
@DavidHigpen DavidHigpen moved this from Todo to In Progress in AgTern Oct 14, 2023
@DavidHigpen DavidHigpen self-assigned this Oct 14, 2023
@DavidHigpen DavidHigpen moved this from In Progress to Done in AgTern Nov 25, 2023
@johndpjr johndpjr moved this from Done to In Progress in AgTern Nov 28, 2023
@DavidHigpen DavidHigpen moved this from In Progress to Done in AgTern Mar 6, 2024
@johndpjr
Copy link
Owner Author

Waiting for the following things to do before merging:

  • Better-organized/refactored repo
  • Having PostgreSQL documentation that details steps needed for migration
  • Functioning DO instance
  • Configuration b/t SQLite and PostgreSQL database types

@johndpjr johndpjr modified the milestones: Sprint 1, 2024 Spring Semester Mar 13, 2024
@johndpjr johndpjr moved this from Done to Backlog in AgTern Mar 13, 2024
@johndpjr johndpjr added the waiting Waiting for something to happen label Mar 14, 2024
@johndpjr johndpjr moved this from Backlog to Todo in AgTern Mar 24, 2024
@arlinbirkby777 arlinbirkby777 removed the status in AgTern Apr 1, 2024
@johndpjr johndpjr modified the milestones: 2024 Spring Semester, 2024 Fall Semester Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Deals with the FastAPI and web-scraping backend enhancement Adds value to a previous feature waiting Waiting for something to happen
Projects
Status: No status
Development

No branches or pull requests

2 participants