Dynamo DB Utility Files (aws_helpers.dynamo_db_utils)

About

This page contains documentation for the files in the /backend/aws_helpers/dynamo_db_utils directory of the Github repo. This page is regularly updated when new changes are added to these files.

Base Dynamo DB (base_db.py)

Base code that defines the structure for the files that handle the manipulation of data of specific Dynamo DB tables. dataclass objects of type BaseData are used to define the schema of the Dynamo DB table (the attributes along with their types).

BaseDDBUtil Class

It provides functions like get_record and update_record to its subclasses that can interact with different Dynamo DB tables. Some functions include:

• create_table: creates the Dynamo DB table if it does not exist, based on instance fields
• create_gsi: creates a global secondary index in the table
• create_record: creates a record in the table by taking in either a BaseData (subclass) instance that holds the record information, or attributes and their values as keyword arguments
• get_record: retrieves a record based on the given id of the partition key of the table
• update_record: updates a record corresponding to the given partition key id, with new values for attributes taken in as keyword arguments (note: this cannot update the partition key id itself)
• delete_record: deletes a record corresponding to the given partition key id

Other Functions of base_db.py

• enumclass: decorator function that produces a class that holds various enums related to the table based on the input BaseData schema. It also takes in optional kwargs, whose keys are attributes that have categorical data, and values are the categorical values in list form.
• changevar: decorator function used to assign static variables to subclasses of BaseDDBUtil that are also utilized in the BaseDDBUtil function implementations. It is sort of analogous to how instance variables of a class are used in instance methods, but the instances of the class can assign different values to the variables.

Example Usage

An implementation of a class that manipulates a phonebook-table, that holds the phone numbers of people, can be shown as follows:
phonebook_db.py

from dataclasses import dataclass
from backend.aws_helpers.dynamo_db_utils.base_db import BaseData, BaseDDBUtil, enumclass, changevar

@dataclass
class PhoneData(BaseData):
    person_id: int
    name: str
    phone_number: str
    relationship: str
    timestamp: str
    
@enumclass(DataClass=PhoneData, relationship=['PARENT', 'SIBLING', 'FRIEND', 'COLLEAGUE', 'ACQUAINTANCE'])
class PhoneEnums:
    pass

@changevar(DataClass=PhoneData, EnumClass=PhoneEnums, partition_key='person_id')
class PhoneDDBUtil(BaseDDBUtil):
    pass

Any BaseDDBUtil function can be performed on instances of PhoneDDBUtil, and the enums in PhoneEnums will be stored as PhoneEnums.Attribute, PhoneEnums.Relationship.

Status Dynamo DB (status_db.py)

Code that creates a Dynamo DB instance to store request_id, status, timestamp for each record. For a given request id, we monitor the status during when the user trains their model (started, in progress, success, fail) and update it (think about the pizza tracker endpoint you see when you order Dominos). This DB serves a similar purpose.

User Dynamo DB (user_db.py)

Code that creates a Dynamo DB instance to store user_id, email, result_uri, onnx_uri, pt_uri, timestamp for each user. Each instance is created when a user creates an account. result_uri, onnx_uri, pt_uri store s3 uris that link to machine learning files created from a user training a dataset.