GitHub - suliman-99/django-seeding: This is a Django package that gives the developer easy implementations for seeding real data in the database

Django Seeding

Introduction

This package helps developers to fill the database with real data instead of filling it manually.

Data can be presented as CSV File , JSON File or in-code.

Dependency-Injection also available to inject your logic by specifying a serializer_class or writing your custom seed method.

Installation

Installing using pip:

pip install django-seeding:

add 'django_seeding' to your INSTALLED_APPS setting:

INSTALLED_APPS = [
    ...
    'django_seeding',
]

Simple Example

Let's take a look at a quick example of using CSVFileModelSeeder seeder from django-seeding to build a simple seeder to insert data in the database.

django_seeding_example/models.py:

from django.db import models


class M1(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

django_seeding_example/seeders.py:

from django_seeding import seeders
from django_seeding.seeder_registry import SeederRegistry 
from django_seeding_example.models import M1

@SeederRegistry.register
class M1Seeder(seeders.CSVFileModelSeeder):
    model = M1
    csv_file_path = 'django_seeding_example/seeders_data/M1Seeder.csv'

django_seeding_example/seeders_data/M1Seeder.csv:

title,description
t1,d1
t2,d2

Now you just need to run this command:

python manage.py seed

Full Usage Documentation:

Now lets go deeper into the different Seeders types with its details:

Attributes List

In general there is a way to know how to deal with these seeders easily:

Model..Seeder needs model class-attribute

Serializer..Seeder needs serializer_class class-attribute

CSVFile..Seeder needs csv_file_path class-attribute

JSONFile..Seeder needs json_file_path class-attribute

All seeders can takes these optional class-attributes:

id: str (So Recommended)

This is what will be stored in the AppliedSeeder table to check if a seeder is already applied or not

It is recommended to set it as the seeder name

So, set it and don't change it because when the value is changed it will be considerd as a new seeder and it will be applied again even that the old seeder with the old name is applied

default value: str(type(seeder))
priority: int|float

Seeders will be sorted depending on this attribute (lower-first)

default value: float('inf')
just_debug: bool

This attribute specify if the seeder will be applied when the server is in the production-mode or not depending in the DEBUG variable in settings file

DEBUG=False & just_debug=True -> don't apply

DEBUG=False & just_debug=False -> apply

DEBUG=True & just_debug=False -> apply

DEBUG=True & just_debug=True -> apply

default value: False

Notice:

@SeederRegistry.register is the decorator that register the seeder, so, if this decorator is not applied then the seeder will not be applied
Model seeders use bulk_create method, so, they are faster than Serializer seeders
Child seeders use bulk_create method with caching fetch for related objects, so, they are faster than Serializer seeders
CSV file reader is using pandas for a better performance and less bugs
Using Model seeders means the field names must match the fields you have defined in your model
Using Serializer seeders means the field names must match the fields you have defined in your serializer
you can define get_ class-methods instead of class-attributes as below:
- get_model
- get_serializer_class
- get_csv_file_path
- get_json_file_path
- get_id
- get_priority
- get_just_debug

Run methods:

To seed with a manual command (Recommended):

python manage.py seed

To seed with runserver manually just add "--seed" in runserver command:

python manage.py runserver --seed

To seed on runserver automatically just set in your project settings:

SEEDING_ON_RUNSERVER = True

Automatically Seeding:

If you set SEEDING_ON_RUNSERVER=True in your settings file You can stop seeding in a runserver by using --dont-seed argument

python manage.py runserver --dont-seed

Managing Debug Mode with the seed Command:

By default, the seed command will use the DEBUG setting from your Django project's settings.py. However, you can override this by explicitly passing the --debug option when running the command.

Force DEBUG to True:

python manage.py seed --debug=True

Force DEBUG to False:

python manage.py seed --debug=False

If no value is specified for --debug, the command will fall back to the project's current DEBUG setting.

Built-in RESTfull APIs for Management

To manage seeders from a dashboard, we provide APIs that offer full control over seeder operations.

Usage:

Route django-seeding views in our seeder app urls.py:

from rest_framework import routers
from django_seeding.apis.views import AppliedSeederViewSet, RegisteredSeederViewSet


router = routers.DefaultRouter()

router.register('registered-seeders', RegisteredSeederViewSet, 'registered-seeders')
router.register('applied-seeders', AppliedSeederViewSet, 'applied-seeders')

urlpatterns = router.urls

Route our seeder app in the project urls.py:

from django.urls import include
...
urlpatterns = [
    ...
    path('seeder/', include('django_seeding_example.urls')),
    ...
]
...

Customization:

You can customize the permissions or any other logic by inheriting the provided viewsets and overriding the necessary methods to meet your specific project requirements.

Postman Collection for API Testing:

To make it easier for you to test the APIs, we have provided a Postman collection containing all the API endpoints. You can download the collection and import it into Postman for quick and easy testing.

Steps to Use the Postman Collection:

Download the Postman collection from this link.
Open Postman, click on "Import", and select the .json file you just downloaded.
You will now have all the API endpoints pre-configured and ready for testing in Postman.

Registered Seeder Endpoint:

In this package, seeder classes must be registered to be applied. This endpoint allows you to:

List all registered seeders: This includes both applied and non-applied seeders.

GET /registered-seeders/
Apply specific seeders: You can apply all seeders or select specific ones.

POST /registered-seeders/seed-all/

we can specify the debug mode for the seeding and the needed ids to be seeded in the body like this (both of them are optional - can be null):
```
{
    "debug": true,
    "ids": [
        "id1",
        "id2"
    ]
}
```

Applied Seeders Endpoint:

Applied seeders are tracked in a dedicated model to prevent them from being applied multiple times. The Applied Seeder endpoints provide full management over the applied seeder records, allowing you to:

List all applied seeders: View all seeders that have been applied.

Get /applied-seeders/
Retrieve a specific applied seeder: View details of a particular seeder.

Get /applied-seeders/{id}/
Mark a seeder as applied: Create a record in the applied seeder model to indicate it has been applied.

POST /applied-seeders/

Request body:
```
{
    "id": "id"
}
```
Update a seeder record: Useful for changing a seeder's ID.

PUT /applied-seeders/{id}/

Request body:
```
{
    "id": "new_id"
}
```
Delete a specific applied seeder: Removes a record to mark the seeder as not applied.

DELETE /applied-seeders/{id}/
Delete all applied seeders: This allows all seeders to be re-applied.

DELETE /applied-seeders/delete-all/

Applied Seeder Table Schema:

id: int (Primary Key)
created_at: datetime
updated_at: datetime

Full Seeders Examples:

Here we will go deeper in the seeders classes and its details

CSVFileModelSeeder (Recommended):

Fast bulk_create seeder

notice that the titles in the csv-file have to match the field names in the model

models.py

class M1(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M1Seeder(seeders.CSVFileModelSeeder):
    id = 'M1Seeder'
    priority = 1
    model = M1
    csv_file_path = 'django_seeding_example/seeders_data/M1Seeder.csv'

seeders_data/M1Seeder.csv

title,description
t1,d1
t2,d2

JSONFileModelSeeder (Recommended):

Fast bulk_create seeder

notice that the keys in the json-file must match the field names in the model

models.py

class M2(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M2Seeder(seeders.JSONFileModelSeeder):
    id = 'M2Seeder'
    priority = 2
    model = M2
    json_file_path = 'django_seeding_example/seeders_data/M2Seeder.json'

seeders_data/M2Seeder.json

[
    {
        "title": "json t1",
        "description": "json d1"
    },
    {
        "title": "json t2",
        "description": "json d2"
    }
]

JSONFileChildSeeder (Recommended):

Blinky-fast bulk-create seeder implemented with caching strategy.

This seeder was conceived to seed child models, i.e. models that at least one field is a foreign key (models.ForeignKey), but can be used instead of JSONFileModelSeeder for general models as well.

Notice that the keys in the json-file must match the field names in the model and also the structure. Parent models are represented as inner dicts.

models.py

class Father(models.Model):
    name = models.TextField()

class Son(models.Model):
    name = models.TextField()
    father = models.ForeignKey(Father, on_delete=models.CASCADE)

seeders.py

@SeederRegistry.register
class SonSeeder(seeders.JSONFileChildSeeder):
    id = 'SonSeeder'
    model = Son
    priority = 10
    json_file_path = 'django_seeding_example/seeders_data/SonSeeder.json'

seeders_data/SonSeeder.json

[
    {
        "name": "json son 1",
        "father": { "name": "json father 1" }
    },
    {
        "name": "json son 2",
        "father": { "name": "json father 2" }
    }
]

Notice that child priority must be greater than parent priority in order to the parent model be seeded before. Not seeding parent before will raise errors! Each field that is a FK must be a dictionary with field names same as its related model.

This seeder class can handle pretty complex relations between models. Let's expand the family (pun intended):

models.py

class Mother(models.Model):
    name = models.TextField()

class Daughter(models.Model):
    name = models.TextField()
    father = models.ForeignKey(Father, on_delete=models.CASCADE)
    mother = models.ForeignKey(Mother, on_delete=models.CASCADE)

    class Meta:
        constraints = [
            UniqueConstraint (
                fields=['name', 'father', 'mother'],
                name='unique_parentage'
            )]

class Grandson(models.Model):
    name = models.TextField()
    parentage = models.ForeignKey(Daughter, on_delete=models.CASCADE)

seeders.py

@SeederRegistry.register
class DaughterSeeder(seeders.JSONFileChildSeeder):
    id = 'DaughterSeeder'
    priority = 10
    model = Daughter
    json_file_path = 'django_seeding_example/seeders_data/DaughterSeeder.json'


@SeederRegistry.register
class GrandsonSeeder(seeders.JSONFileChildSeeder):
    id = 'GrandsonSeeder'
    model = Grandson
    json_file_path = 'django_seeding_example/seeders_data/GrandsonSeeder.json'

seeders_data/DaughterSeeder.json

[
    {
        "name": "json daughter 1",
        "father": { "name": "json father 1" },
        "mother": { "name": "json mother 1" }
    },
    {
        "name": "json daughter 2",
        "father": { "name": "json father 2" },
        "mother": { "name": "json mother 2" }
    }
]

seeders_data/GrandsonSeeder.json

[
    {
        "name": "json grandson 1",
        "parentage": {
            "name": "json daughter 1",
            "father": { "name": "json father 1" },
            "mother": { "name": "json mother 1" }
        }
    },
    {
        "name": "json grandson 2",
        "parentage": {
            "name": "json daughter 2",
            "father": { "name": "json father 2" },
            "mother": { "name": "json mother 2" }
        }
    }
]

CSVFileSerializerSeeder:

Slow one-by-one seeder

notice that the titles in the csv-file have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M3(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializers.py

class M3Serializer(serializers.ModelSerializer):
    class Meta:
        model = M3
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M3Seeder(seeders.CSVFileSerializerSeeder):
    id = 'M3Seeder'
    priority = 3
    serializer_class = M3Serializer
    csv_file_path = 'django_seeding_example/seeders_data/M3Seeder.csv'

seeders_data/M3Seeder.csv

title,description
t1,d1
t2,d2

JSONFileSerializerSeeder:

Slow one-by-one seeder

notice that the keys in the json-file have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M4(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializers.py

class M4Serializer(serializers.ModelSerializer):
    class Meta:
        model = M4
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M4Seeder(seeders.JSONFileSerializerSeeder):
    id = 'M4Seeder'
    priority = 4
    serializer_class = M4Serializer
    json_file_path = 'django_seeding_example/seeders_data/M4Seeder.json'

seeders_data/M4Seeder.json

[
    {
        "title": "json t1",
        "description": "json d1"
    },
    {
        "title": "json t2",
        "description": "json d2"
    }
]

EmptySeeder (Recommended):

Fast bulk_create seeder

models.py

class M5(models.Model):
    title = models.CharField(max_length=100, null=True)
    description = models.TextField(null=True)

seeders.py

@SeederRegistry.register
class M5Seeder(seeders.EmptySeeder):
    id = 'M5Seeder'
    priority = 5
    model = M5
    records_count = 2

ModelSeeder (Recommended):

Fast bulk_create seeder

notice that the keys in the data class-attribute have to match the field names in the model

models.py

class M6(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M6Seeder(seeders.ModelSeeder):
    id = 'M6Seeder'
    priority = 6
    model = M6
    data = [
        {
            "title": "in-code t1",
            "description": "in-code d1"
        },
        {
            "title": "in-code t2",
            "description": "in-code d2"
        },
    ]

SerializerSeeder:

Slow one-by-one seeder

notice that the keys in the data class-attribute have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M7(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializer.py

class M7Serializer(serializers.ModelSerializer):
    class Meta:
        model = M7
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M7Seeder(seeders.SerializerSeeder):
    id = 'M7Seeder'
    priority = 7
    serializer_class = M7Serializer
    data = [
        {
            "title": "in-code t1",
            "description": "in-code d1"
        },
        {
            "title": "in-code t2",
            "description": "in-code d2"
        },
    ]

Seeder:

Here you can write your logic as you want in the seed method

models.py

class Post(models.Model):
    content = models.TextField()


class Comment(models.Model):
    post = models.ForeignKey(Post, on_delete=models.CASCADE)
    content = models.TextField()

seeders.py

@SeederRegistry.register
class CustomSeeder(seeders.Seeder):
    id = 'CustomSeeder'
    priority = 8
    
    def seed(self):
        post1 = Post.objects.create(content='post1')
        post2 = Post.objects.create(content='post1')

        comment1 = Comment.objects.create(post=post1, content='comment1')
        comment2 = Comment.objects.create(post=post1, content='comment2')
        comment3 = Comment.objects.create(post=post2, content='comment3')
        comment4 = Comment.objects.create(post=post2, content='comment4')

Contributing

If you have suggestions for how Django Seeding could be improved, or want to report a bug, open an issue! We'd love all and any contributions.

For more, check out the Contributing Guide.

Contact

Suliman Awad - [email protected] - Linkedin

Project Link: https://github.com/suliman-99/django-seeding

License

MIT License

For more, check out the License File.

Name		Name	Last commit message	Last commit date
Latest commit History 75 Commits
.devcontainer		.devcontainer
.github/workflows		.github/workflows
django_project		django_project
django_seeding		django_seeding
django_seeding_example		django_seeding_example
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
Pipfile		Pipfile
Pipfile.lock		Pipfile.lock
README.md		README.md
index.js		index.js
manage.py		manage.py
pyproject.toml		pyproject.toml
setup.py		setup.py

License

suliman-99/django-seeding

Folders and files

Latest commit

History

Repository files navigation

Django Seeding

Table of Contents

Introduction

Installation

Simple Example

Full Usage Documentation:

Seeders List:

Attributes List

In general there is a way to know how to deal with these seeders easily:

All seeders can takes these optional class-attributes:

Notice:

Run methods:

Automatically Seeding:

Managing Debug Mode with the seed Command:

Built-in RESTfull APIs for Management

Usage:

Customization:

Postman Collection for API Testing:

Registered Seeder Endpoint:

Applied Seeders Endpoint:

Applied Seeder Table Schema:

Full Seeders Examples:

CSVFileModelSeeder (Recommended):

JSONFileModelSeeder (Recommended):

JSONFileChildSeeder (Recommended):

CSVFileSerializerSeeder:

JSONFileSerializerSeeder:

EmptySeeder (Recommended):

ModelSeeder (Recommended):

SerializerSeeder:

Seeder:

Contributing

Contact

License

About

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases 1

Packages 0

Contributors 3

Languages

Packages