GitHub - liviomendonca/django-seeding: This is a Django package that gives the developer easy implementations for seeding real data in the database

Django Seeding

Introduction

This package helps developers to fill the database with real data instead of filling it manually.

Data can be presented as CSV File , JSON File or in-code.

Dependency-Injection also available to inject your logic by specifying a serializer_class or writing your custom seed method.

Installation

Installing using pip:

pip install django-seeding:

add 'django_seeding' to your INSTALLED_APPS setting:

INSTALLED_APPS = [
    ...
    'django_seeding',
]

Simple Example

Let's take a look at a quick example of using CSVFileModelSeeder seeder from django-seeding to build a simple seeder to insert data in the database.

django_seeding_example/models.py:

from django.db import models


class M1(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

django_seeding_example/seeders.py:

from django_seeding import seeders
from django_seeding.seeder_registry import SeederRegistry 
from django_seeding_example.models import M1

@SeederRegistry.register
class M1Seeder(seeders.CSVFileModelSeeder):
    model = M1
    csv_file_path = 'django_seeding_example/seeders_data/M1Seeder.csv'

django_seeding_example/seeders_data/M1Seeder.csv:

title,description
t1,d1
t2,d2

Now you just need to run this command:

python manage.py seed

Full Usage Documentation:

Now lets go deeper into the different Seeders types with its details:

Attributes List

In general there is a way to know how to deal with these seeders easily:

Model..Seeder needs model class-attribute

Serializer..Seeder needs serializer_class class-attribute

CSVFile..Seeder needs csv_file_path class-attribute

JSONFile..Seeder needs json_file_path class-attribute

All seeders can takes these optional class-attributes:

id: str (So Recommended)

This is what will be stored in the AppliedSeeder table to check if a seeder is already applied or not

It is recommended to set it as the seeder name

So, set it and don't change it because when the value is changed it will be considerd as a new seeder and it will be applied again even that the old seeder with the old name is applied

default value: str(type(seeder))
priority: int|float

Seeders will be sorted depending on this attribute (lower-first)

default value: float('inf')
just_debug: bool

This attribute specify if the seeder will be applied when the server is in the production-mode or not depending in the DEBUG variable in settings file

DEBUG=False & just_debug=True -> don't apply

DEBUG=False & just_debug=False -> apply

DEBUG=True & just_debug=False -> apply

DEBUG=True & just_debug=True -> apply

default value: False

Notice:

@SeederRegistry.register is the decorator that register the seeder, so, if this decorator is not applied then the seeder will not be applied
Model seeders use bulk_create method, so, they are faster than Serializer seeders
CSV file reader is using pandas for a better performance and less bugs
Using Model seeders means the field names must match the fields you have defined in your model
Using Serializer seeders means the field names must match the fields you have defined in your serializer

you can define get_ class-methods instead of class-attributes as below:

get_model
get_serializer_class
get_csv_file_path
get_json_file_path
get_id
get_priority
get_just_debug

Run methods:

To seed with a manual command (Recommended):

python manage.py seed

To seed with runserver manually just add "--seed" in runserver command:

python manage.py runserver --seed

To seed on runserver automatically just set in your project settings:

SEEDING_ON_RUNSERVER = True

Notice:

If you set SEEDING_ON_RUNSERVER=True in your settings file You can stop seeding in a runserver by using --dont-seed argument

python manage.py runserver --dont-seed

Full Examples:

Here we will go deeper in the seeders classes and its details

CSVFileModelSeeder (Recommended):

Fast bulk_create seeder

notice that the titles in the csv-file have to match the field names in the model

models.py

class M1(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M1Seeder(seeders.CSVFileModelSeeder):
    id = 'M1Seeder'
    priority = 1
    model = M1
    csv_file_path = 'django_seeding_example/seeders_data/M1Seeder.csv'

seeders_data/M1Seeder.csv

title,description
t1,d1
t2,d2

JSONFileModelSeeder (Recommended):

Fast bulk_create seeder

notice that the keys in the json-file must match the field names in the model

models.py

class M2(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M2Seeder(seeders.JSONFileModelSeeder):
    id = 'M2Seeder'
    priority = 2
    model = M2
    json_file_path = 'django_seeding_example/seeders_data/M2Seeder.json'

seeders_data/M2Seeder.json

[
    {
        "title": "json t1",
        "description": "json d1"
    },
    {
        "title": "json t2",
        "description": "json d2"
    }
]

JSONFileChildSeeder

Blinky-fast bulk-create seeder implemented with caching strategy.

This seeder was concieved to seed child models, i.e. models that at least one field is a foreign key (models.ForeignKey), but can be used instead of JSONFileModelSeeder for general models as well.

Notice that the keys in the json-file must match the field names in the model and also the structure. Parent models are represented as inner dicts.

models.py

class Father(models.Model):
    name = models.TextField()

class Son(models.Model):
    name = models.TextField()
    father = models.ForeignKey(Father, on_delete=models.CASCADE)

seeders.py

@SeederRegistry.register
class SonSeeder(seeders.JSONFileChildSeeder):
    id = 'SonSeeder'
    model = Son
    priority = 10
    json_file_path = 'django_seeding_example/seeders_data/SonSeeder.json'

seeders_data/SonSeeder.json

[
    {
        "name": "json son 1",
        "father": { "name": "json father 1" }
    },
    {
        "name": "json son 2",
        "father": { "name": "json father 2" }
    }
]

Notice that child priority must be greater than parent priority in order to the parent model be seeded before. Not seeding parent before will raise errors! Each field that is a FK must be a dictionary with field names same as its related model.

This seeder class can handle pretty complex relations between models. Let's expand the family (pun intended):

models.py

class Mother(models.Model):
    name = models.TextField()

class Daughter(models.Model):
    name = models.TextField()
    father = models.ForeignKey(Father, on_delete=models.CASCADE)
    mother = models.ForeignKey(Mother, on_delete=models.CASCADE)

    class Meta:
        constraints = [
            UniqueConstraint (
                fields=['name', 'father', 'mother'],
                name='unique_parentage'
            )]

class Grandson(models.Model):
    name = models.TextField()
    parentage = models.ForeignKey(Daughter, on_delete=models.CASCADE)

seeders.py

@SeederRegistry.register
class DaughterSeeder(seeders.JSONFileChildSeeder):
    id = 'DaughterSeeder'
    priority = 10
    model = Daughter
    json_file_path = 'django_seeding_example/seeders_data/DaughterSeeder.json'


@SeederRegistry.register
class GrandsonSeeder(seeders.JSONFileChildSeeder):
    id = 'GrandsonSeeder'
    model = Grandson
    json_file_path = 'django_seeding_example/seeders_data/GrandsonSeeder.json'

seeders_data/DaughterSeeder.json

[
    {
        "name": "json daughter 1",
        "father": { "name": "json father 1" },
        "mother": { "name": "json mother 1" }
    },
    {
        "name": "json daughter 2",
        "father": { "name": "json father 2" },
        "mother": { "name": "json mother 2" }
    }
]

seeders_data/GrandsonSeeder.json

[
    {
        "name": "json grandson 1",
        "parentage": {
            "name": "json daughter 1",
            "father": { "name": "json father 1" },
            "mother": { "name": "json mother 1" }
        }
    },
    {
        "name": "json grandson 2",
        "parentage": {
            "name": "json daughter 2",
            "father": { "name": "json father 2" },
            "mother": { "name": "json mother 2" }
        }
    }
]

CSVFileSerializerSeeder:

Slow one-by-one seeder

notice that the titles in the csv-file have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M3(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializers.py

class M3Serializer(serializers.ModelSerializer):
    class Meta:
        model = M3
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M3Seeder(seeders.CSVFileSerializerSeeder):
    id = 'M3Seeder'
    priority = 3
    serializer_class = M3Serializer
    csv_file_path = 'django_seeding_example/seeders_data/M3Seeder.csv'

seeders_data/M3Seeder.csv

title,description
t1,d1
t2,d2

JSONFileSerializerSeeder:

Slow one-by-one seeder

notice that the keys in the json-file have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M4(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializers.py

class M4Serializer(serializers.ModelSerializer):
    class Meta:
        model = M4
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M4Seeder(seeders.JSONFileSerializerSeeder):
    id = 'M4Seeder'
    priority = 4
    serializer_class = M4Serializer
    json_file_path = 'django_seeding_example/seeders_data/M4Seeder.json'

seeders_data/M4Seeder.json

[
    {
        "title": "json t1",
        "description": "json d1"
    },
    {
        "title": "json t2",
        "description": "json d2"
    }
]

EmptySeeder (Recommended):

Fast bulk_create seeder

models.py

class M5(models.Model):
    title = models.CharField(max_length=100, null=True)
    description = models.TextField(null=True)

seeders.py

@SeederRegistry.register
class M5Seeder(seeders.EmptySeeder):
    id = 'M5Seeder'
    priority = 5
    model = M5
    records_count = 2

ModelSeeder (Recommended):

Fast bulk_create seeder

notice that the keys in the data class-attribute have to match the field names in the model

models.py

class M6(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

seeders.py

@SeederRegistry.register
class M6Seeder(seeders.ModelSeeder):
    id = 'M6Seeder'
    priority = 6
    model = M6
    data = [
        {
            "title": "in-code t1",
            "description": "in-code d1"
        },
        {
            "title": "in-code t2",
            "description": "in-code d2"
        },
    ]

SerializerSeeder:

Slow one-by-one seeder

notice that the keys in the data class-attribute have to match the field names in the serializer

This seeder is used to inject a serializer to implement custom create logic

models.py

class M7(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()

serializer.py

class M7Serializer(serializers.ModelSerializer):
    class Meta:
        model = M7
        fields = ['title', 'description']

    def create(self, validated_data):
        validated_data['title'] = '__' + validated_data['title'] + '__'
        validated_data['description'] = '__' + validated_data['description'] + '__'
        return super().create(validated_data)

seeders.py

@SeederRegistry.register
class M7Seeder(seeders.SerializerSeeder):
    id = 'M7Seeder'
    priority = 7
    serializer_class = M7Serializer
    data = [
        {
            "title": "in-code t1",
            "description": "in-code d1"
        },
        {
            "title": "in-code t2",
            "description": "in-code d2"
        },
    ]

Seeder:

Here you can write your logic as you want in the seed method

models.py

class Post(models.Model):
    content = models.TextField()


class Comment(models.Model):
    post = models.ForeignKey(Post, on_delete=models.CASCADE)
    content = models.TextField()

seeders.py

@SeederRegistry.register
class CustomSeeder(seeders.Seeder):
    id = 'CustomSeeder'
    priority = 8
    
    def seed(self):
        post1 = Post.objects.create(content='post1')
        post2 = Post.objects.create(content='post1')

        comment1 = Comment.objects.create(post=post1, content='comment1')
        comment2 = Comment.objects.create(post=post1, content='comment2')
        comment3 = Comment.objects.create(post=post2, content='comment3')
        comment4 = Comment.objects.create(post=post2, content='comment4')

Contributing

If you have suggestions for how Django Seeding could be improved, or want to report a bug, open an issue! We'd love all and any contributions.

For more, check out the Contributing Guide.

Contact

Suliman Awad - [email protected] - Linkedin

Project Link: https://github.com/suliman-99/django-seeding

License

MIT License

For more, check out the License File.

Name		Name	Last commit message	Last commit date
Latest commit History 60 Commits
.devcontainer		.devcontainer
.github/workflows		.github/workflows
django_project		django_project
django_seeding		django_seeding
django_seeding_example		django_seeding_example
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
Pipfile		Pipfile
Pipfile.lock		Pipfile.lock
README.md		README.md
index.js		index.js
manage.py		manage.py
pyproject.toml		pyproject.toml
setup.py		setup.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Django Seeding

Table of Contents

Introduction

Installation

Simple Example

Full Usage Documentation:

Seeders List:

Attributes List

In general there is a way to know how to deal with these seeders easily:

All seeders can takes these optional class-attributes:

Notice:

Run methods:

Notice:

Full Examples:

CSVFileModelSeeder (Recommended):

JSONFileModelSeeder (Recommended):

JSONFileChildSeeder

CSVFileSerializerSeeder:

JSONFileSerializerSeeder:

EmptySeeder (Recommended):

ModelSeeder (Recommended):

SerializerSeeder:

Seeder:

Contributing

Contact

License

About

Releases

Packages

Languages

License

liviomendonca/django-seeding

Folders and files

Latest commit

History

Repository files navigation

Django Seeding

Table of Contents

Introduction

Installation

Simple Example

Full Usage Documentation:

Seeders List:

Attributes List

In general there is a way to know how to deal with these seeders easily:

All seeders can takes these optional class-attributes:

Notice:

Run methods:

Notice:

Full Examples:

CSVFileModelSeeder (Recommended):

JSONFileModelSeeder (Recommended):

JSONFileChildSeeder

CSVFileSerializerSeeder:

JSONFileSerializerSeeder:

EmptySeeder (Recommended):

ModelSeeder (Recommended):

SerializerSeeder:

Seeder:

Contributing

Contact

License

About

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages