Django shell, ORM, ...

Open the python shell:

docker-compose run --rm web python manage.py shell_plus

NOTE: We use python3 :)

---

Some Django shell commands:

Retrieve categories:

• Retrieve all the non omitted categories: TaxonomyNode.objects.filter(omitted=False)

• Retrieve all the beginner categories: TaxonomyNode.objects.filter(beginner_task=True)

• Retrieve all the advanced categories: TaxonomyNode.objects.filter(advanced_task=True)

Then you can iterate though the QuerySet as if it was a python list, and get some information.

qs = TaxonomyNode.objects.filter(advanced_task=True)
for taxonomy_node in qs:
    print(taxonomy_node.name)

Write a file:

Files have to be written on a folder that is mounted to an accessible folder (e.g. /mnt/vmdata/asplab-web/asplab-shared/freesound-datasets/fsdatasets_releases:/fsdatasets_releases/)

advanced_categories_dict = {
    node.node_id: 
    {
        'name': node.name, 
        'examples': node.list_freesound_examples,
        'num_ground_truth_annotations': node.num_ground_truth_annotations
    } 
    for node in TaxonomyNode.objects.filter(advanced_task=True)
}

json.dump(advanced_categories_dict, 
          open('/fsdatasets_releases/advanced_categories_dict.json', 'w'))

The json will be available at /mnt/vmdata/asplab-web/asplab-shared/freesound-datasets/fsdatasets_releases/advanced_categories_dict.json

Extract expert votes:

expert_votes_info = list(Vote.objects.filter(from_expert=True).values_list(
    'candidate_annotation__taxonomy_node__node_id',
    'created_by__username',
    'vote',
    'candidate_annotation__sound_dataset__sound__freesound_id',
))

json.dump(expert_votes_info, open('/fsdatasets_releases/expert_votes_info.json', 'w'))

Extract curated annotations (this is the portion of test set reviewed so far with the curation tool)

Only positive ground truth (PP & PNP) - this includes also propagated ground truth annotations:

curated_annotations_info_positive = list(GroundTruthAnnotation.objects\
                                         .filter(from_candidate_annotations__votes__from_expert=True)\
                                         .distinct()\
                                         .values_list('taxonomy_node__node_id', 
                                                      'ground_truth', 
                                                      'sound_dataset__sound__freesound_id'))
json.dump(curated_annotations_info_positive, open('/fsdatasets_releases/curated_annotations_info_positive_Jan19.json', 'w'))

All candidates (includes negative ground truth):

curated_annotations_info_all = list(CandidateAnnotation.objects.filter(votes__from_expert=True)\
                                    .distinct()\
                                    .values_list('taxonomy_node__node_id', 
                                                 'ground_truth', 
                                                 'sound_dataset__sound__freesound_id'))
json.dump(curated_annotations_info_all, open('/fsdatasets_releases/curated_annotations_info_all_Jan20.json', 'w'))

Only positive ground truth (PP & PNP) after a given date - this includes also propagated ground truth annotations:

curated_annotations_posit_date = list(GroundTruthAnnotation.objects.filter(
                                      from_candidate_annotations__votes__created_at__gte=datetime.date(2019, 2, 5), from_candidate_annotations__votes__from_expert=True)\
                                      .distinct()\
                                      .values_list('taxonomy_node__node_id',
                                                   'ground_truth', 
                                                   'sound_dataset__sound__freesound_id'))

json.dump(curated_annotations_posit_date, open('/fsdatasets_releases/curated_annotations_posit_date_Feb05_Feb20.json', 'w'))