automated_MINDSJSON-writer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# automated_MINDSJSON-writer\n",
    "### This script produces a collection of MINDS JSONs, which describe the dataset according to HBP minimal metadata criteria (MINDS v1). This script is initially intended to be added at the end of the \"TVB_connectome_pipeline\" HBP Collab."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To be done by user:\n",
    "1. specify paths in Part I.\n",
    "2. specify schema variants/names in Part III.\n",
    "3. In some cases (e.g. dataset schema), due to assumptions made, the user may want to modify attributes of classes in Part II. e.g.: dataset desciption/license, or species ontologicalTerm.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I. Set-up\n",
    "\n",
    "import packages, set up links, create directory structure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### import packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "from braceexpand import braceexpand\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### link to BIDS-organized dataset\n",
    "\n",
    "specify path to dataset, organized according to the BIDS specification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "BIDSroot = \"\"\n",
    "participants_tsv = pd.read_csv(BIDSroot+\"/participants.tsv\", sep='\\t',dtype={'participant_id': str})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### create \"MINDS JSONs\" directory structure\n",
    "specify path to MINDS JSON collection. Create the directory structure specified by MINDS v1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MINDSroot = \"\"\n",
    "for x in list(braceexpand(MINDSroot+'/core/{activity,agecategory,dataset,person,preparation,sex,species,specimengroup}/v1.0.0')): os.makedirs(x, exist_ok=True)\n",
    "for x in list(braceexpand(MINDSroot+'/ethics/{approval,authority}/v1.0.0')): os.makedirs(x, exist_ok=True)\n",
    "for x in list(braceexpand(MINDSroot+'/experiment/{method,subject}/v1.0.0')): os.makedirs(x, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II. Define function & classes: used to create JSONs from schemas\n",
    "\n",
    "The function *openMINDSschemaWriter* accepts arguments for schema type, etc.\n",
    "The class *baseSchema* defines attributes common to all schemas. It also defines a default function for naming & writing JSON files (files are named: schemaName-0X.json)\n",
    "    \n",
    "All further classes are based on the *baseSchema* class. They define additional attributes (fill the keys of the specific schemas (JSON templates)). Additionally, in some cases, they may implement a modified function to rename JSONs; JSONs will be named according to the schema's *name* field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def openMINDSschemaWriter(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI=\"\"):\n",
    "    if schema_type == \"person\":\n",
    "        return personSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"sex\":\n",
    "        return sexSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"species\":\n",
    "        return speciesSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"authority\":\n",
    "        return authoritySchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"approval\":\n",
    "        return approvalSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"method\":\n",
    "        return methodsSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"preparation\":\n",
    "        return preparationSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"activity\":\n",
    "        return activitySchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"agecategory\":\n",
    "        return agecategorySchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"specimengroup\":\n",
    "        return specimengroupSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"simulator\":\n",
    "        return simulatorSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"subject\":\n",
    "        return subjectSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)\n",
    "    if schema_type == \"dataset\":\n",
    "        return datasetSchema(schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot, relatedIRI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class baseSchema:\n",
    "    def __init__(self, schema_type, MINDSroot, schema_dir, schema_name, schema_json, dataRoot):\n",
    "        self.schema_type = schema_type\n",
    "        self.schema_dir = MINDSroot + schema_dir\n",
    "        self.schema_name = schema_name\n",
    "        self.schema_json = schema_json\n",
    "        self.fileNum = str(len([file for file in os.listdir(self.schema_dir) if os.path.isfile(os.path.join(self.schema_dir, file))]) + 1).zfill(3)\n",
    "    def writeJSON(self):\n",
    "        #generic JSON naming scheme [<schemaName>-0X.json]. override in specific class if \"name\" naming required.\n",
    "        with open(self.schema_dir+\"/\"+self.schema_type+\"-\"+self.fileNum+\".json\",\"w\") as f:\n",
    "            json.dump(self.schema_json, f, indent=4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class personSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        shortName = \"\"\n",
    "        shortName = shortName + self.schema_name.split(\",\")[0] + \", \"\n",
    "        for w in self.schema_name.split(\",\")[1].split():\n",
    "            shortName = shortName + w[0].upper() + \". \"\n",
    "        shortName.rstrip()\n",
    "        self.schema_json[\"shortName\"] = shortName\n",
    "\n",
    "class sexSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "    def writeJSON(self): #overwrite baseSchema \"writeJSON\". #name.\n",
    "        with open(self.schema_dir+\"/\"+self.schema_name+\".json\",\"w\") as f:\n",
    "            json.dump(self.schema_json, f, indent=4)\n",
    "            \n",
    "class speciesSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"ontologicalTerm\"] = [{\"@id\": \"ontologies/core/metazoa/v1.0.0/63b90ba0-66be-4969-8a6a-d19ebea01115\"}]\n",
    "    def writeJSON(self): #overwrite baseSchema \"writeJSON\". #name.\n",
    "        with open(self.schema_dir+\"/homo-sapiens.json\",\"w\") as f:\n",
    "            json.dump(self.schema_json, f, indent=4)\n",
    "               \n",
    "class authoritySchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "\n",
    "class approvalSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"generatedBy\"] = [{\"@id\": \"minds/ethics/authority/v1.0.0/\"+file} for file in os.listdir(MINDSroot + \"/ethics/authority/v1.0.0\")]\n",
    "\n",
    "class methodsSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"relatedIRI\"] = self.relatedIRI\n",
    "\n",
    "class preparationSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        \n",
    "class activitySchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"approval\"] = [{\"@id\": \"minds/ethics/approval/v1.0.0/\"+file} for file in os.listdir(MINDSroot + \"/ethics/approval/v1.0.0\")]\n",
    "        self.schema_json[\"authority\"] = [{\"@id\": \"minds/ethics/authority/v1.0.0/\"+file} for file in os.listdir(MINDSroot + \"/ethics/authority/v1.0.0\")]\n",
    "        if self.schema_name == \"MRI-T1w\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-001.json\"}]\n",
    "        elif self.schema_name == \"MRI-T2W\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-002.json\"}]\n",
    "        elif self.schema_name == \"MRI-T2STAR\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-003.json\"}]\n",
    "        elif self.schema_name == \"MRI-FLAIR\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-004.json\"}]\n",
    "        elif self.schema_name == \"resting state fMRI\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-005.json\"},\n",
    "                                           {\"@id\": \"minds/experiment/method/v1.0.0/method-08.json\"}]\n",
    "        elif self.schema_name == \"DWI\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-006.json\"}]\n",
    "        elif self.schema_name == \"PET\":\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-007.json\"}]\n",
    "            \n",
    "        elif self.schema_name == \"DWI-ImageProcessing\": #(6 & 9-15)\n",
    "            methodNums = [str(6).zfill(3)] \\\n",
    "                        + [str(i).zfill(3) for i in range(9,15+1)]\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-{}.json\".format(i)} for i in methodNums]\n",
    "            \n",
    "        elif self.schema_name == \"T1-imageProcessing\": #(1 & 16-29)\n",
    "            methodNums = [str(1).zfill(3)] \\\n",
    "                        + [str(i).zfill(3) for i in range(16,29+1)]\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-{}.json\".format(i)} for i in methodNums]\n",
    "        \n",
    "        elif self.schema_name == \"rsfMRI-ImageProcessing\": #(5 & 8 & 30-34 & 20 & 35 & 19 & 36-40)\n",
    "            methodNums = [str(i).zfill(3) for i in [5,8]] \\\n",
    "                        + [str(i).zfill(3) for i in range(30,34+1)] \\\n",
    "                        + [str(20).zfill(3)] \\\n",
    "                        + [str(35).zfill(3)] \\\n",
    "                        + [str(19).zfill(3)] \\\n",
    "                        + [str(i).zfill(3) for i in range(36,40+1)]\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-{}.json\".format(i)} for i in methodNums]\n",
    "        \n",
    "        elif self.schema_name == \"PET-ImageProcessing\": #(7 & 41-44 & 34 & 22)\n",
    "            methodNums = [str(7).zfill(3)] \\\n",
    "                        + [str(i).zfill(3) for i in range(41,44+1)] \\\n",
    "                        + [str(34).zfill(3)] \\\n",
    "                        + [str(22).zfill(3)]\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-{}.json\".format(i)} for i in methodNums]\n",
    "        \n",
    "        elif self.schema_name == \"PhenotypicandAssessmentData\": #(45-55)\n",
    "            methodNums = [str(i).zfill(3) for i in range(45,55+1)]\n",
    "            self.schema_json[\"methods\"] = [{\"@id\": \"minds/experiment/method/v1.0.0/method-{}.json\".format(i)} for i in methodNums]\n",
    "                \n",
    "            \n",
    "        self.schema_json[\"preparation\"] = [{\"@id\": \"minds/core/preparation/v1.0.0/\"+file} for file in os.listdir(MINDSroot + \"/core/preparation/v1.0.0\")]\n",
    "\n",
    "class agecategorySchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "\n",
    "class specimengroupSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"subjects\"] = []\n",
    "\n",
    "        \n",
    "class subjectSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        participants_tsv = pd.read_csv(BIDSroot + \"/participants.tsv\", sep='\\t',dtype={'participant_id': str})\n",
    "        self.schema_json[\"age\"] = str(participants_tsv[participants_tsv[\"participant_id\"]==self.schema_name][\"Age\"].item()) + \" years\"\n",
    "        self.schema_json[\"ageCategory\"] = [{\"@id\" : \"minds/core/agecategory/v1.0.0/agecategory-01.json\"}] #if participants_tsv[participants_tsv[\"participant_id\"]==self.schema_name][\"Age\"].item() > 18 else []\n",
    "        self.schema_json[\"sex\"] = [{\"@id\" : \"minds/core/sex/v1.0.0/male.json\"}] if participants_tsv[participants_tsv[\"participant_id\"]==self.schema_name][\"Sex\"].item() == \"M\" else [{\"@id\" : \"minds/core/sex/v1.0.0/female.json\"}]\n",
    "        self.schema_json[\"species\"] = [{\"@id\" : \"minds/core/species/v1.0.0/homo-sapiens.json\"}]\n",
    "    def writeJSON(self): #overwrite baseSchema \"writeJSON\". #name by subject ID.\n",
    "        with open(self.schema_dir+\"/\"+self.schema_name.replace(\" \",\"\")+\".json\",\"w\") as f:\n",
    "            json.dump(self.schema_json, f, indent=4)\n",
    "\n",
    "class datasetSchema(baseSchema):\n",
    "    def fillJSON(self):\n",
    "        self.schema_json[\"@type\"] = \"https://schema.hbp.eu/minds/\"+self.schema_type+\".schema.json\"\n",
    "        self.schema_json[\"@id\"] = \"minds\"+ self.schema_dir.split(MINDSroot,1)[1] + \"/\" + self.schema_type + \"-\" + self.fileNum + \".json\"\n",
    "        self.schema_json[\"name\"] = self.schema_name\n",
    "        self.schema_json[\"description\"] = \"Example MINDS metadata collection, created using data organized according to the BIDS specification.\",\n",
    "        self.schema_json[\"owners\"] = [{\"@id\": \"minds/core/person/v1.0.0/\"+os.listdir(MINDSroot + \"/core/person/v1.0.0\")[0]}],\n",
    "        self.schema_json[\"contributors\"] = [ {\"@id\": \"minds/core/person/v1.0.0/\"+file} for file in os.listdir(MINDSroot + \"/core/person/v1.0.0\")],\n",
    "        self.schema_json[\"embargoStatus\"] = \"free\",\n",
    "        self.schema_json[\"license\"] = [{\"@id\": \"licenses/core/information/v1.0.0/7377a480-6066-4c47-9be8-67c586713ed7\"}],\n",
    "        self.schema_json[\"activities\"] = [{\"@id\": \"minds/core/activity/v1.0.0\"+\"/\"+file} for file in os.listdir(MINDSroot + \"/core/activity/v1.0.0\")]\n",
    "        self.schema_json[\"specimenGroups\"] = [{\"@id\": \"minds/core/specimengroups/v1.0.0\"+\"/\"+file} for file in os.listdir(MINDSroot + \"/core/specimengroup/v1.0.0\")]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## III. Specify schemas and instances. Fill & create JSONs using the functions/classes specified above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### person"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "person_schema = {\n",
    "    \"@type\": \"\",\n",
    "    \"@id\": \"\",\n",
    "    \"name\": \"\",\n",
    "    \"shortName\": \"\",\n",
    "}\n",
    "\n",
    "person_vec = [\"Pai, Roopa Kalsank\", \"Doe, John\", \"Doe, Jane\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for person_name in person_vec:\n",
    "    person = openMINDSschemaWriter(\"person\", MINDSroot, \"/core/person/v1.0.0\", person_name, person_schema, BIDSroot)\n",
    "    person.fillJSON()\n",
    "    person.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### sex"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sex_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\",\n",
    "}\n",
    "\n",
    "sex_vec = [\"female\",\"male\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sex_name in sex_vec:\n",
    "    sex = openMINDSschemaWriter(\"sex\", MINDSroot, \"/core/sex/v1.0.0\", sex_name, sex_schema, BIDSroot)\n",
    "    sex.fillJSON()\n",
    "    sex.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### species"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "species_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\",\n",
    "  \"ontologicalTerm\": []\n",
    "}\n",
    "\n",
    "species_vec = [\"Homo sapiens\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for species_name in species_vec:\n",
    "    species = openMINDSschemaWriter(\"species\", MINDSroot, \"/core/species/v1.0.0\", species_name, species_schema, BIDSroot)\n",
    "    species.fillJSON()\n",
    "    species.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ethics authority"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "authority_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\"\n",
    "}\n",
    "authority_vec = [\"Ethics Board of ABC University\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for authority_name in authority_vec:\n",
    "    authority = openMINDSschemaWriter(\"authority\", MINDSroot, \"/ethics/authority/v1.0.0\", authority_name, authority_schema, BIDSroot)\n",
    "    authority.fillJSON()\n",
    "    authority.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ethics approval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "approval_schema = {\n",
    "    \"@type\": \"\",\n",
    "    \"@id\": \"\",\n",
    "    \"name\": \"\",\n",
    "    \"generatedBy\": []\n",
    "}\n",
    "approval_vec = [\"EA/ID/01\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for approval_name in approval_vec:\n",
    "    approval = openMINDSschemaWriter(\"approval\", MINDSroot, \"/ethics/approval/v1.0.0\", approval_name, approval_schema, BIDSroot)\n",
    "    approval.fillJSON()\n",
    "    approval.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Depends on what scans you've taken of the subject.\n",
    "2. In this test case, the number of methods files will depend on the analysis \"condition\", i.e. the number of criteria. Make one \"methods\" file for: T1, T2, T2STAR, FLAIR, DTI, fMRI, FieldMapping, AV45 PET, AV1451 PET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "method_schema = {\n",
    "    \"@type\": \"\",\n",
    "    \"@id\": \"\",\n",
    "    \"name\": \"\"\n",
    "}\n",
    "\n",
    "method_vec = [\"T1-weighted magnetic resonance imaging (T1w-MRI)\",\n",
    "              \"T2-weighted magnetic resonance imaging (T2w-MRI)\",\n",
    "              \"T2star-weighted magnetic resonance imaging (T2star-MRI)\",\n",
    "              \"FLAIR magnetic resonance imaging (FLAIR-MRI)\",\n",
    "              \"functional magnetic resonance imaging (fMRI)\",\n",
    "              \"diffusion-weighted magnetic resonance imaging (DWI)\",\n",
    "              \"positron-emission tomography (PET)\",\n",
    "              \"resting state\",\n",
    "              #DWI processing (6 & 9-15)\n",
    "              \"noise removal\",\n",
    "              \"Gibbs ringing artifact removal\",\n",
    "              \"eddy current correction\",\n",
    "              \"motion correction\",\n",
    "              \"bias field correction\",\n",
    "              \"global intensity normalization\",\n",
    "              \"tractography\",\n",
    "               #T1 processing (1 & 16-29)\n",
    "              \"ACPC alignment\",\n",
    "              \"brain extraction\",\n",
    "              \"readout distortion correction\",\n",
    "              \"bias field correction\",\n",
    "              \"registration to MNI space\",\n",
    "              \"segmentation\",\n",
    "              \"parcellation\",\n",
    "              \"spline interpolated downsample\",\n",
    "              \"cortical reconstruction\",\n",
    "              \"adjust pial surface\",\n",
    "              \"convert to GIFTI format\",\n",
    "              \"create cortical ribbon\",\n",
    "              \"myelin mapping\",\n",
    "              \"multimodal surface matching\",\n",
    "               #rsfMRI processing (5 & 8 & 30-34 & 20 & 35 & 19 & 36-40)\n",
    "              \"motion correction\",\n",
    "              \"EPI distortion correction\",\n",
    "              \"boundary based registration\",\n",
    "              \"gradient non-linearity distortion correction\",\n",
    "              \"convert to CIFTI format\",\n",
    "              #\"registration to MNI space\" <---point to prev. one\n",
    "              \"intensity normalization\",\n",
    "              #\"bias removal\" <-- point to prev. one\n",
    "              \"ribbon volume to surface mapping\",\n",
    "              \"surface smoothing\",\n",
    "              \"subcortical processing\",\n",
    "              \"generation of dense timeseries\",\n",
    "              \"denoise using spatial ICA\",\n",
    "               #PET processing (7 & 41-44 & 34 & 22)\n",
    "              \"registration to T1w\",\n",
    "              \"partial volume correction\",\n",
    "              \"normalize by cerebellar white matter signal\",\n",
    "              \"volume to surface mapping\",\n",
    "              #\"conversion to CIFTI format\", <---use prev one\n",
    "              #\"parcellation\", <---use prev one\n",
    "              \n",
    "              #Phenotypic and Assessment Data (45-55)\n",
    "              \"Alzheimer's Disease Assessment Scale\",\n",
    "              \"Clinical Dementia Rating Scale\",\n",
    "              \"Everyday Cognition - Patient\",\n",
    "              \"Everyday Cognition – Study Partner\",\n",
    "              \"Functional Activities Questionnaire\",\n",
    "              \"Financial Capacity Instrument\",\n",
    "              \"Geriatric Depression Scale - 15 item\",\n",
    "              \"Mini Mental State Examination\",\n",
    "              \"Montreal Cognitive Assessment\",\n",
    "              \"Neuropsychological Battery\",\n",
    "              \"Neuropsychiatric Inventory\"\n",
    "             ]\n",
    "#vec order important\n",
    "\n",
    "IRI_vec = [\"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              #DWI processing\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "               #T1 processing\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "               #rsfMRI processing\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              #\"registration to MNI space\" <---point to prev. one\n",
    "              \"\",\n",
    "              #\"bias removal\" <-- use prev. one\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "               #PET processing\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              #\"conversion to CIFTI format\", <---use prev one\n",
    "              #\"parcellation\", <---use prev one\n",
    "              \n",
    "              #Phenotypic and Assessment Data\n",
    "              \"http://uri.interlex.org/base/ilx_0346295\",\n",
    "              \"http://uri.interlex.org/base/ilx_0102250\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"http://uri.interlex.org/base/ilx_0106984\",\n",
    "              \"\",\n",
    "              \"\",\n",
    "              \"http://uri.interlex.org/base/ilx_0239290\"\n",
    "             ]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i,method_name in enumerate(method_vec):\n",
    "    method = openMINDSschemaWriter(\"method\", MINDSroot, \"/experiment/method/v1.0.0\", method_name, method_schema, BIDSroot,IRI_vec[i])\n",
    "    method.fillJSON()\n",
    "    method.writeJSON()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preparation_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\"\n",
    "}\n",
    "\n",
    "preparation_vec = [\"in-vivo\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for preparation_name in preparation_vec:\n",
    "    preparation = openMINDSschemaWriter(\"preparation\", MINDSroot, \"/core/preparation/v1.0.0\", preparation_name, preparation_schema, BIDSroot)\n",
    "    preparation.fillJSON()\n",
    "    preparation.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### activity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One *activity* needed for each *method*. Multiple *methods* can combine to create an *activity*. e.g. fMRI (method) + resting state (method) = resting state fMRI (activity)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "activity_schema = {\n",
    "    \"@type\": \"\",\n",
    "    \"@id\": \"\",\n",
    "    \"name\": \"\",\n",
    "    \"approval\": [],\n",
    "    \"authority\": [],\n",
    "    \"methods\": [],\n",
    "    \"preparation\":[]\n",
    "}\n",
    "\n",
    "activity_vec = [\"MRI-T1w\",\n",
    "                \"MRI-T2W\",\n",
    "                \"MRI-T2STAR\",\n",
    "                \"MRI-FLAIR\",\n",
    "                \"resting state fMRI\",\n",
    "                \"DWI\",\n",
    "                \"PET\",\n",
    "                \"DWI-ImageProcessing\",\n",
    "                \"T1-imageProcessing\",\n",
    "                \"rsfMRI-ImageProcessing\"\n",
    "                \"PET-ImageProcessing\",\n",
    "                \"PhenotypicandAssessmentData\"]\n",
    "#vec order important"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for activity_name in activity_vec:\n",
    "    activity = openMINDSschemaWriter(\"activity\", MINDSroot, \"/core/activity/v1.0.0\", activity_name, activity_schema, BIDSroot)\n",
    "    activity.fillJSON()\n",
    "    activity.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### age category"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "agecat_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\"\n",
    "}\n",
    "age_vec = [\"adult\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for agecat_name in age_vec:\n",
    "    agecat = openMINDSschemaWriter(\"agecategory\", MINDSroot, \"/core/agecategory/v1.0.0\", agecat_name, agecat_schema, BIDSroot)\n",
    "    agecat.fillJSON()\n",
    "    agecat.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### specimen group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "specimengroup_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\",\n",
    "  \"subjects\": []\n",
    "}\n",
    "\n",
    "specimengroup_vec = [\"Cognitively Normal (CN)\", \"Mild Cognitive Impairment (MCI)\", \"Alzheimer's Disease (AD)\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for specimengroup_name in specimengroup_vec:\n",
    "    specimengroup = openMINDSschemaWriter(\"specimengroup\", MINDSroot, \"/core/specimengroup/v1.0.0\", specimengroup_name, specimengroup_schema, BIDSroot)\n",
    "    specimengroup.fillJSON()\n",
    "    specimengroup.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### subject"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subject_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\",\n",
    "  \"age\": \"\",\n",
    "  \"ageCategory\": [],\n",
    "  \"sex\": [],\n",
    "  \"species\": []\n",
    "}\n",
    "\n",
    "subject_vec = [sub for sub in participants_tsv[\"participant_id\"].tolist()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for subject_name in subject_vec:\n",
    "    subject = openMINDSschemaWriter(\"subject\", MINDSroot, \"/experiment/subject/v1.0.0\", subject_name, subject_schema, BIDSroot)\n",
    "    subject.fillJSON()\n",
    "    subject.writeJSON()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### loop back and add relevant data to specimen group JSONs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sg_JSON_file in os.listdir(MINDSroot + \"/core/specimengroup/v1.0.0\"):\n",
    "    with open(MINDSroot + \"/core/specimengroup/v1.0.0\"+\"/\"+sg_JSON_file,\"r\") as f:\n",
    "        sg_JSON_data = json.load(f)\n",
    "        sg_name = sg_JSON_data[\"name\"]\n",
    "        if \"CN\" in sg_name:\n",
    "            for sub in participants_tsv[\"participant_id\"].tolist():\n",
    "                if participants_tsv[participants_tsv[\"participant_id\"]==sub][\"Research Group\"].item() == \"CN\":\n",
    "                    sg_JSON_data[\"subjects\"].append({\"@id\": \"minds/experiment/subject/v1.0.0/\"+sub+\".json\"})\n",
    "        elif \"MCI\" in sg_name:\n",
    "            for sub in participants_tsv[\"participant_id\"].tolist():\n",
    "                if \"MCI\" in participants_tsv[participants_tsv[\"participant_id\"]==sub][\"Research Group\"].item():\n",
    "                    sg_JSON_data[\"subjects\"].append({\"@id\": \"minds/experiment/subject/v1.0.0/\"+sub+\".json\"})\n",
    "        elif \"AD\" in sg_name:\n",
    "            for sub in participants_tsv[\"participant_id\"].tolist():\n",
    "                if participants_tsv[participants_tsv[\"participant_id\"]==sub][\"Research Group\"].item() == \"AD\":\n",
    "                    sg_JSON_data[\"subjects\"].append({\"@id\": \"minds/experiment/subject/v1.0.0/\"+sub+\".json\"})\n",
    "    with open(MINDSroot + \"/core/specimengroup/v1.0.0\"+\"/\"+sg_JSON_file,\"w\") as f:\n",
    "        json.dump(sg_JSON_data, f, indent=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### finally, write the dataset JSON"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notes/assumptions:\n",
    "1. owners: default: first \"person\"\n",
    "2. contributors: all persons\n",
    "3. embargoStatus: leave as is for now\n",
    "4. license: leave as is for now\n",
    "5. description: hardcode or editable? for the moment, hardcode inside class definition. if other schemas also take descriptions, add \"description\" as function argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_schema = {\n",
    "  \"@type\": \"\",\n",
    "  \"@id\": \"\",\n",
    "  \"name\": \"\",\n",
    "  \"description\": \"\",\n",
    "  \"owners\": [],\n",
    "  \"contributors\": [],\n",
    "  \"embargoStatus\": \"\",\n",
    "  \"license\": [],\n",
    "  \"activities\": [],\n",
    "  \"specimenGroups\": []\n",
    "}\n",
    "\n",
    "dataset_vec = [\"Test dataset JSON-LD schema.\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for dataset_name in dataset_vec:\n",
    "    dataset = openMINDSschemaWriter(\"dataset\", MINDSroot, \"/core/dataset/v1.0.0\", dataset_name, dataset_schema, BIDSroot)\n",
    "    dataset.fillJSON()\n",
    "    dataset.writeJSON()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}