Conversion_to_CV-Tank.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Converting a Normal EPANET .inp File to a CV-Tank IWS File [1,2]\n",
    "This notebook takes an input EPANET files with demands input normally as a CWS base demand and adds the necessary elements to convert it into a CV-Tank simulation  \n",
    "For details on the CV-Tank method refer to [1,2]  \n",
    "A simplified schematic of the modified demand node in CV-Tank is seen below:  \n",
    "  \n",
    "  \n",
    "![](CV-Tank.png)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First, we import the necessary libraries and packages\n",
    "**WNTR** for building EPANET network models in Python  \n",
    "**NUMPY & PANDAS** for data handling and processing  \n",
    "**re** for searching and matching text in the .inp file using regular expressions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import wntr  \n",
    "import numpy as np \n",
    "import pandas as pd\n",
    "import re\n",
    "import pathlib"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specifying paths for simulation files and preprocessing the input\n",
    "**Warning:** *Paths in this script (and the rest of this repo) are absolute unless running the network files provided within the repo*  \n",
    "Input filename (with extensions) as string.  \n",
    "For running the .inp files in this repository, you can use this relative path `\"../Network-Files/Network X/\"` where X is the network number "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected File:  CampisanoNet2\n"
     ]
    }
   ],
   "source": [
    " # Replace with appropriate path and filename\n",
    "directory=pathlib.Path(\"\")\n",
    "filename=pathlib.Path(\"CampisanoNet2_4hr.inp\")\n",
    "name_only=str(filename.stem)[0:-4]\n",
    "print(\"Selected File: \",name_only)\n",
    "path=directory/filename"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Necessary Assumptions Input\n",
    "Converting a CWS demand-driven analysis into an IWS pressure-driven analysis requires some assumptions in all methods  \n",
    "The resistance of the service connection between the demand node and the household (end-user) is uncertain and is modelled using two assumptions  \n",
    "The **desired head (pressure)** is the pressure at the demand node at which (or above) the consumer can satisfy their full demand in the supply duration (or possible less)  \n",
    "The **minimum head (pressure)** is the minimum pressure at the demand node required for flow to begin passing through the service connection  \n",
    "These two assumptions dictate the flow-pressure relationship that determines the pressure-dependent flow through the service connection as follows:\n",
    "\n",
    "$$ Q\\, = \\!Q_{des}\\sqrt{\\frac{H_{j}-H^{min}}{H^{des}-H^{min}}} \\quad[1]$$ \n",
    "Where Q is the flow through the service connection, $Q_{des}$ is the desired (base) demand, $H_j$ is the head at the demand node $j$, $H^{min}$ is the minimum head, and $H^{des}$ is the desired head\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "desired_pressure=2     # Set the desired pressure\n",
    "minimum_pressure=-1      # Set the minimum pressure\n",
    "pressure_diff=desired_pressure-minimum_pressure  "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extracting information from the input file\n",
    "To modify the .inp file, demand node IDs, elevations, x and y coordinates  \n",
    "We use wntr to build the network model of the input file and use wntr's junctions module to extract the details of each node"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "demand_nodes=[]       # For storing list of nodes that have non-zero demands\n",
    "desired_demands=[]    # For storing demand rates desired by each node for desired volume calculations\n",
    "elevations=[]         # For storing elevations of demand nodes\n",
    "xcoordinates=[]       # For storing x coordinates of demand nodes\n",
    "ycoordinates=[]       # For storing y coordinates of demand nodes\n",
    "all_nodes=[]          # For storing list of node ids of all nodes\n",
    "all_elevations=[]     # For storing elevations of all nodes\n",
    "## MAYBE SAVE ALL NODE IDS IN DATAFRAME WITH ELEVATION AND BASE DEMAND AND THEN FILTER DATA FRAME LATER FOR DEMAND NODES ONLY\n",
    "\n",
    "# Creates a network model object using EPANET .inp file\n",
    "network=wntr.network.WaterNetworkModel(path)\n",
    "\n",
    "# Iterates over the junction list in the Network object\n",
    "for node in network.junctions():\n",
    "    all_nodes.append(node[1].name)\n",
    "    all_elevations.append(node[1].elevation)\n",
    "    # For all nodes that have non-zero demands\n",
    "    if node[1].base_demand != 0:\n",
    "        # Record node ID (name), desired demand (base_demand) in CMS, elevations, x and y coordinates\n",
    "        demand_nodes.append(node[1].name)\n",
    "        desired_demands.append(node[1].base_demand)\n",
    "        elevations.append(node[1].elevation)\n",
    "        xcoordinates.append(node[1].coordinates[0])\n",
    "        ycoordinates.append(node[1].coordinates[1])\n",
    "\n",
    "\n",
    "\n",
    "# Get the supply duration in minutes (/60) as an integer\n",
    "supply_duration=int(network.options.time.duration/60)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Tank Section\n",
    "Each Simple Tank represents the demand volume of the respective demand node throughout the supply cycle  \n",
    "The volume of each Simple Tank is calculated is:  \n",
    "$$ V_{des}\\,=\\,Q_{des}\\,(m^3/s)\\,\\times\\,60\\,(s/min)\\,\\times\\,Supply\\,Duration\\,(min)$$  \n",
    "Using the desired volume, and unifying tank height at 1 m (for ease of postprocessin), the diameter of the tank can be calculated as:  \n",
    "$$D\\,(m)\\,=\\sqrt{\\frac{4V_{des}}{\\pi}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adds the phrase TankforNode to each node id and stores it as a tank id\n",
    "tankids=['TankforNode'+str(id) for id in demand_nodes] \n",
    "# elevations\n",
    "tank_elevs = [elevation + minimum_pressure for elevation in elevations]\n",
    "# Calculate desired demand volumes and then calculates the diameters of the simple tanks\n",
    "volumes=[demand* 60 * supply_duration for demand in desired_demands]\n",
    "diameters_tanks=[round(np.sqrt(volume * 4 / np.pi),4) for volume in volumes]\n",
    "# List of zeros for each tank to be used as the values for Initial Level, Minimum Level, and Minimum Volume \n",
    "zeros=[0.0000] *len(tankids)\n",
    "# Sets Maximum levels for all tanks as 1\n",
    "MaxLevel=[1.0000]*len(tankids)\n",
    "# No Volume curve is assigned to any of the tanks\n",
    "VolCurve=['    ']*len(tankids)\n",
    "# Semicolons to end each tank line\n",
    "semicolons=[';']*len(tankids)\n",
    "# Assemble all lists into a dataframe where each row is the definition for one simple tank\n",
    "tanks_section=pd.DataFrame(list(zip(tankids,tank_elevs,zeros,zeros,MaxLevel,diameters_tanks,zeros,VolCurve,semicolons)))\n",
    "# Exports the tank section as a list of strings where each entry is a line of the tanks section\n",
    "tanks_section=tanks_section.to_string(header=False,index=False,col_space=10).splitlines()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Added Pipes\n",
    "Similar to the tanks section, each field for the pipe entries is stored in a separate list  \n",
    "The added pipes connect the created tanks to the original demand nodes and are responsible for simulating the pressure-dependent flow relationship  \n",
    "The length of the pipe is calculated according to the required head flow relationship dictated by the minimum and desired pressure as:  \n",
    "$$L=(H^{des}-H^{min})\\frac{C^{1.852}D^{4.87}}{10.67Q_{des}^{1.852}}$$  \n",
    "The diameter is unified at 0.05 m (50 mm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adds the phrase PipeforNode to each node id and stores it as a pipe id\n",
    "pipeids=['PipeforNode'+str(id) for id in demand_nodes]\n",
    "# Calculates the length of each pipe to simulate the head-flow relationship\n",
    "lengths=[round(pressure_diff*130**1.852*0.05**4.87/10.67/(demand)**1.852 , 4) for demand in desired_demands]\n",
    "# Sets all diameters to 1 m (1000 mm)\n",
    "diameters_pipes=[50]*len(pipeids)\n",
    "# Sets all Hazen-Williams Coefficients as 130\n",
    "hazen=[130]*len(pipeids)\n",
    "# Sets all created pipes to work as Check Valved to prevent backflow\n",
    "status=['CV']*len(pipeids)\n",
    "# Assemble all lists into a dataframe where each row is the definition for one simple tank\n",
    "pipes_addendum=pd.DataFrame(list(zip(pipeids,demand_nodes,tankids,lengths,diameters_pipes,hazen,zeros,status,semicolons)))\n",
    "# Exports the pipe section as a list of strings where each entry is a line of the pipes section\n",
    "pipes_addendum=pipes_addendum.to_string(header=False,index=False,col_space=10).splitlines()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Tank Coordinates\n",
    "Each tank's coordinates are translated by a XX m in both x and y directions  \n",
    "The coordinates of the tank nodes are merely for display and do not affect the simulation in any way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Translates the tanks by a 100 m in both axes \n",
    "xcoordinates=[x+2 for x in xcoordinates]\n",
    "ycoordinates=[y+2 for y in ycoordinates]\n",
    "\n",
    "# Assemble all lists into a dataframe where each row is the coordinates for one simple tank\n",
    "coordinates_add=pd.DataFrame(list(zip(tankids,xcoordinates,ycoordinates)))\n",
    "# Exports the coordinate section as a list of strings where each entry is a line of the coordinates section\n",
    "coordinates_add=coordinates_add.to_string(header=False,index=False,col_space=10).splitlines()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rewriting Base Demands\n",
    "After creating the simple tanks, the base demands for all demand junctions has to be reset to zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of zero base demands for all nodes\n",
    "zerodemands=[0]*len(all_nodes)\n",
    "# White space indicating no patterns\n",
    "pattern=['     ']*len(all_nodes)\n",
    "semicolons=[';']*len(all_nodes)\n",
    "nodes=pd.DataFrame(list(zip(all_nodes,all_elevations,zerodemands,pattern,semicolons)))\n",
    "nodes=nodes.to_string(header=False,index=False).splitlines()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing the modified STM .inp File\n",
    "We first read through the original .inp file to find the line positions of each of the sections  \n",
    "Then each added or modified section is written into a new file and saved"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# opens .inp file to read\n",
    "file=open(path,'r')\n",
    "lines=[]            # list to store all lines in the .inp file\n",
    "linecount=0         # Counter for the number of lines\n",
    "junctions_marker=0  # To store the line number at which the junctions section starts\n",
    "tanks_marker=0      # To store the line number at which the tanks section starts\n",
    "pipes_marker=0      # To store the line number at which the pumps section starts\n",
    "coords_marker=0     # To store the line number at which the vertices section starts\n",
    "\n",
    "# Loops over each line in the input file \n",
    "for line in file:\n",
    "    # Record the position of the phrase [JUNCTIONS] and add 2 to skip the header line\n",
    "    if re.search('\\[JUNCTIONS\\]',line):\n",
    "        junctions_marker=linecount+2\n",
    "    # Record the position of the phrase [TANKS] and add 2 to skip the header line\n",
    "    if re.search('\\[TANKS\\]',line):\n",
    "        tanks_marker=linecount+3\n",
    "     # Record the position of the phrase [PUMPS] and subtract 1 to add pipes to the end of the pipe section\n",
    "    if re.search('\\[PUMPS\\]',line):\n",
    "        pipes_marker=linecount-1\n",
    "     # Record the position of the phrase [Vertices] and subtract 1 to add Tank cooridnates to the end of the coordinates section\n",
    "    if re.search('\\[VERTICES\\]',line):\n",
    "        coords_marker=linecount-1\n",
    "    linecount+=1\n",
    "    # Store all lines in a list\n",
    "    lines.append(line)\n",
    "file.close()\n",
    "\n",
    "# Translate the pipes marker by the length of the tank section that will be added before it (as it will displace all subsequent lines)\n",
    "pipes_marker+=len(tanks_section)\n",
    "# Translate the coordinates marker by the length of the added tanks and pipes\n",
    "coords_marker+=len(tanks_section)+len(pipes_addendum)\n",
    "\n",
    "# Inserts the created sections in their appropriate location in the list of lines\n",
    "lines[junctions_marker:junctions_marker+len(nodes)]=nodes\n",
    "lines[tanks_marker:tanks_marker]=tanks_section\n",
    "lines[pipes_marker:pipes_marker]=pipes_addendum\n",
    "lines[coords_marker:coords_marker]=coordinates_add\n",
    "\n",
    "# Opens a new file in the same directory to write the modified network .inp file in\n",
    "file=open(path.parent / (name_only+'_CV-Tank.inp'),'w')\n",
    "c=0     #line counter\n",
    "\n",
    "# All lines added by this script are missing a new line character at the end, the conditional statements below add the new line character for these lines only and writes all lines to the file\n",
    "for line in lines:\n",
    "    if c>=junctions_marker and c<=junctions_marker+len(nodes):\n",
    "        file.write(line+'\\n')\n",
    "    elif c>=tanks_marker and c<=tanks_marker+len(tanks_section):\n",
    "        file.write(line+'\\n')\n",
    "    elif c>=pipes_marker and c<=pipes_marker+len(pipes_addendum):\n",
    "        file.write(line+'\\n')\n",
    "    elif c>=coords_marker and c<=coords_marker+len(coordinates_add):\n",
    "        file.write(line+'\\n')\n",
    "    else: file.write(line)    \n",
    "    c+=1\n",
    "file.close()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create network model from input file\n",
    "network = wntr.network.WaterNetworkModel(path.parent / (name_only+'_CV-Tank.inp'))\n",
    "list_of_valves = network.pipe_name_list\n",
    "for pipe in network.pipes():\n",
    "    for valve_loc in list_of_valves:\n",
    "        if pipe[0] == valve_loc:\n",
    "            end_node_i = network.get_node(pipe[1].end_node)\n",
    "            network.add_junction(str(end_node_i)+\"INT\", elevation = network.get_node(end_node_i).elevation, coordinates = end_node_i.coordinates)\n",
    "            network.add_valve(\"TCV\"+pipe[0],str(end_node_i)+\"INT\", pipe[1].end_node_name,0.4,\"TCV\",0,0)\n",
    "            pipe[1].end_node = network.get_node(str(end_node_i)+\"INT\")\n",
    "            \n",
    "\n",
    "path2 = path.parent / (name_only+'_CV-Tank_'+str(len(list_of_valves))+'valves_1.inp')\n",
    "wntr.epanet.io.InpFile().write(path2, wn = network)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References:\n",
    "##### [1] A. Batterman and S. Macke, “A Strategy to Reduce Technical Water Losses for Intermittent Water Supply Systems,” Fachhochschule Nordostniedersachsen, 2001. Accessed: Feb. 14, 2022. [Online]. Available: http://sdteffen.de/diplom/thesis.pdf\n",
    "##### [2] D. D. J. Taylor, A. H. Slocum, and A. J. Whittle, “Demand Satisfaction as a Framework for Understanding Intermittent Water Supply Systems,” Water Resources Research, vol. 55, no. 7, pp. 5217–5237, 2019, doi: 10.1029/2018WR024124.\n",
    "##### [3] B. M. Janet Wagner, U. Shamir, and D. H. Marks, “WATER DISTRIBUTION RELIABILITY: SIMULATION METHODS.”"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.12 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  },
  "vscode": {
   "interpreter": {
    "hash": "dd6a0ae2e89424b22bc0c3fd46707be6f981ef35bdac29f85707eba2fc78ecd7"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}