examples/pst_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Pst class\n",
    "\n",
    "The `pst_handler` module contains the `Pst` class for dealing with pest control files.  It relies heavily on `pandas` to deal with tabular sections, such as parameters, observations, and prior information.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import os\n",
    "import numpy as np\n",
    "import pyemu\n",
    "from pyemu import Pst"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to pass the name of a pest control file to instantiate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pst_name = os.path.join(\"henry\",\"pest.pst\")\n",
    "p = Pst(pst_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of the relevant parts of the pest control file are attributes of the `pst` class with the same name:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.parameter_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.observation_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.prior_information.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A residual file (`.rei` or `res`) can also be passed to the `resfile` argument at instantiation to enable some simple residual analysis and weight adjustments.  If the residual file is in the same directory as the pest control file and has the same base name, it will be accessed automatically:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `pst` class has some `@decorated` convience methods related to the residuals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.phi,p.phi_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some additional `@decorated` convience methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.npar,p.nobs,p.nprior)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.par_groups,p.obs_groups)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(type(p.par_names)) # all parameter names\n",
    "print(type(p.adj_par_names)) # adjustable parameter names\n",
    "print(type(p.obs_names)) # all observation names\n",
    "print(type(p.nnz_obs_names)) # non-zero weight observations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"control_data\" section of the pest control file is accessible in the `Pst.control_data` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('jacupdate = {0}'.format(p.control_data.jacupdate))\n",
    "print('numlam = {0}'.format(p.control_data.numlam))\n",
    "p.control_data.numlam = 100\n",
    "print('numlam has been changed to --> {0}'.format(p.control_data.numlam))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Pst` class also exposes a method to get a new `Pst` instance with a subset of parameters and or obseravtions.  Note this method does not propogate prior information to the new instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pnew = p.get(p.par_names[:10],p.obs_names[-10:])\n",
    "print(pnew.prior_information)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also write a pest control file with altered parameters, observations, and/or prior information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pnew.write(\"test.pst\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some other methods in `Pst` include:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add preferred value regularization with weights proportional to parameter bounds\n",
    "pyemu.utils.helpers.zero_order_tikhonov(pnew)\n",
    "pnew.prior_information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add preferred value regularization with unity weights\n",
    "pyemu.utils.helpers.zero_order_tikhonov(pnew,parbounds=False)\n",
    "pnew.prior_information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some more `res` functionality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# adjust observation weights to account for residual phi components\n",
    "#pnew = p.get()\n",
    "print(p.phi, p.nnz_obs, p.phi_components)\n",
    "p.adjust_weights_discrepancy()\n",
    "print(p.phi, p.nnz_obs, p.phi_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "adjust observation weights by an arbitrary amount by groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.phi, p.nnz_obs, p.phi_components)\n",
    "grp_dict = {\"head\":100}\n",
    "p.adjust_weights(obsgrp_dict=grp_dict)\n",
    "print(p.phi, p.nnz_obs, p.phi_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "adjust observation weights by an arbitrary amount by individual observations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.phi, p.nnz_obs, p.phi_components)\n",
    "obs_dict = {\"h_obs01_1\":25}\n",
    "p.adjust_weights(obs_dict=obs_dict)\n",
    "print(p.phi, p.nnz_obs, p.phi_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "setup weights inversely proportional to the observation values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.adjust_weights_discrepancy()\n",
    "print(p.phi, p.nnz_obs, p.phi_components)\n",
    "p.proportional_weights(fraction_stdev=0.1,wmax=20.0)\n",
    "print(p.phi, p.nnz_obs, p.phi_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}