UpdateFM40BasicCrosswalk_2M_SC_CC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Surface Fuel Updates.. buckle up\n",
    "\n",
    "There will be 4 GridFire runs representing Year 5, Year 10, Year 15, and Year 20.\n",
    "\n",
    "We have 4 year intervals in our ee_treatment simulation images, the pixel values and their Year Interval correspond:\n",
    "\n",
    "1 (years 1-5) \n",
    "2 (years 6-10) \n",
    "3 (years 11-15)\n",
    "4 (years 16-20)\n",
    "\n",
    "We want to generate newly updated FM40 layers for each GridFire run year. \n",
    "we will be updating only those treated pixels with value 1, 2, 3, and 4 from our ee_treatment() images.\n",
    "\n",
    "For each of those pixel values (1,2,3,4) we will create a new updated FM40 image each from the original and re-treatment images. \n",
    "\n",
    "The workflow is this:\n",
    "\n",
    "For original treatment and re-treatment (retreatment_only) images:\n",
    "* mask to only pixels with that given value, \n",
    "* use a simplified lookup table .csv on Cloud Storage (containing from_codes and to_codes) to remap the baseline FM40 values to their updated values in those treated pixels\n",
    "* then fill in the 'untreated pixels' with the baseline FM40 image\n",
    "\n",
    "Finally, mosaic both remapped FM40s together, taking the lowest FM40 at each pixel. \n",
    "\n",
    "Expected Output is three Updated FM40 images per RunID\n",
    "\n",
    "Example: RUNID1 = SN WUI 500k\n",
    "Outputs:\n",
    "* GEE Asset - projects/mas-gee/assets/treatment_scenarios/RunID1/FM40_pctGroup25 (updated in ee_treatment pixels==1)\n",
    "* GEE Asset - projects/mas-gee/assets/treatment_scenarios/RunID1/FM40_pctGroup50 (updated in ee_treatment pixels==2)\n",
    "* GEE Asset - projects/mas-gee/assets/treatment_scenarios/RunID1/FM40_pctGroup75 (updated in ee_treatment pixels==3)\n",
    "* GEE Asset - projects/mas-gee/assets/treatment_scenarios/RunID1/FM40_pctGroup100 (updated in ee_treatment pixels==4)\n",
    "\n",
    "We also are exporting them to Google Drive in case there is a need\n",
    "* Google Drive - RunID1_FM40_pctGroup25.tif\n",
    "* Google Drive - RunID1_FM40_pctGroup50.tif\n",
    "* Google Drive - RunID1_FM40_pctGroup75.tif\n",
    "* Google Drive - RunID1_FM40_pctGroup100.tif\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "import ee\n",
    "import os\n",
    "import math\n",
    "from datetime import datetime\n",
    "from src.utils import ee_treatments\n",
    "from src.utils import runs\n",
    "from src.utils.check_exists import check_exists\n",
    "from src.utils.yml_params import get_export_params\n",
    "from src.utils.ee_csv_parser import parse_txt, to_numeric\n",
    "yml_file = os.path.join(os.getcwd(),'config.yml')\n",
    "CRS,EXPORT_SCALE = get_export_params(yml_file)\n",
    "\n",
    "ee.Initialize(project='mas-gee')\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# In development phase\n",
    "tag = 'fm40_updates'\n",
    "ee.data.setWorkloadTag(tag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "INPUT YOUR TREATED IMAGE PATH BELOW, THEN RUN ALL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plug in treatment scenario ee.Image path\n",
    "ee_treated_path = \"projects/mas-gee/assets/treatment_scenarios/RunID27/SC_Fire_2m_20231020\"\n",
    "retreatment_path = \"projects/mas-gee/assets/treatment_scenarios/RunID27/SC_Fire_2m_20231103_retreatment_only\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "ee_treated = ee.Image(ee_treated_path)\n",
    "retreated = ee.Image(retreatment_path)\n",
    "baseline_fm40 = ee.Image(\"projects/pyregence-ee/assets/conus/landfire/fbfm40/LF2022_FBFM40_220\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[91, 92, 93, 98, 99, 101, 102, 103, 104, 121, 122, 123, 141, 142, 143, 144, 145, 146, 147, 161, 162, 163, 165, 181, 182, 183, 184, 185, 186, 187, 188, 189, 201, 202]\n",
      "[91, 92, 93, 98, 99, 101, 101, 101, 101, 121, 121, 121, 141, 141, 141, 141, 142, 141, 142, 161, 161, 161, 161, 181, 181, 181, 181, 181, 181, 181, 181, 181, 201, 201]\n"
     ]
    }
   ],
   "source": [
    "# test ee blob table encoding working properly\n",
    "blob = ee.Blob(\"gs://landfire/LFTFCT_tables/million-acres-LUTs/LANDFIRE fuel model subs - FM40.csv\")\n",
    "table = parse_txt(blob)\n",
    "\n",
    "# read in the encoded value list as numeric\n",
    "from_codes = to_numeric(ee.List(table.get(\"Original FM40_number\")))\n",
    "# read the list of values to remap to as numeric\n",
    "to_codes = to_numeric(ee.List(table.get(\"Updated FM40_number\")))\n",
    "\n",
    "print(from_codes.getInfo())\n",
    "print(to_codes.getInfo())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- For 2.3M intensity, SC and CC regions, we have re-treatments in road grid pixels, and unique floating point numbers distinguish a pixel that was treated twice from pixels only treated once. (See the `SCandCC_*_2Mcustom.ipynb` notebooks)\n",
    "\n",
    "For run1 (Year5): \n",
    "* crosswalk only pixel value 1 (areas only treated once, no re-treatment)\n",
    "\n",
    "For run2 (Year10):\n",
    "* crosswalk pixel value 1.5 (areas treated in YI1 (Year5) and re-treated in YI2 (Year10))\n",
    "* crosswalk pixel value 2 (areas only treated in YI2 (Year10))\n",
    "\n",
    "For run3 (Year20):\n",
    "* crosswalk pixel value 3.5 (areas treated in YI3 (Year15) and re-treated in YI4 (Year20))\n",
    "* crosswalk pixel value 4 (areas only treated in YI4 (Year20)) -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.M SC/CC Custom Workflow\n",
    "In `TreatmentSimulator_2MCustom` notebooks we exported `ee_treatments` and `retreatment_only` images separately\n",
    "\n",
    "In this Notebook we:\n",
    "1) FM40 crosswalk each treatment image separately for the 4 GF runs \n",
    "\n",
    "*run1 (Year5 | pixel value=1), run2 (Year10 | pixel value=2) , run3 (Year15 | pixel value=3) and run4 (Year20 | pixel value=4)*\n",
    "\n",
    "\n",
    "2) then merging the two updated FM40s together in each GFrun pair, taking the min in overlapping pixels (remaps only go down, see above `from_codes` and `to_codes` lists)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dumb mode - it works but i'm ashamed\n",
    "\n",
    "# run 1 (pctGroup25)\n",
    "if 'WUI' in ee_treated_path: # when we ran ee_treatmetns for WUI, 1's and 4's shared same footprint and 4 burned over 1's (took the max)\n",
    "    run1_ee_treatment = ee_treated.eq(4).selfMask()\n",
    "else:\n",
    "    run1_ee_treatment = ee_treated.eq(1).selfMask() # for RFFC and Fire runs\n",
    "\n",
    "run1_ee_treatment_remap = baseline_fm40.updateMask(run1_ee_treatment).remap(from_codes,to_codes)\n",
    "run1_ee_treatment_fm40 = run1_ee_treatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',25)\n",
    "\n",
    "# we don't re-treat in Year5 (run1 | YI1 whatever you want to call it) \n",
    "run1_fm40_mosaic = run1_ee_treatment_fm40 \n",
    "\n",
    "# run 2 (pctGroup50)\n",
    "run2_ee_treatment = ee_treated.eq(2).selfMask()\n",
    "run2_retreatment = retreated.eq(2).selfMask()\n",
    "\n",
    "run2_ee_treatment_remap = baseline_fm40.updateMask(run2_ee_treatment).remap(from_codes,to_codes)\n",
    "run2_ee_treatment_fm40 = run2_ee_treatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',50)\n",
    "\n",
    "run2_retreatment_remap = baseline_fm40.updateMask(run2_retreatment).remap(from_codes,to_codes)\n",
    "run2_retreatment_fm40 = run2_retreatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',50)\n",
    "\n",
    "run2_fm40_mosaic = (ee.Image.cat(run2_ee_treatment_fm40,\n",
    "                                 run2_retreatment_fm40)\n",
    "                    .reduce(ee.Reducer.min())\n",
    "                    .rename('new_fm40')\n",
    "                    .set('pctGroup',50))\n",
    "\n",
    "# run 3  (pctGroup75)\n",
    "run3_ee_treatment = ee_treated.eq(3).selfMask()\n",
    "run3_retreatment = retreated.eq(3).selfMask()\n",
    "\n",
    "run3_ee_treatment_remap = baseline_fm40.updateMask(run3_ee_treatment).remap(from_codes,to_codes)\n",
    "run3_ee_treatment_fm40 = run3_ee_treatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',75)\n",
    "\n",
    "run3_retreatment_remap = baseline_fm40.updateMask(run3_retreatment).remap(from_codes,to_codes)\n",
    "run3_retreatment_fm40 = run3_retreatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',75)\n",
    "\n",
    "run3_fm40_mosaic = (ee.Image.cat(run3_ee_treatment_fm40,\n",
    "                                 run3_retreatment_fm40)\n",
    "                    .reduce(ee.Reducer.min())\n",
    "                    .rename('new_fm40')\n",
    "                    .set('pctGroup',75))\n",
    "\n",
    "updated_fm40s = [run1_fm40_mosaic,\n",
    "                 run2_fm40_mosaic,\n",
    "                 run3_fm40_mosaic]\n",
    "\n",
    "# run 4  (only for RFFC and Fire runs, we've already handled WUI in run1 above)\n",
    "run4_ee_treatment = ee_treated.eq(4).selfMask()\n",
    "run4_retreatment = retreated.eq(4).selfMask()\n",
    "\n",
    "run4_ee_treatment_remap = baseline_fm40.updateMask(run4_ee_treatment).remap(from_codes,to_codes)\n",
    "run4_ee_treatment_fm40 = run4_ee_treatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',100)\n",
    "\n",
    "run4_retreatment_remap = baseline_fm40.updateMask(run4_retreatment).remap(from_codes,to_codes)\n",
    "run4_retreatment_fm40 = run4_retreatment_remap.unmask(baseline_fm40).rename('new_fm40').set('pctGroup',100)\n",
    "\n",
    "run4_fm40_mosaic = (ee.Image.cat(run4_ee_treatment_fm40,\n",
    "                                 run4_retreatment_fm40)\n",
    "                    .reduce(ee.Reducer.min())\n",
    "                    .rename('new_fm40')\n",
    "                    .set('pctGroup',100))\n",
    "\n",
    "if 'WUI' in ee_treated_path:\n",
    "    updated_fm40s = [run1_fm40_mosaic,\n",
    "                     run2_fm40_mosaic,\n",
    "                     run3_fm40_mosaic,\n",
    "                     run1_fm40_mosaic]\n",
    "else:\n",
    "    updated_fm40s = [run1_fm40_mosaic,\n",
    "                    run2_fm40_mosaic,\n",
    "                    run3_fm40_mosaic,\n",
    "                    run4_fm40_mosaic]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5967316457874ac69539deb715df336e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map(center=[20, 0], controls=(WidgetControl(options=['position', 'transparent_bg'], widget=HBox(children=(Togg…"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import geemap\n",
    "Map = geemap.Map()\n",
    "\n",
    "Map.addLayer(baseline_fm40,{'min':90,'max':205},'baseline fm40')\n",
    "\n",
    "# run 1 output steps\n",
    "Map.addLayer(run1_ee_treatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},' run1 ee_treatments')\n",
    "Map.addLayer(run1_fm40_mosaic,{'min':90,'max':205,},'updated fm40 mosaic run1') \n",
    "\n",
    "# run 2 output steps\n",
    "Map.addLayer(run2_ee_treatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run2 ee_treatments')\n",
    "Map.addLayer(run2_retreatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run2 retreated')\n",
    "Map.addLayer(run2_ee_treatment_fm40,{'min':90,'max':205,},'run2 ee_treatment fm40')\n",
    "Map.addLayer(run2_retreatment_fm40,{'min':90,'max':205,},'run2 retreatment fm40') \n",
    "Map.addLayer(run2_fm40_mosaic,{'min':90,'max':205,},'run2 updated fm40 mosaic') \n",
    "\n",
    "# run 3 output steps\n",
    "Map.addLayer(run3_ee_treatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run3 ee_treatments')\n",
    "Map.addLayer(run3_retreatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run3 retreated')\n",
    "Map.addLayer(run3_ee_treatment_fm40,{'min':90,'max':205,},'run3 ee_treatment fm40')\n",
    "Map.addLayer(run3_retreatment_fm40,{'min':90,'max':205,},'run3 retreatment fm40') \n",
    "Map.addLayer(run3_fm40_mosaic,{'min':90,'max':205,},'run3 updated fm40 mosaic') \n",
    "\n",
    "# run 4 output steps\n",
    "Map.addLayer(run4_ee_treatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run4 ee_treatments')\n",
    "Map.addLayer(run4_retreatment,{'min':1,'max':4,'palette':['blue','red','green','purple']},'run4 retreated')\n",
    "Map.addLayer(run4_ee_treatment_fm40,{'min':90,'max':205,},'run4 ee_treatment fm40')\n",
    "Map.addLayer(run4_retreatment_fm40,{'min':90,'max':205,},'run4 retreatment fm40') \n",
    "Map.addLayer(run4_fm40_mosaic,{'min':90,'max':205,},'run4 updated fm40 mosaic') \n",
    "\n",
    "# Map.centerObject(treated,11)\n",
    "\n",
    "Map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Export each updated FM40 (run1,run2,run3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Export Started for projects/mas-gee/assets/treatment_scenarios/RunID27/FM40_pctGroup25\n",
      "Export Started (Drive): RunID27_FM40_pctGroup25\n",
      "Export Started for projects/mas-gee/assets/treatment_scenarios/RunID27/FM40_pctGroup50\n",
      "Export Started (Drive): RunID27_FM40_pctGroup50\n",
      "Export Started for projects/mas-gee/assets/treatment_scenarios/RunID27/FM40_pctGroup75\n",
      "Export Started (Drive): RunID27_FM40_pctGroup75\n",
      "Export Started for projects/mas-gee/assets/treatment_scenarios/RunID27/FM40_pctGroup100\n",
      "Export Started (Drive): RunID27_FM40_pctGroup100\n"
     ]
    }
   ],
   "source": [
    "# Export each ee_treatment FM40 in list\n",
    "# use California as output region\n",
    "ca = ee.FeatureCollection(\"TIGER/2018/States\").filter(ee.Filter.eq(\"NAME\",\"California\")).first().geometry()\n",
    "\n",
    "# use RUNID string as sub-folder in output Gdrive folder\n",
    "runID_drive = os.path.basename(os.path.dirname(ee_treated_path)) # to get RUNID string\n",
    "\n",
    "for img in list(range(4)):\n",
    "    output = ee.Image(updated_fm40s[img]).toByte()\n",
    "    pctGroup = output.get('pctGroup').getInfo()\n",
    "    desc = f\"UpdateFM40_pctGroup{pctGroup}\"\n",
    "    output_path = os.path.dirname(ee_treated_path)+\"/\"+f\"FM40_pctGroup{pctGroup}\"\n",
    "  \n",
    "    ee_treatments.export_img(output,desc,output_path,ca,30,'EPSG:3310')\n",
    "    \n",
    "    ee_treatments.export_image_to_drive(image=output,\n",
    "                                        description=desc+'-Drive',\n",
    "                                        region=ca,\n",
    "                                        scale=30,\n",
    "                                        crs='EPSG:3310',\n",
    "                                        folder=f\"MAS_FM40_for_GridFire\",\n",
    "                                        fileNamePrefix=f\"{runID_drive}_FM40_pctGroup{pctGroup}\",\n",
    "                                        maxPixels=1e12)\n",
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gee",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}