#########################################################
fetch blob data from occ side to sot side
#########################################################

Scripts:
========
fetch_telemetry.py
---------------------
Fetch the latest telemetry data using maude.get_blobs() and then strucutre them into
the CSH blob_<part>.json files.

check_msid_status.py
--------------------
called by copy_data_from_occ.py; check status of msids

read_comm_time.py
-----------------
read comm timing and create a table
input:  http://cxc.harvard.edu/mta/ASPECT/arc/
output: <html_dir>/comm_list.hmtl
        <house_keeping>/comm_list

mk_limit_table.py
-----------------
update msid limit table
input: /data/mta4/MTA/data/op_limits/glimmondb.sqlite3
output: <house_keeping>/limit_table (also in SOH_ASVT)

update_msididx_data.py
----------------------
update msididx.blb
input: <house_keeping>/msididx_base
       <house_keeping>/limit_table
output:<html_dir>/msididx.json


next_comm_check.py
------------------
create a display time span till the next comm
input: none
output: <web_dir>/ctest.xml


Supplemental Script:
====================
check_msid_exist_list.py    ---  print out msids in the list which cannot be extracted with maude

check_msid_exist.py         ---  a given msid, it will tell whether it can be extracted with maude


Data:
=====
blob.json:
example: {"msid":"AOPCADMD","index":"2416","time":"2018131120936.285","value":"NPNT","f": "1"}
        msid    --- msid of this data
        index   --- index of this msid corresponding to that of msididx.json
        time    --- time: 2018:131:12:09:36.285
        value   --- current value
        f       --- this is a dummy input for this app

    blob.json is constructed locally by copy_data_from_occ.py using madue function

msididx.json:
example: ,{"name": "AOPCADMD", "idx": 2416, "description": "PCAD MOD E", "sc": ["STBY","NPNT","NMAN","NSUN","PWRF","RMAN","NULL"]}
        name    --- msid
        idx     --- index of this msid
        description --- description of this msid
        sc          --- possible values
        limit/switch/set  --- if switch value takes a cartain value, set is used for expected value. 
        "lim": [{"switch":{"AOPCADMD":"NMAN"},
                  "set":{"wl":-0.03490658,"cl":-0.026179935,"ch":0.026179935,"wh":0.03490658}},
                {"switch":{"AOPCADMD":"NPNT"},"
                 "set":{"wl":-0.0000977384,"cl":-0.0000488692,"ch":0.0000488692,"wh":0.0000977384}},
                    ...
                where wl --- lower warning
                      cl --- lower caution
                      ch --- upper caution
                      wh --- upper warning

msididx.json was originally copied from:
    https://occweb.cfa.harvard.edu/occweb/web/fot_web/software/sandbox/SOT_area/msididx.json


Direcotry:
==========
bin_dir:        /data/mta4/Script/SOH/
house_keeping:  /data/mta4/Script/SOH/house_keeping/
outdir:         /data/mta4/www/CSH/

Web Address:
============
/data/mta4/www/CSH

https://cxc.cfa.harvard.edu/mta/CSH/soh.html
https://cxc.cfa.harvard.edu/mta/CSH/soh_main.html
https://cxc.cfa.harvard.edu/mta/CSH/soh_snap.html

Environment Settings
====================
/proj/sot/ska/bin/python

cron job
========
boba-v as mta:
34 1 * * * cd /data/mta4/Script/SOH/; /data/mta4/Script/SOH/read_comm_time.py > $HOME/Logs/soh_comm_log.cron 
36 1 * * * cd /data/mta4/Script/SOH/; /data/mta4/Script/SOH/mk_limit_table.py > $HOME/Logs/soh_limit_log.cron
40 1 * * * cd /data/mta4/Script/SOH/; /data/mta4/Script/SOH/update_msididx_data.py > $HOME/Logs/soh_msididx_log.cron 
* * * * * cd /data/mta4/Script/SOH/; /data/mta4/Script/SOH/fetch_telem_wrap_script >> $HOME/Logs/soh_fetch_telem.cron
*  * * * * /data/mta/Script/SOH/next_comm_check.py

***************
Notes on htmls 
***************
<outdir> contains the html files, json data, and backbone.js related scripts.

js/lib          --- contains library of backbone.js related javascripts files

js/models       --- contains models to be used
                    mid.js      --- msid model
                    blob.js     --- blob model
                    msidinfo.js --- msididx mode

js/view         --- contains view (html page construction related) javascript
                    msidview.js --- this craete view of msid. it can contain some computation related to the msid.
js/collection   --- collections of javascripts

***********************
Note on special limits
***********************
See: https://docs.google.com/spreadsheets/d/180iuA1_TPXOGpoOlZuk5OMqVH-O1-HD861XB5CSHKOg/edit#gid=0