Running ALE on HRSC

[thareUSGS]

I see ALE running on PDS IMG and ISIS cubes in example, but I get the same not supported error. Confused. I refreshed ALE install just in case but no luck.

ISIS 8bit cubes here and PDS3 IMGs located here:
/scratch/thare/SOCET_GXP/test_models_GXP/HRSC/Gale_Crater_orbit_h4235/stereo_images

here is how I am running ALE

import ale
import json
import os

import glob
for infile in glob.glob("*8b*.cub"):
   print(infile)
   usgscsm_str = ale.loads(infile)
   csm_isd = os.path.splitext(infile)[0] + '.json'
   with open(csm_isd, 'w') as isd_file:
        isd_file.write(usgscsm_str)

[jessemapel]

HRSC wasn't matching well with ISIS for ephemeris data so we've disabled it from load(s) until it lines up better. For right now you need to manually specify the driver to generate an ISD. Here's the modified version of your code, you will need to set ISISROOT and ISISDATA for the kernel expansion to work correctly:

import ale
import json
import os

import glob
for infile in glob.glob("*8b*.cub"):
   print(infile)
   kernels = ale.util.generate_kernels_from_cube(infile, expand=True)
   with ale.drivers.mex_drivers.MexHrscIsisLabelNaifSpiceDriver(infile, props={'kernels': kernels}) as driver:
      usgscsm_isd = ale.formatters.formatter.to_isd(driver)
   usgscsm_str = json.dumps(usgscsm_isd, cls=ale.drivers.AleJsonEncoder)
   csm_isd = os.path.splitext(infile)[0] + '.json'
   with open(csm_isd, 'w') as isd_file:
        isd_file.write(usgscsm_str)

[thareUSGS]

I had to get an extra kernel but I am still getting an error. I tried to debug but didn't get very far.

Traceback (most recent call last):
  File "/scratch/thare/SOCET_GXP/test_models_GXP/HRSC/Gale_Crater_orbit_h4235/stereo_images/run_ALE_cub.py", line 13, in <module>
    usgscsm_isd = ale.formatters.formatter.to_isd(driver)
  File "/home/thare/.conda/envs/ale/lib/python3.9/site-packages/ale/formatters/formatter.py", line 143, in to_isd
    'line' : driver.detector_center_line,
  File "/home/thare/.conda/envs/ale/lib/python3.9/site-packages/ale/base/data_naif.py", line 530, in detector_center_line
    return float(spice.gdpool('INS{}_BORESIGHT_LINE'.format(self.ikid), 0, 1)[0])
  File "/home/thare/.conda/envs/ale/lib/python3.9/site-packages/spiceypy/spiceypy.py", line 123, in with_errcheck
    res = f(*args, **kwargs)
  File "/home/thare/.conda/envs/ale/lib/python3.9/site-packages/spiceypy/spiceypy.py", line 143, in wrapper
    raise stypes.NotFoundError(
spiceypy.utils.exceptions.NotFoundError: Spice returns not found for function: gdpool

[Kelvinrr]

Hm, probably missing a text kernel. The function generate_kernels_from_cube was originally written at a Starbucks with 3 PSLs in my system so it might be acting up not interacting with HRSC well.

According to the stack trace through, there might need to be a specialization for CCD center for HRSC as HRSC uses a _CCD_CENTER array and not _BORESIGHT_LINE/_BORESIGHT_SAMPLE for CCD center. It just seems odd as I recall getting a HRSC ISD successfully before.

[Kelvinrr]

@thareUSGS #425 got the driver running, let me know if those changes work for you.

[thareUSGS]

@Kelvinrr not sure how to get the new code setup to run. For example do I still need to "you will need to set ISISROOT and ISISDATA for the kernel expansion to work correctly"

1. I have my current Anaconda ALE environment from conda-forge
2. Download latest "main" zip from github, unzip, "python setup.py develop"
3. set ISIS environment up for ISIS3DATA
4. do I still run it as Jesse stated above (link) or I stated in the first post?

I did try both, hoping I setup the environment correct and I still get 'No Such Driver for Label' (from the first methods) or from Jesse's I get:

The attempt to load "/usgs/cpkgs/isis3/data/mex/kernels/sclk/MEX_160826_STEP.TSC" by the routine FURNSH failed. It could not be located.

So I created my own ISIS3DATA area with that file and I get errors like

File "/home/thare/.conda/envs/ale/lib/python3.9/site-packages/ale/base/data_naif.py", line 530, in detector_center_line

which makes me think my environment is not ordered correctly? which is:

for p in sys.path:
... print(p)
...

/home/thare/.conda/envs/ale/lib/python39.zip
/home/thare/.conda/envs/ale/lib/python3.9
/home/thare/.conda/envs/ale/lib/python3.9/lib-dynload
/home/thare/.conda/envs/ale/lib/python3.9/site-packages
/scratch/thare/SOCET_GXP/test_models_GXP/HRSC/Gale_Crater_orbit_h4235/stereo_images/ale-master

Any tips would be appreciated.

[Kelvinrr]

@thareUSGS You should be running the same code that Jesse posted.

That error about the missing SCLK is so odd, we definitely do not have that SCLK kernel in our data area, but we have others. After updating the driver and testing output, I had to re-spiceinit the cubes because I encountered the same error. Were these spiceinit-ed by someone outside USGS Astro? I imagine they must of used a different SCLK that we do not have when they first spiceinit-ed the cubes?

[oleg-alexandrov]

In my experiments MexHrscPds3NaifSpiceDriver is working correctly, but MexHrscIsisLabelNaifSpiceDriver is not. The issue seems to come from the fact that the latter's implementation of the line_scan_rate() function does not scale the ephemeris times as expected later by the functions which do time look up. I used the latest ale code from GitHub when looking at this.

I still get a few pixel of disagreement at some locations between the ISIS and CSM models for the h4235_0001_s22 dataset. Not fully sure if it is because anything I do. It would be nice if at some point this is evaluated in knoten.

[thareUSGS]

With this new report on MexHrscIsisLabelNaifSpiceDriver I will re-open this issue and hope we can get some testing in place (e.g. knoten).