trimming frames throws error when saving to h5

[Goreg12345]

Let's say I record 100k frames and want to use the parameter frame_trim to trim the first 10k and last 10k frames.

Then, nframes is 80k and frame_range goes from 10k to 90k.

When creating the h5 file, memory is allocated for scalars and frames for 80k frames.

helpers/data::create_extract_h5

# Creating scalar dataset
  for scalar in list(scalars_attrs.keys()):
      h5_file.create_dataset(f'scalars/{scalar}', (nframes,), 'float32', compression='gzip')
      h5_file[f'scalars/{scalar}'].attrs['description'] = scalars_attrs[scalar]

  # Cropped Frames
  h5_file.create_dataset('frames', (nframes, config_data['crop_size'][0], config_data['crop_size'][1]),
                   config_data['frame_dtype'], compression='gzip')

When extracting, the h5 file is filled by indexing via the framerange:

helpers/extract::write_extracted_chunk_to_h5

  # Writing computed scalars to h5 file
  for scalar in scalars:
      h5_file[f'scalars/{scalar}'][frame_range] = results['scalars'][scalar][offset:]

  # Writing frames and mask to h5
  h5_file['frames'][frame_range] = results['depth_frames'][offset:]

frame_range is between 10k-90k, but only 80k frames are allocated, so it will throw and error as soon as it tries to set the frame number 80001:

Traceback (most recent call last):
  File "/root/miniconda3/envs/moseq2-app/bin/moseq2-extract", line 8, in <module>
    sys.exit(cli())
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/util.py", line 111, in invoke
    return super().invoke(ctx)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/cli.py", line 179, in extract
    extract_wrapper(input_file, output_dir, config_data, num_frames=num_frames, skip=skip_completed)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/wrappers.py", line 381, in extract_wrapper
    output_mov_path=movie_filename)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/extract.py", line 161, in process_extract_batches
    write_extracted_chunk_to_h5(h5_file, results, config_data, scalars, frame_range, offset)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_extract/helpers/extract.py", line 38, in write_extracted_chunk_to_h5
    h5_file[f'scalars/{scalar}'][frame_range] = results['scalars'][scalar][offset:]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/dataset.py", line 685, in __setitem__
    selection = sel.select(self.shape, args, dsid=self.id)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 90, in select
    sel[args]
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 401, in __getitem__
    start, count, step, scalar = _handle_simple(self.shape, vector)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 466, in _handle_simple
    x,y,z = _translate_int(int(arg), length)
  File "/root/miniconda3/envs/moseq2-app/lib/python3.7/site-packages/h5py/_hl/selections.py", line 486, in _translate_int
    raise ValueError("Index (%s) out of range (0-%s)" % (exp, length-1))
ValueError: Index (59196) out of range (0-59195)

Solution would be to subtract first_frame_idx from frame_range before writing the data.