CAT_NO_ACCESS_PERMISSION and USER_PACKSTRUCT_INPUT_ERR on open/read/close activity

[kjsanger]

iRODS: 4.2.7, Linux
Plugin: 2.7.0

A local user on a (federated) zone, with read-only access to data on that zone used baton-get to read a file. baton-get uses the iRODS open/read/close API, which triggers irods::access_time updates in this version of the plugin. The same user also uses iget (which does not trigger irods::access_time updates in this version of the plugin)

Expected:

The file to be downloaded successfully with both clients, with only baton-get triggering an access time update.

Observed:

iget successfully downloads the file and irods::access_time is not set.

baton-get fails when the update of irods::acces_time is attempted. Setting access time fails with CAT_NO_ACCESS_PERMISSION, but an underlying USER_PACKSTRUCT_INPUT_ERR.

xxxxxx@xxxxxxx:~/xxxxx/genotype_data_check$ baton-get --avu < qc_plex_list_060522_oneplate.txt 
remote addresses: xxx.xxx.xxx.xxx ERROR: unpackNonpointerItem: strlen of msg > dim size, content: iRODS Exception:
    file: /irods_plugin/libirods_rule_engine_plugin-apply_access_time.cpp
    function: void (anonymous namespace)::update_access_time_for_data_object(rsComm_t *, const std::string &, const std::string &)
    line: 36
    code: -818000 (CAT_NO_ACCESS_PERMISSION)
    message:
        failed to set access time for [xxxxxx/xxxxxx.csv]
        : [-]	/irods/server/re/include/irods_re_plugin.hpp:325:irods::error irods::dynamic_operation_execution_manager<std::__1::tuple<>, RuleExecInfo *, irods::rule_execution_manager_pack::DONT_AUDIT_RULE>::call(std::string, std::string, OP, As &&...) [T = std::__1::tuple<>, C = RuleExecInfo *, Audit = irods::rule_execution_manager_pack::DONT_AUDIT_RULE, OP = std::__1::function<irods::error (const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack &&)>, As = <const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack>] :  status [CAT_NO_ACCESS_PERM 
remote addresses: 10.160.2.194 ERROR: readAndProcApiReply:unpackStruct error. status = -308000 status = -308000 USER_PACKSTRUCT_INPUT_ERR
{"collection": "/xxxxxx, "data_object": "xxxxxx.csv", "error": {"message": "Failed to close data object: '/xxxxxx/xxxxxx.csv' error -818000 CAT_NO_ACCESS_PERMISSION", "code": -818000}}
2022-05-06T10:34:06Z WARN Processed 1 items with 1 errors

In this case is CAT_NO_ACCESS_PERMISSION genuine, or an artefact of failure due to USER_PACKSTRUCT_INPUT_ERR?

[kript]

N.B. (sorry for the pedantry)

The file to be downloaded successfully with both clients, with only baton-get triggering an access time update.

Is only expected because of #200 - we would normally expect iget to also trigger an access time update.

[kjsanger]

While this error was reproducible multiple times on the day, this is no longer the case. These all now succeed:

• python-irodsclient session.data_objects.get()
• baton-get --avu
• baton-get (i.e. no metadata)

Trying to manually remove irods::access_time first does not change the outcome. However, I'm not sure that I can effectively remove it with imeta rm -d because when I look with imeta ls -d, it is immediately restored.

Maybe the error occurs only on the initial absence of irods::access_time?

[trel]

First, that seems like great news.

Second, imeta ls -d adds irods::access_time?! That feels... surprising.

[kjsanger]

My error. It had just got a new access time, so my imeta rm -d had no effect. Confirmed and double-checked that I'm removing it this time. I also made the error of calling session.data_objects.get() with only its first argument when I should have used the two argument version.

Now things are a little different:

As rodsadmin on the federated zone:

• baton-get --avu succeeds and updates irods::access_time
• session.data_object_get(remote, local) was not tried (because I don't have a convenient install at the moment)

As a regular user on my home zone (federated to the above):

• baton-get --avu succeeds but does not update irods::access_time
• session.data_object_get(remote, local) fails with:

  File "/software/team117/kdj/lib/miniconda/envs/irods-4.2.7/lib/python3.9/site-packages/irods/manager/data_object_manager.py", line 122, in get
    self._download(path, local_path, num_threads = num_threads, **options)
  File "/software/team117/kdj/lib/miniconda/envs/irods-4.2.7/lib/python3.9/site-packages/irods/manager/data_object_manager.py", line 99, in _download
    with open(local_file, 'wb') as f, self.open(obj, 'r', **options) as o:
  File "/software/team117/kdj/lib/miniconda/envs/irods-4.2.7/lib/python3.9/site-packages/irods/manager/data_object_manager.py", line 326, in open
    desc = conn.recv().int_info
  File "/software/team117/kdj/lib/miniconda/envs/irods-4.2.7/lib/python3.9/site-packages/irods/connection.py", line 128, in recv
    raise get_exception_by_code(msg.int_info, err_msg)
irods.exception.SYS_RESC_DOES_NOT_EXIST: None

which I guess might be because it's looking at the wrong zone. I'm calling it so:

    try:
        env_file = os.environ['IRODS_ENVIRONMENT_FILE']
    except KeyError:
        env_file = os.path.expanduser('~/.irods/irods_environment.json')

    ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH,
                                             cafile=None, capath=None, cadata=None)
    ssl_settings = {'ssl_context': ssl_context}

    with iRODSSession(irods_env_file=env_file, zone="seq", **ssl_settings) as session:
        obj = session.data_objects.get("/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/S091_1662457010.csv",
                                       "foo.csv")

I've tried both with and without the zone="seq" argument. I'm using python-irodsclient 1.1.3 from pip.

[kjsanger]

I can now reproduce this 100% of the time, including with the Pyton API. Both a plain baton-get and baton-get --avu give the same error:

remote addresses: xxx.xxx.xxx.xxx ERROR: unpackNonpointerItem: strlen of msg > dim size, content: iRODS Exception:
    file: /irods_plugin/libirods_rule_engine_plugin-apply_access_time.cpp
    function: void (anonymous namespace)::update_access_time_for_data_object(rsComm_t *, const std::string &, const std::string &)
    line: 36
    code: -818000 (CAT_NO_ACCESS_PERMISSION)
    message:
        failed to set access time for [/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/S091_1662457010.csv]
        : [-]   /irods/server/re/include/irods_re_plugin.hpp:325:irods::error irods::dynamic_operation_execution_manager<std::__1::tuple<>, RuleExecInfo *, irods::rule_execution_manager_pack::DONT_AUDIT_RULE>::call(std::string, std::string, OP, As &&...) [T = std::__1::tuple<>, C = RuleExecInfo *, Audit = irods::rule_execution_manager_pack::DONT_AUDIT_RULE, OP = std::__1::function<irods::error (const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack &&)>, As = <const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack>] :  status [CAT_NO_ACCESS_PERM
remote addresses: xxx.xxx.xxx.xxx ERROR: readAndProcApiReply:unpackStruct error. status = -308000 status = -308000 USER_PACKSTRUCT_INPUT_ERR
{"coll": "/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/", "obj": "S091_1662457010.csv", "error": {"message": "Failed to close data object: '/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/S091_1662457010.csv' error -818000 CAT_NO_ACCESS_PERMISSION", "code": -818000}}
2022-05-12T12:13:25Z WARN Processed 1 items with 1 errors

Note that in this case (connecting to the same zone as before, but from a different host) we no longer see the packStruct error.

With the Python API:

Traceback (most recent call last):
  File "/home/xxx/kdj_tmp/./test.py", line 24, in <module>
    main()
  File "/home/xxx/kdj_tmp/./test.py", line 19, in main
    obj = session.data_objects.get("/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/S091_1662457010.csv",
  File "/home/xxx/kdj_tmp/miniconda/envs/irods_test/lib/python3.9/site-packages/irods/manager/data_object_manager.py", line 122, in get
    self._download(path, local_path, num_threads = num_threads, **options)
  File "/home/xxx/kdj_tmp/miniconda/envs/irods_test/lib/python3.9/site-packages/irods/manager/data_object_manager.py", line 108, in _download
    f.write(chunk)
  File "/home/xxx/kdj_tmp/miniconda/envs/irods_test/lib/python3.9/site-packages/irods/data_object.py", line
174, in close
    self.conn.close_file(self.desc, **self.options)
  File "/home/xxx/kdj_tmp/miniconda/envs/irods_test/lib/python3.9/site-packages/irods/connection.py", line 605, in close_file
    self.recv()
  File "/home/xxx/kdj_tmp/miniconda/envs/irods_test/lib/python3.9/site-packages/irods/connection.py", line 128, in recv
    raise get_exception_by_code(msg.int_info, err_msg)
irods.exception.CAT_NO_ACCESS_PERMISSION: iRODS Exception:
    file: /irods_plugin/libirods_rule_engine_plugin-apply_access_time.cpp
    function: void (anonymous namespace)::update_access_time_for_data_object(rsComm_t *, const std::string &, const
std::string &)
    line: 36
    code: -818000 (CAT_NO_ACCESS_PERMISSION)
    message:
        failed to set access time for [/seq/fluidigm/00/13/aa/1662457010/02/4b/ab/S091_1662457010.csv]
        : [-]   /irods/server/re/include/irods_re_plugin.hpp:325:irods::error irods::dynamic_operation_execution_manager<std::__1::tuple<>, RuleExecInfo *, irods::rule_execution_manager_pack::DONT_AUDIT_RULE>::call(std::string, std::string, OP, As &&...) [T = std::__1::tuple<>, C = RuleExecInfo *, Audit = irods::rule_execution_manager_pack::DONT_AUDIT_RULE, OP = std::__1::function<irods::error (const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack &&)>, As = <const std::__1::basic_string<char> &, irods::re_pack_inp<std::__1::tuple<> > &, irods::unpack>] :  status [CAT_NO_ACCESS_PERM

So the core issue looking more like it relates to permissions and the packStruct is separate. The iRODS user these clients are running as, is a read-only account because of a policy of least privilege.

[trel]

Excellent. Thanks for the detective work.

If this is due to simply matching the wrong username (b/c federation), then this is a symptom/duplicate of #187.

If this is more subtle and due to the read-only nature of the connected user, then we will need to think slightly differently about how to address it.