Fix bug where the configuration of the hbaseRootdir at the role level is ignored

[maltesander]

Description

The problem was defaulting in the product config callbacks like:

 result.insert( 
     HBASE_ROOTDIR.to_string(), 
     Some( 
         self.hbase_rootdir 
             .as_deref() 
             .unwrap_or(HBASE_ROOT_DIR_DEFAULT) 
             .to_string(), 
     ), 
 ); 

If the hbaseRootdir was only set on the role, we defaulted the rolegroup to HBASE_ROOT_DIR_DEFAULT ("/hbase"), which during merging overrode the value set on the role. The solution here is to remove the defaulting and it works.
One problem remained, if hbaseRootdir was not set at all, there was no default set anymore. The product config did not add the default due to the field being marked as not required.
In order to keep the defaulting behavior as is and not mess with existing defaulted installation, the default for hbase.root.dir in the product config was changed to "/hbase" and the field marked as required (on all roles).
The docs state though:

The directory shared by region servers and into which HBase persists. The URL should be 'fully-qualified' to include the filesystem scheme. For example, to specify the HDFS directory '/hbase' where the HDFS instance's namenode is running at namenode.example.org on port 9000, set this value to: hdfs://namenode.example.org:9000/hbase. By default, we write to whatever ${hbase.tmp.dir} is set too -- usually /tmp -- so change this configuration or else all data will be lost on machine restart.

So i think we can only use it on the regionservers? Ping @lfrancke

Edit: Lars would bet its at least required in the masters, so i would keep it as is (required in all roles in the product config)

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

Preview Give feedback

1. Changes are OpenShift compatible
2. CRD changes approved
3. CRD documentation for all fields, following the style guide.
4. Helm chart can be installed and deployed operator works
5. Integration tests passed (for non trivial changes)
6. Changes need to be "offline" compatible

Reviewer

Preview Give feedback

1. (Integration-)Test cases added
2. Changelog updated
3. Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

Preview Give feedback

1. Feature Tracker has been updated
2. Proper release label has been added
3. Roadmap has been updated

[siegfriedweber]

LGTM

[lfrancke]

I'm a bit confused by the comment. That seems to indicate that it's not required by the RegionServer?

[siegfriedweber]

I'm a bit confused by the comment. That seems to indicate that it's not required by the RegionServer?

The master apparently tells the RegionServers where the root dir is located. I have not checked this assumption in the code but I tested it:

If I set different root dirs for the roles ...

$ git diff
diff --git a/tests/templates/kuttl/smoke/30-install-hbase.yaml.j2 b/tests/templates/kuttl/smoke/30-install-hbase.yaml.j2
index ac1bed8..ea5d883 100644
--- a/tests/templates/kuttl/smoke/30-install-hbase.yaml.j2
+++ b/tests/templates/kuttl/smoke/30-install-hbase.yaml.j2
@@ -21,6 +21,7 @@ spec:
 {% endif %}
   masters:
     config:
+      hbaseRootdir: /rtmaster
       logging:
         enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
     roleGroups:
@@ -32,6 +33,7 @@ spec:
         replicas: 2
   regionServers:
     config:
+      hbaseRootdir: /rtregion
       logging:
         enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
     roleGroups:
@@ -43,6 +45,7 @@ spec:
         replicas: 2
   restServers:
     config:
+      hbaseRootdir: /rtrest
       logging:
         enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
     roleGroups:

... then the one set for the master is effective:

$ bin/hdfs dfs -fs hdfs://test-hdfs/ -ls /
Found 1 items
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster
$ bin/hdfs dfs -fs hdfs://test-hdfs/ -ls /rtmaster
Found 12 items
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/.hbck
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/.tmp
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/MasterData
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/WALs
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:10 /rtmaster/archive
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/corrupt
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:01 /rtmaster/data
-rw-r--r--   3 stackable supergroup         42 2024-11-15 11:00 /rtmaster/hbase.id
-rw-r--r--   3 stackable supergroup          7 2024-11-15 11:00 /rtmaster/hbase.version
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/mobdir
drwxr-xr-x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/oldWALs
drwx--x--x   - stackable supergroup          0 2024-11-15 11:00 /rtmaster/staging

[maltesander]

To add to this, we (or I?) decided to still set the hbaseRootdir on the region and rest server (even if not required) in order to keep existing "functionality" or at least avoid restarts in exitsting installations due to config changes.