Exclude CockroachDB's hidden column from a catalog

[stonewhitener]

CockroachDB adds a hidden column called crdb_internal_* when a hash-sharded primary key is used, that should not be directly used via the catalog.

https://www.cockroachlabs.com/docs/stable/hash-sharded-indexes#create-a-table-with-a-hash-sharded-primary-key

[apavlo]

Why didn't you just add crdb_internal_ here?

[stonewhitener]

This is because the hidden columns to be excluded here must be determined using the startsWith() method, not the equals() method applied to each element of excludedColumns. excludeColumns is only used to exclude a hidden column rowid for CockroachDB.

[stonewhitener]

For example, the DDL below generates a hidden column crdb_internal_ts_shard_20.

CREATE TABLE products (
    ts DECIMAL PRIMARY KEY USING HASH WITH (bucket_count = 20),
    product_id INT8
);

A suffix of a hidden column name is determined based on the primary key name and bucket size specified in the DDL. We can not exclude such columns by simply applying equals().

[apavlo]

Actually, what if we create a list of compiled regular expressions and then have it check that instead having separate logic for equals and startsWith.