Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add table metadata service #2434

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

inv-jishnu
Copy link
Contributor

@inv-jishnu inv-jishnu commented Dec 19, 2024

Description

Added table metadata service and some related DTO classes

Related issues and/or PRs

NA

Changes made

Added table metadata service and some related DTO classes. The tablemetadata service is also used in data loader API.
The spotbug exlude is the same as the one added in other PR to resolve issue due lombok.

Checklist

The following is a best-effort checklist. If any items in this checklist are not applicable to this PR or are dependent on other, unmerged PRs, please still mark the checkboxes after you have read and understood each item.

  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes.
  • Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
  • Tests (unit, integration, etc.) have been added for the changes.
  • My changes generate no new warnings.
  • Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

Road map to merge remaining data loader core files. Current status

  • General

  • Export

  • Import

    • Dto classes and utilities: 1 PR incoming
    • Import processor and task code: 2-3 PRs incoming
    • Code for Import transaction batch and data chunk import: 1 PR Incoming
    • ControlFile related Dtos: 1 PR incoming
    • Import logger: 1 PR incoming
    • Import Manager: 1 PR incoming

Release notes

NA

@inv-jishnu inv-jishnu added the enhancement New feature or request label Dec 19, 2024
@inv-jishnu inv-jishnu marked this pull request as draft December 19, 2024 10:02
@inv-jishnu inv-jishnu self-assigned this Dec 19, 2024
@ypeckstadt ypeckstadt marked this pull request as ready for review December 19, 2024 23:48
@ypeckstadt ypeckstadt mentioned this pull request Dec 19, 2024
6 tasks

/** The namespace of the table in ScalarDB. */
@JsonProperty("namespace")
private String namespace;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] This seems the name of a namespace as well as the name of a table below. So, this should be namespaceName for consistency?

@Getter
public class TableMetadataRequest {

private final String namespace;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] Same as above

*
* @return A set of predefined metadata column names.
*/
public static Set<String> getMetadataColumns() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using com.scalar.db.transaction.consensuscommit.ConsensusCommitUtils#getTransactionMetaColumns to avoid duplicated logic?

Some other methods might be replaced with ConsensusCommitUtils methods.

Copy link
Collaborator

@brfrn169 brfrn169 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple comments. Please take a look when you have time!

* @param columnNames A set of all column names in the table.
* @return {@code true} if the column is a metadata column; {@code false} otherwise.
*/
public static boolean isMetadataColumn(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use ConsensusCommitUtils.isTransactionMetaColumn():

public static boolean isTransactionMetaColumn(String columnName, TableMetadata tableMetadata) {

* @param tableMetadata The metadata of the table.
* @return {@code true} if the column is a metadata column; {@code false} otherwise.
*/
public static boolean isMetadataColumn(String columnName, TableMetadata tableMetadata) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants