Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(EntityService): batched transactions and ebean updates #8456

Merged
merged 43 commits into from
Sep 3, 2023
Merged

feat(EntityService): batched transactions and ebean updates #8456

merged 43 commits into from
Sep 3, 2023

Commits on Jul 17, 2023

  1. refactor(rentention): refactor retention

    @david-leifker
    david-leifker committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    0e37907 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. feat(sql): ebean transaction batches

    @david-leifker
    david-leifker committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    70c5b5e View commit details
    Browse the repository at this point in the history

  2. lint

    @david-leifker
    david-leifker committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    3f13b99 View commit details
    Browse the repository at this point in the history

  3. fixing tests

    @david-leifker
    david-leifker committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    496362d View commit details
    Browse the repository at this point in the history

  4. fix-up patch

    @david-leifker
    david-leifker committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    dd55590 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2023

  1. EntityService refactor

    @david-leifker
    david-leifker committed Jul 22, 2023
    Configuration menu
    Copy the full SHA
    e65d540 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into ebean-batched-trans… 

    …actions

# Conflicts:
#	metadata-io/src/main/java/com/linkedin/metadata/entity/EntityServiceImpl.java
#	metadata-io/src/main/java/com/linkedin/metadata/entity/EntityUtils.java
#	metadata-io/src/test/java/com/linkedin/metadata/entity/EbeanEntityServiceTest.java
#	metadata-io/src/test/java/com/linkedin/metadata/entity/EntityServiceTest.java
#	metadata-service/openapi-servlet/src/main/java/io/datahubproject/openapi/util/MappingUtil.java
#	metadata-service/restli-servlet-impl/src/main/java/com/linkedin/metadata/resources/entity/AspectResource.java
#	metadata-service/restli-servlet-impl/src/test/java/com/linkedin/metadata/resources/entity/AspectResourceTest.java
#	metadata-service/services/src/main/java/com/linkedin/metadata/entity/DeleteEntityService.java
    @david-leifker
    david-leifker committed Jul 22, 2023
    Configuration menu
    Copy the full SHA
    e795eca View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Fix merge issues

    @david-leifker
    david-leifker committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f87b74e View commit details
    Browse the repository at this point in the history

  2. Complete futures for async produce

    @david-leifker
    david-leifker committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    fc4e48b View commit details
    Browse the repository at this point in the history

  3. Add metrics around exceptions, skip retry on key aspects

    @david-leifker
    david-leifker committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    85b51e4 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/master' into ebean-batched-trans… 

    …actions

# Conflicts:
#	metadata-service/factories/src/test/java/com/linkedin/metadata/boot/steps/RestoreGlossaryIndicesTest.java
    @david-leifker
    david-leifker committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    c29a206 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Use fully qualified path for docker task lookup

    @david-leifker
    david-leifker committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    843cf06 View commit details
    Browse the repository at this point in the history

  2. Avoid pulling mae/mce consumers in smoke-tests (not used)

    @david-leifker
    david-leifker committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    d19c533 View commit details
    Browse the repository at this point in the history

  3. try non-standalone

    @david-leifker
    david-leifker committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    f448e0e View commit details
    Browse the repository at this point in the history

  4. drop kafka-setup

    @david-leifker
    david-leifker committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    38f2ba8 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. kafka is required

    @david-leifker
    david-leifker committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    3122414 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. disable ES threshold

    @david-leifker
    david-leifker committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    da47ea1 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. Merge remote-tracking branch 'datahub-project/master' into ebean-batc… 

    …hed-transactions

# Conflicts:
#	.github/workflows/docker-unified.yml
#	metadata-io/src/main/java/com/linkedin/metadata/entity/EntityServiceImpl.java
    @david-leifker
    david-leifker committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    fcf883b View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. fix(test): increase siblings.js test stability

    @david-leifker
    david-leifker committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    846d95b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into siblings-cypress-fix

    @david-leifker
    david-leifker authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    05c9dad View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/siblings-cypress-fix' into ebean… 

    …-batched-transactions

# Conflicts:
#	metadata-io/src/main/java/com/linkedin/metadata/entity/AspectDao.java
#	metadata-io/src/main/java/com/linkedin/metadata/entity/EntityServiceImpl.java
#	smoke-test/tests/cypress/cypress/e2e/siblings/siblings.js
    @david-leifker
    david-leifker committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    2e6f377 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    abcfae0 View commit details
    Browse the repository at this point in the history

  5. lint

    @david-leifker
    david-leifker committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    9ef881b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. drop mce/mae consumers, space issues

    @david-leifker
    david-leifker committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    1fe33b6 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. add multi-threading test

    @david-leifker
    david-leifker committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    bd752be View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into ebean-batched-trans… 

    …actions
    @david-leifker
    david-leifker committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    4c5fbae View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/master' into ebean-batched-trans… 

    …actions

# Conflicts:
#	datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/mutate/util/DeleteUtils.java
#	datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/mutate/util/DeprecationUtils.java
#	datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/mutate/util/DomainUtils.java
#	datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/mutate/util/LabelUtils.java
#	datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/mutate/util/OwnerUtils.java
#	metadata-io/src/main/java/com/linkedin/metadata/entity/EntityUtils.java
#	metadata-io/src/main/java/com/linkedin/metadata/entity/ebean/EbeanAspectDao.java
    @david-leifker
    david-leifker committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    95cbe64 View commit details
    Browse the repository at this point in the history

  4. fix merge, ebean upgrade

    @david-leifker
    david-leifker committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    8b66642 View commit details
    Browse the repository at this point in the history

  5. Improve tests 

    Improve tx handling
Prevent deadlock dropped writes
    @david-leifker
    david-leifker committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    2f6da4c View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 11, 2023
    Configuration menu
    Copy the full SHA
    9ba652e View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2023

  1. Fix deserialization 

    Fix lint
    @david-leifker
    david-leifker committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    12eb77e View commit details
    Browse the repository at this point in the history

  2. only add additional mcp for upsert/create ops

    @david-leifker
    david-leifker committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    01e489b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 12, 2023
    Configuration menu
    Copy the full SHA
    92a3cf1 View commit details
    Browse the repository at this point in the history

  4. fix AspectUtils test from patch to upsert

    @david-leifker
    david-leifker committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    028f3aa View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Merge remote-tracking branch 'origin/master' into ebean-batched-trans… 

    …actions
    @david-leifker
    david-leifker committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    714d67d View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2023

  1. temp fix incorrect head tag

    @david-leifker
    david-leifker committed Aug 19, 2023
    Configuration menu
    Copy the full SHA
    cb98802 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 22, 2023
    Configuration menu
    Copy the full SHA
    96a3e56 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2023

  1. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 25, 2023
    Configuration menu
    Copy the full SHA
    c6822c5 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2023

  1. Merge branch 'ebean-batched-transactions' of github.com:david-leifker… 

    …/datahub into ebean-batched-transactions
    @david-leifker
    david-leifker committed Aug 26, 2023
    Configuration menu
    Copy the full SHA
    7ff96a6 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. generate default aspects even on patch

    @david-leifker
    david-leifker committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    7ff7abb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Aug 29, 2023
    Configuration menu
    Copy the full SHA
    738c19b View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2023

  1. fix patch with default aspects

    @david-leifker
    david-leifker committed Sep 2, 2023
    Configuration menu
    Copy the full SHA
    7f66861 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into ebean-batched-transactions

    @david-leifker
    david-leifker authored Sep 2, 2023
    Configuration menu
    Copy the full SHA
    8428c90 View commit details
    Browse the repository at this point in the history