Code

Composite directory poc#1

Code

Closed

nisgoel-amazon wants to merge 14 commits intomainrayshrey/OpenSearch:mainfrom composite-directory-pocrayshrey/OpenSearch:composite-directory-pocCopy head branch name to clipboard

+1,339-184

Commits

Commits on May 9, 2024

• Composite Directory POC

  Show description for c5b021b
  rayshrey

  committed
  c5b021b

  Copy full SHA for c5b021b

• Refactor TransferManager interface to RemoteStoreFileTrackerAdapter

  Show description for 46a63d5
  rayshrey

  committed
  46a63d5

  Copy full SHA for 46a63d5

• Implement block level fetch for Composite Directory

  Show description for 35f6f1e
  rayshrey

  committed
  35f6f1e

  Copy full SHA for 35f6f1e

• Removed CACHE state from FileTracker

  Show description for ec27fbe
  rayshrey

  committed
  ec27fbe

  Copy full SHA for ec27fbe

• Fixes after latest pull

  Show description for f93655d
  rayshrey

  committed
  f93655d

  Copy full SHA for f93655d

• Add new setting for warm, remove store type setting, FileTracker and RemoteStoreFileTrackerAdapter, CompositeDirectoryFactory and update Composite Directory implementation

  Show description for 04bad8a
  rayshrey

  committed
  04bad8a

  Copy full SHA for 04bad8a

• Modify TransferManager - replace BlobContainer with Functional Interface to fetch an InputStream instead

  Show description for d09b75a
  rayshrey

  committed
  d09b75a

  Copy full SHA for d09b75a

• Reuse OnDemandBlockSnapshotIndexInput instead of OnDemandBlockCompositeIndexInput

  Show description for 90ace4a
  rayshrey

  committed
  90ace4a

  Copy full SHA for 90ace4a

• Modify constructors to avoid breaking public api contract and code review fixes

  Show description for 93d1f2a
  rayshrey

  committed
  93d1f2a

  Copy full SHA for 93d1f2a

• Add experimental annotations for newly created classes and review comment fixes

  Show description for 3280a6d
  rayshrey

  committed
  3280a6d

  Copy full SHA for 3280a6d

• Use ref count as a temporary measure to prevent file from eviction until uploaded to Remote

  Show description for 4ee744d
  rayshrey

  committed
  4ee744d

  Copy full SHA for 4ee744d

Commits on May 10, 2024

• Remove method level locks

  Show description for 3292b36
  rayshrey

  committed
  3292b36

  Copy full SHA for 3292b36

• Handle tmp file deletion

  Show description for 5013be9
  rayshrey

  committed
  5013be9

  Copy full SHA for 5013be9

Commits on May 13, 2024

• Nit fixes

  Show description for 7491c62
  rayshrey

  committed
  7491c62

  Copy full SHA for 7491c62