Code

Multireaderfilestream Redesign#4595

Code

Merged

percyliang merged 22 commits intomastercodalab/codalab-worksheets:masterfrom multireaderfilestream-redesigncodalab/codalab-worksheets:multireaderfilestream-redesignCopy head branch name to clipboardNov 3, 2024

+281-34

Commits

Commits on May 15, 2024

• Initial Modifications, adding min-max heap to track min and max reader performantly

  dma1dma1

  committed
  a6796a9

Commits on May 16, 2024

• Change update function

  dma1dma1

  committed
  17449a4

• Read done

  dma1dma1

  committed
  6de3afb

• Some seek

  dma1dma1

  committed
  0b109c2

• Fixed race condition

  dma1dma1

  committed
  9490a01

• Implementation Done

  dma1dma1

  committed
  035ba30

Commits on May 21, 2024

• Added tests for tar gz

  dma1dma1

  committed
  1c73adc

Commits on May 23, 2024

• Fixed tar.gz test, added memory test

  dma1dma1

  committed
  8450e30

• Added memory profiler

  dma1dma1

  committed
  2e5e6ce

• Small test changes

  dma1dma1

  committed
  28d642f

• Modifying Chunk size

  dma1dma1

  committed
  f491e0e

Commits on May 27, 2024

• Using getmembers to get tarinfo

  dma1dma1

  committed
  8b01add

• Added multireaderfilestream tests

  dma1dma1

  committed
  dedf6ca

• Cleanup

  dma1dma1

  committed
  04b524b

Commits on May 28, 2024

• Test fixes

  dma1dma1

  committed
  7f33c66

• Changes requested

  dma1dma1

  committed
  8d2a3d6

• Changes for clarity

  dma1dma1

  committed
  adf0f9e

• More changes for clarity

  dma1dma1

  committed
  fae4335

Commits on May 30, 2024

• Better code structure + comments

  dma1dma1

  committed
  9571b30

Commits on May 31, 2024

• Even more changes for clarity

  dma1dma1

  committed
  c3fd7ae

• Lower iterations per disk check to match chunk size

  dma1dma1

  committed
  7984f8a

• Change names

  dma1dma1

  committed
  3e584df