Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug in InputSplitBase::Chunk::Append() ? #648

Open
ztma opened this issue Apr 29, 2021 · 0 comments
Open

Bug in InputSplitBase::Chunk::Append() ? #648

ztma opened this issue Apr 29, 2021 · 0 comments

Comments

@ztma
Copy link

ztma commented Apr 29, 2021

When split->ReadChunk(...) returns a true value and size is set to zero, the code double the buffer size, but in the next loop, variable 'size' should also be adjust accordingly before call split->ReadChunk again.

bool InputSplitBase::Chunk::Append(InputSplitBase *split, size_t buffer_size) {
  size_t previous_size = end - begin;
  data.resize(data.size() + buffer_size);
  while (true) {
    // leave one tail chunk
    size_t size = buffer_size * sizeof(uint32_t);
    // set back to 0 for string safety
    data.back() = 0;
    if (!split->ReadChunk(reinterpret_cast<char *>(BeginPtr(data)) + previous_size, &size))
      return false;
    if (size == 0) {
      data.resize(data.size() * 2);
    } else {
      begin = reinterpret_cast<char *>(BeginPtr(data));
      end = begin + previous_size + size;
      break;
    }
  }
  return true;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant