Skip to content

Commit

Permalink
238: Implement S3 Sinks, part 1 (no S3 sinks yet implemented) (#252)
Browse files Browse the repository at this point in the history
- Updates acquire-common to latest `main`
- Accounts for redefinition of `VideoFrame::bytes_of_frame` for
8-aligned `VideoFrame` pointers
- Removes `SinkCreator` and `SinkType` concepts
- Replaces `FileCreator` with `SinkCreator` object
- Replaces bare pointers to `Sink` with shared pointers thereto
- Factors code common to creating both chunk and shard `Sink`s 
- Creates metadata `Sink`s directly in the Zarr object
- Adds test/README.md
  • Loading branch information
aliddell authored Jul 2, 2024
1 parent 6342d62 commit 0b6065b
Show file tree
Hide file tree
Showing 23 changed files with 689 additions and 668 deletions.
2 changes: 2 additions & 0 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ add_library(${tgt} MODULE
common.hh
common.cpp
writers/sink.hh
writers/sink.creator.hh
writers/sink.creator.cpp
writers/file.sink.hh
writers/file.sink.cpp
writers/writer.hh
Expand Down
21 changes: 5 additions & 16 deletions src/common.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -224,20 +224,9 @@ common::sample_type_to_string(SampleType t) noexcept
}
}

void
common::write_string(const std::string& path, const std::string& str)
{
if (auto p = fs::path(path); !fs::exists(p.parent_path()))
fs::create_directories(p.parent_path());

struct file f = { 0 };
auto is_ok = file_create(&f, path.c_str(), path.size());
is_ok &= file_write(&f, // file
0, // offset
(uint8_t*)str.c_str(), // cur
(uint8_t*)(str.c_str() + str.size()) // end
);
EXPECT(is_ok, "Write to \"%s\" failed.", path.c_str());
TRACE("Wrote %d bytes to \"%s\".", str.size(), path.c_str());
file_close(&f);
size_t
common::align_up(size_t n, size_t align)
{
EXPECT(align > 0, "Alignment must be greater than zero.");
return align * ((n + align - 1) / align);
}
11 changes: 6 additions & 5 deletions src/common.hh
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,12 @@ sample_type_to_dtype(SampleType t);
const char*
sample_type_to_string(SampleType t) noexcept;

/// @brief Write a string to a file.
/// @param path The path of the file to write.
/// @param str The string to write.
void
write_string(const std::string& path, const std::string& value);
/// @brief Align a size to a given alignment.
/// @param n Size to align.
/// @param align Alignment.
/// @return Aligned size.
size_t
align_up(size_t n, size_t align);
} // namespace acquire::sink::zarr::common
} // namespace acquire::sink::zarr

Expand Down
Loading

0 comments on commit 0b6065b

Please sign in to comment.