-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Introduced basic_storage_backend interface (#33)
Refactored 'binsrv::filesystem_storage' class. Instead, we now have a new class
named 'binsrv::storage', that incorporate all the binlog storage logic
(iterating over existing objects and initialisation, analysing the content of
binlog index, updating binlog index, opening/closing event stream and writing
event data). It now uses 'binsrv::basic_storage_backend' abstract interface to
perform backend-specific operations.
Added one concrete 'binsrv::filesystem_storage_backend' class inherited from the
'binsrv::basic_storage_backend' abstract class that implements the interface
using the local filesystem storage (a dedicated directory on a filesystem).
Another planned implementation is an "AWS S3"-based storage backend (not
included in this commit).
'binsrv::master_config' class renamed to 'binsrv::main_config'.
'master_config.json' sample configuration file renamed into 'main_config.json'.
'binsrv::storage_config' class extended with additional data field "type" which
is supposed to determine the type of the backend storage to be used.
'main_config.json' and 'binlog_streaming.binsrv' MTR test case updated
correspondingly.
Introduced 'binsrv::storage_backend_factory' class that based on the value of
the 'type' field in the 'binsrv::storage_config' will return concrete
implementation of the 'binsrv::basic_storage_backend' interface. Currently,
only "fs" value is supported.
Revisited exception types thrown via
'util::exception_location().raise<xxx>(...)' helper:
* std::invalid_argument - one of the provided argument is incorrect;
* std::logic_error - invariant / pre-condition / post-condition violation;
* std::runtime_error - an error related to hardware, resource unavailability,
etc. where simply repeating the call may resolve the
problem (like IO errors);
* std::out_of_range - size/limits-related errors.- Loading branch information
1 parent
b9aa9f8
commit 997ed77
Showing
26 changed files
with
616 additions
and
309 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ | |
| "password": "" | ||
| }, | ||
| "storage": { | ||
| "type": "fs", | ||
| "path": "./storage" | ||
| } | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| #include "binsrv/basic_storage_backend.hpp" | ||
|
|
||
| #include <stdexcept> | ||
| #include <string> | ||
| #include <string_view> | ||
|
|
||
| #include "util/byte_span_fwd.hpp" | ||
| #include "util/exception_location_helpers.hpp" | ||
|
|
||
| namespace binsrv { | ||
|
|
||
| [[nodiscard]] storage_object_name_container | ||
| basic_storage_backend::list_objects() { | ||
| return do_list_objects(); | ||
| } | ||
|
|
||
| [[nodiscard]] std::string | ||
| basic_storage_backend::get_object(std::string_view name) { | ||
| return do_get_object(name); | ||
| } | ||
|
|
||
| void basic_storage_backend::put_object(std::string_view name, | ||
| util::const_byte_span content) { | ||
| return do_put_object(name, content); | ||
| } | ||
|
|
||
| void basic_storage_backend::open_stream(std::string_view name) { | ||
| if (stream_opened_) { | ||
| util::exception_location().raise<std::logic_error>( | ||
| "cannot open a new stream as the previous one has not been closed"); | ||
| } | ||
|
|
||
| do_open_stream(name); | ||
| stream_opened_ = true; | ||
| } | ||
|
|
||
| void basic_storage_backend::write_data_to_stream(util::const_byte_span data) { | ||
| if (!stream_opened_) { | ||
| util::exception_location().raise<std::logic_error>( | ||
| "cannot write to the stream as it has not been opened"); | ||
| } | ||
| do_write_data_to_stream(data); | ||
| } | ||
|
|
||
| void basic_storage_backend::close_stream() { | ||
| if (!stream_opened_) { | ||
| util::exception_location().raise<std::logic_error>( | ||
| "cannot close the stream as it has not been opened"); | ||
| } | ||
| do_close_stream(); | ||
| stream_opened_ = false; | ||
| } | ||
|
|
||
| } // namespace binsrv |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #ifndef BINSRV_BASIC_STORAGE_BACKEND_HPP | ||
| #define BINSRV_BASIC_STORAGE_BACKEND_HPP | ||
|
|
||
| #include "binsrv/basic_storage_backend_fwd.hpp" // IWYU pragma: export | ||
|
|
||
| #include <string> | ||
| #include <string_view> | ||
|
|
||
| #include "util/byte_span_fwd.hpp" | ||
|
|
||
| namespace binsrv { | ||
|
|
||
| class basic_storage_backend { | ||
| public: | ||
| basic_storage_backend() = default; | ||
| basic_storage_backend(const basic_storage_backend &) = delete; | ||
| basic_storage_backend(basic_storage_backend &&) noexcept = delete; | ||
| basic_storage_backend &operator=(const basic_storage_backend &) = delete; | ||
| basic_storage_backend &operator=(basic_storage_backend &&) = delete; | ||
|
|
||
| virtual ~basic_storage_backend() = default; | ||
|
|
||
| [[nodiscard]] storage_object_name_container list_objects(); | ||
| [[nodiscard]] std::string get_object(std::string_view name); | ||
| void put_object(std::string_view name, util::const_byte_span content); | ||
|
|
||
| void open_stream(std::string_view name); | ||
| void write_data_to_stream(util::const_byte_span data); | ||
| void close_stream(); | ||
|
|
||
| private: | ||
| bool stream_opened_{false}; | ||
|
|
||
| [[nodiscard]] virtual storage_object_name_container do_list_objects() = 0; | ||
| [[nodiscard]] virtual std::string do_get_object(std::string_view name) = 0; | ||
| virtual void do_put_object(std::string_view name, | ||
| util::const_byte_span content) = 0; | ||
|
|
||
| virtual void do_open_stream(std::string_view name) = 0; | ||
| virtual void do_write_data_to_stream(util::const_byte_span data) = 0; | ||
| virtual void do_close_stream() = 0; | ||
| }; | ||
|
|
||
| } // namespace binsrv | ||
|
|
||
| #endif // BINSRV_BASIC_STORAGE_BACKEND_HPP |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| #ifndef BINSRV_BASIC_STORAGE_BACKEND_FWD_HPP | ||
| #define BINSRV_BASIC_STORAGE_BACKEND_FWD_HPP | ||
|
|
||
| #include <cstdint> | ||
| #include <memory> | ||
| #include <string> | ||
| #include <unordered_map> | ||
|
|
||
| namespace binsrv { | ||
|
|
||
| class basic_storage_backend; | ||
|
|
||
| using basic_storage_backend_ptr = std::unique_ptr<basic_storage_backend>; | ||
|
|
||
| // the following hash calculation helper allows to look for keys represented | ||
| // by either const char*, std::string_view of const std::string in unordered | ||
| // containers with std::string key transparently without converting the value | ||
| // being searched to std::string | ||
| struct transparent_string_like_hash { | ||
| using is_transparent = void; | ||
| std::size_t operator()(const char *key) const noexcept { | ||
| return std::hash<std::string_view>{}(key); | ||
| } | ||
| std::size_t operator()(std::string_view key) const noexcept { | ||
| return std::hash<std::string_view>{}(key); | ||
| } | ||
| std::size_t operator()(const std::string &key) const noexcept { | ||
| return std::hash<std::string>{}(key); | ||
| } | ||
| }; | ||
| // the container thatr uses transparent_string_like_hash also needs a | ||
| // transparent version of KeyEqual template argument (std::equal_to<void>) | ||
| using storage_object_name_container = | ||
| std::unordered_map<std::string, std::uint64_t, transparent_string_like_hash, | ||
| std::equal_to<>>; | ||
| } // namespace binsrv | ||
|
|
||
| #endif // BINSRV_BASIC_STORAGE_BACKEND_FWD_HPP |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.