Skip to content

Commit

Permalink
[Doc] Documentation for FDB.h
Browse files Browse the repository at this point in the history
Added some documentation for the C++-API of the FDB class.
There are still some open questions linked to some of the functions.

This is a first draft.
  • Loading branch information
tbkr committed Oct 29, 2024
1 parent 72fd6af commit 497a12e
Showing 1 changed file with 182 additions and 7 deletions.
189 changes: 182 additions & 7 deletions src/fdb5/api/FDB.h
Original file line number Diff line number Diff line change
Expand Up @@ -71,72 +71,247 @@ class FDB {

// -------------- Primary API functions ----------------------------

/** @brief Archive binary data to a FDB.
*
* @param handle eckit::message::Message to data to archive
*/
void archive(eckit::message::Message msg);

/** @brief Archive binary data to a FDB.
*
* Reads messages from the eckit#DataHandle and calls archive() the the
* corresponding messages.
*
* @param handle eckit::DataHandle reference data to archive
*/
void archive(eckit::DataHandle& handle);

/** @brief Archive binary data to a FDB .
*
* Internally creates a DataHandle and calls archive().
*
* @param data Pointer to the binary data to archive
* @param length Size of the data to archive with the given
*/
void archive(const void* data, size_t length);
// warning: not high-perf API - makes sure that all the requested fields are archived and there are no data exceeding the request

/** @brief Archive binary data of a mars request and the corresponding binary data to a FDB.
*
* @warning: not high-perf API - makes sure that all the requested fields are archived and there are no data exceeding the request
*
* @param request, a mars request
* @param handle, a data handle pointing to the data
*/
void archive(const metkit::mars::MarsRequest& request, eckit::DataHandle& handle);

// disclaimer: this is a low-level API. The provided key and the corresponding data are not checked for consistency
// Optional callback function is called upon receiving field location from the store.
/** @brief Archive binary data to a FDB.
*
* @warning this is a low-level API. The provided key and the corresponding data are not checked for consistency
* Optional callback function is called upon receiving field location from the store.
*
* @param key Key used for indexing and archiving the data
* @param data Pointer to the binary data to archive
* @param length Size of the data to archive with the given @p key
*/
void archive(const Key& key, const void* data, size_t length);

/// Flushes all buffers and closes all data handles into a consistent DB state
/// @note always safe to call
/** @brief Flush all buffers and closes all data handles into a consistent DB state
*
* @note always safe to call
*/
void flush();

/** @brief Read binary data from an URI
*
* @param uri, eckit uri to the data source
*
* @return DataHandle for reading the requested data from
*/
eckit::DataHandle* read(const eckit::URI& uri);

/** @brief Read binary data from an list of URI
*
* @param vector of uris, eckit uris to the data source
* @param sorted, bool whether the given uris should be sorted
*
* @return DataHandle for reading the requested data from
*/
eckit::DataHandle* read(const std::vector<eckit::URI>& uris, bool sorted = false);

/** @brief Read binary from a ListIterator
*
* @param a list iterator, which resembles a set of fields, which should be read
* @param sorted, bool whether the given uris should be sorted
*
* @return DataHandle for reading the requested data from
*/
eckit::DataHandle* read(ListIterator& it, bool sorted = false);

/** @brief Retrieve data which is specified by a MARS request
*
* @param request MarsRequest which describes the data which should be retrieved
*
* @return DataHandle for reading the requested data from
*/
eckit::DataHandle* retrieve(const metkit::mars::MarsRequest& request);

/**
* @TODO: Comment me pls.
*
* @param request, a mars request which should be inspected
*
* @return ListIterator
*/
ListIterator inspect(const metkit::mars::MarsRequest& request);

/** @brief List data present at the archive and which can be retrieved
*
* @param request FDBToolRequest stating which data should be queried
* @param deduplicate bool, whether the returned iterator should ignore duplicates
* @param length Size of the data to archive with the given @p key
*
* @return ListIterator for iterating over the set of found items
*/
ListIterator list(const FDBToolRequest& request, bool deduplicate=false);

/** @brief Dump the structural content of the FDB
*
* In particular, in the TOC formulation, enumerate the different entries
* in the Table of Contents (including INIT and CLEAR entries).
* The dump will include information identifying the data files that are
* referenced, and the "Axes" which describe the maximum possible extent of
* the data that is contained in the database.
*
* @return DumpIterator for iterating over the set of found items
*/
DumpIterator dump(const FDBToolRequest& request, bool simple=false);

/// TODO: Is this function superfluous given the control() function?
StatusIterator status(const FDBToolRequest& request);

/** @brief Wipe data from the database
*
* @param request FDBToolRequest stating which data should be queried
* @param doit flag for committing to the wipe (default is dry-run)
* @param porcelain flag print only a list of files to be deleted / that are deleted
* @param unsafeWipeAll flag for omitting all security checks and force a wipe
*
* @return WipeIterator for iterating over the set of wiped items
*/
WipeIterator wipe(const FDBToolRequest& request, bool doit=false, bool porcelain=false, bool unsafeWipeAll=false);

/** @brief Move content of one FDB database.
*
* This locks the source database, make it possible to create a second
* database in another root, duplicates all data. Source data are not automatically removed.
*
* @param request, a fdb tool request for the data which should be move
* @param uri, destination to which the data should be moved
*
* @return MoveIterator for iterating over the set of found items
*/
MoveIterator move(const FDBToolRequest& request, const eckit::URI& dest);

/** @brief Remove duplicate data from the database
*
* Purge duplicate entries from the database and remove the associated data
* (if the data is owned, not adopted).
* Data in the FDB5 is immutable. It is masked, but not damaged or deleted,
* when it is overwritten with new data using the same key.
* If an index or data file only contains masked data
* (i.e. no data that can be obtained using normal requests),
* then those indexes and data files may be removed.
* If an index refers to data that is not owned by the FDB
* (in particular data which has been adopted from an existing FDB4),
* this data will not be removed.
*
* @param request, a fdb tool request for the data which should be purged
* @param doit, bool if true the purge is triggered, otherwise a dry-run is executed
* @param porcelain, bool for printing only those files which are deleted
*
* @return PurgeIterator for iterating over the set of found items
*/
PurgeIterator purge(const FDBToolRequest& request, bool doit=false, bool porcelain=false);

/** @brief Prints information about FDB databases, aggregating the information over all the databases visited into a final summary.
*
* @param request, FDB tool request for which the stats should be shown
*
* @return StatsIterator for iterating over the set of found items
*/
StatsIterator stats(const FDBToolRequest& request);


/** @brief
* TODO: Comment me pls
* @param request, FDB tool request
* @param action, control action
* @param identifiers, identifiers
*
* @return ControlIterator for iterating over the set of found items
*/
ControlIterator control(const FDBToolRequest& request,
ControlAction action,
ControlIdentifiers identifiers);

/** @brief
* TODO: Comment me pls
*
* @param request, FDB tool request
* @param level, maximum level the axis visitor should respect
*
* @return IndexAxis
*/
IndexAxis axes(const FDBToolRequest& request, int level=3);

/** @brief
* TODO: Comment me pls
* @param request, FDB tool request
* @param level, maximum level the axis visitor should respect
*
* @return AxisIterator
*/
AxesIterator axesIterator(const FDBToolRequest& request, int level=3);

/** @brief Check whether a specific control identifier is enabled
*
* @param controlIdentifier, a given control identifier
*
* @return bool, true or false, depending on the internal status of the FDB
*/
bool enabled(const ControlIdentifier& controlIdentifier) const;

/** @brief Return whether a flush of the FDB is needed
*
* @return true, if an archive has happened and a flush is needed
*/
bool dirty() const;

/** @brief Register an archive callback
*
* @param callback, an archive callback which should be triggered during archive
*/
void registerArchiveCallback(ArchiveCallback callback);

/** @brief Register a flush callback
*
* @param callback, an flush callback which should be triggered during flushing
*/
void registerFlushCallback(FlushCallback callback);

// -------------- API management ----------------------------

/// ID used for hashing in the Rendezvous hash. Should be unique.
const std::string id() const;

FDBStats stats() const;

FDBStats internalStats() const;

const std::string& name() const;

const Config& config() const;

void disable();

bool disabled() const;

private: // methods
Expand Down

0 comments on commit 497a12e

Please sign in to comment.