Skip to content

Commit

Permalink
Code cleanup (netdata#17237)
Browse files Browse the repository at this point in the history 
* renames in dbengine

* remove leftovers from memory mode save and map

* fix docs about 3 tiers by default

* split linked-lists, bitmaps and storage-points from libnetdata.h
  • Loading branch information
@ktsaou
ktsaou authored Mar 23, 2024
1 parent e910ccc commit 00f897a
Show file tree
Hide file tree
Showing 19 changed files with 447 additions and 434 deletions.
3 changes: 3 additions & 0 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,9 @@ set(LIBNETDATA_FILES
src/libnetdata/dictionary/dictionary-hashtable.h
src/libnetdata/dictionary/dictionary-item.h
src/libnetdata/dictionary/dictionary-callbacks.h
src/libnetdata/linked-lists.h
src/libnetdata/storage-point.h
src/libnetdata/bitmap.h
)

if(ENABLE_PLUGIN_EBPF)
Expand Down
2 changes: 0 additions & 2 deletions src/collectors/cgroups.plugin/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,6 @@ a few errors in error.log complaining about files it cannot find, but immediatel
5. Existing dashboard sessions will continue to see them, but of course they will not refresh
6. Obsolete charts will be removed from memory, 1 hour after the last user viewed them (configurable
with `[global].cleanup obsolete charts after seconds = 3600` (at `netdata.conf`).
7. When obsolete charts are removed from memory they are also deleted from disk (configurable
with `[global].delete obsolete charts files = yes`)

### Monitored container metrics

Expand Down
4 changes: 1 addition & 3 deletions src/daemon/config/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Please note that your data history will be lost if you have modified `history` p
|:---------------------------------------------:|:----------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| mode | `dbengine` | `dbengine`: The default for long-term metrics storage with efficient RAM and disk usage. Can be extended with `dbengine page cache size MB` and `dbengine disk space MB`. <br />`ram`: The round-robin database will be temporary and it will be lost when Netdata exits. <br />`alloc`: Similar to `ram`, but can significantly reduce memory usage, when combined with a low retention and does not support KSM. <br />`none`: Disables the database at this host, and disables health monitoring entirely, as that requires a database of metrics. Not to be used together with streaming. |
| retention | `3600` | Used with `mode = ram/alloc`, not the default `mode = dbengine`. This number reflects the number of entries the `netdata` daemon will by default keep in memory for each chart dimension. Check [Memory Requirements](https://github.com/netdata/netdata/blob/master/src/database/README.md) for more information. |
| storage tiers | `1` | The number of storage tiers you want to have in your dbengine. Check the tiering mechanism in the [dbengine's reference](https://github.com/netdata/netdata/blob/master/src/database/engine/README.md#tiering). You can have up to 5 tiers of data (including the _Tier 0_). This number ranges between 1 and 5. |
| storage tiers | `3` | The number of storage tiers you want to have in your dbengine. Check the tiering mechanism in the [dbengine's reference](https://github.com/netdata/netdata/blob/master/src/database/engine/README.md#tiering). You can have up to 5 tiers of data (including the _Tier 0_). This number ranges between 1 and 5. |
| dbengine page cache size MB | `32` | Determines the amount of RAM in MiB that is dedicated to caching for _Tier 0_ Netdata metric values. |
| dbengine tier **`N`** page cache size MB | `32` | Determines the amount of RAM in MiB that is dedicated for caching Netdata metric values of the **`N`** tier. <br /> `N belongs to [1..4]` |
| dbengine disk space MB | `256` | Determines the amount of disk space in MiB that is dedicated to storing _Tier 0_ Netdata metric values and all related metadata describing them. This option is available **only for legacy configuration** (`Agent v1.23.2 and prior`). |
Expand All @@ -103,8 +103,6 @@ Please note that your data history will be lost if you have modified `history` p
| cleanup obsolete charts after secs | `3600` | See [monitoring ephemeral containers](https://github.com/netdata/netdata/blob/master/src/collectors/cgroups.plugin/README.md#monitoring-ephemeral-containers), also sets the timeout for cleaning up obsolete dimensions |
| gap when lost iterations above | `1` | |
| cleanup orphan hosts after secs | `3600` | How long to wait until automatically removing from the DB a remote Netdata host (child) that is no longer sending data. |
| delete obsolete charts files | `yes` | See [monitoring ephemeral containers](https://github.com/netdata/netdata/blob/master/src/collectors/cgroups.plugin/README.md#monitoring-ephemeral-containers), also affects the deletion of files for obsolete dimensions |
| delete orphan hosts files | `yes` | Set to `no` to disable non-responsive host removal. |
| enable zero metrics | `no` | Set to `yes` to show charts when all their metrics are zero. |

> ### Info
Expand Down
4 changes: 2 additions & 2 deletions src/daemon/global_statistics.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,7 @@ static void global_statistics_charts(void) {
// ----------------------------------------------------------------

#ifdef ENABLE_DBENGINE
if (tier_page_type[0] == PAGE_GORILLA_METRICS)
if (tier_page_type[0] == RRDENG_PAGE_TYPE_GORILLA_32BIT)
{
static RRDSET *st_tier0_gorilla_pages = NULL;
static RRDDIM *rd_num_gorilla_pages = NULL;
Expand Down Expand Up @@ -887,7 +887,7 @@ static void global_statistics_charts(void) {
rrdset_done(st_tier0_gorilla_pages);
}

if (tier_page_type[0] == PAGE_GORILLA_METRICS)
if (tier_page_type[0] == RRDENG_PAGE_TYPE_GORILLA_32BIT)
{
static RRDSET *st_tier0_compression_info = NULL;

Expand Down
12 changes: 3 additions & 9 deletions src/daemon/main.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1054,12 +1054,6 @@ static void backwards_compatible_config() {
config_move(CONFIG_SECTION_GLOBAL, "cleanup orphan hosts after seconds",
CONFIG_SECTION_DB, "cleanup orphan hosts after secs");

config_move(CONFIG_SECTION_GLOBAL, "delete obsolete charts files",
CONFIG_SECTION_DB, "delete obsolete charts files");

config_move(CONFIG_SECTION_GLOBAL, "delete orphan hosts files",
CONFIG_SECTION_DB, "delete orphan hosts files");

config_move(CONFIG_SECTION_GLOBAL, "enable zero metrics",
CONFIG_SECTION_DB, "enable zero metrics");

Expand Down Expand Up @@ -1171,11 +1165,11 @@ static void get_netdata_configured_variables() {

const char *page_type = config_get(CONFIG_SECTION_DB, "dbengine page type", "gorilla");
if (strcmp(page_type, "gorilla") == 0)
tier_page_type[0] = PAGE_GORILLA_METRICS;
tier_page_type[0] = RRDENG_PAGE_TYPE_GORILLA_32BIT;
else if (strcmp(page_type, "raw") == 0)
tier_page_type[0] = PAGE_METRICS;
tier_page_type[0] = RRDENG_PAGE_TYPE_ARRAY_32BIT;
else {
tier_page_type[0] = PAGE_METRICS;
tier_page_type[0] = RRDENG_PAGE_TYPE_ARRAY_32BIT;
netdata_log_error("Invalid dbengine page type ''%s' given. Defaulting to 'raw'.", page_type);
}

Expand Down
2 changes: 1 addition & 1 deletion src/database/engine/journalfile.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -672,7 +672,7 @@ static void journalfile_restore_extent_metadata(struct rrdengine_instance *ctx,
uuid_t *temp_id;
uint8_t page_type = jf_metric_data->descr[i].type;

if (page_type > PAGE_TYPE_MAX) {
if (page_type > RRDENG_PAGE_TYPE_MAX) {
if (!bitmap256_get_bit(&page_error_map, page_type)) {
netdata_log_error("DBENGINE: unknown page type %d encountered.", page_type);
bitmap256_set_bit(&page_error_map, page_type, 1);
Expand Down
Loading

0 comments on commit 00f897a

Please sign in to comment.