[pull] master from netdata:master

CHANGELOG.md

@@ -6,6 +6,21 @@
 
 **Merged pull requests:**
 
+- go.d smartctl: use scan-open when "no\_check\_power\_mode" is "never" [\#18146](https://github.com/netdata/netdata/pull/18146) ([ilyam8](https://github.com/ilyam8))
+- Regenerate integrations.js [\#18145](https://github.com/netdata/netdata/pull/18145) ([netdatabot](https://github.com/netdatabot))
+- go.d smartctl: do scan only once on startup if interval is 0 [\#18144](https://github.com/netdata/netdata/pull/18144) ([ilyam8](https://github.com/ilyam8))
+- ndsudo add smartctl scan-open [\#18143](https://github.com/netdata/netdata/pull/18143) ([ilyam8](https://github.com/ilyam8))
+- Regenerate integrations.js [\#18141](https://github.com/netdata/netdata/pull/18141) ([netdatabot](https://github.com/netdatabot))
+- go.d smartctl add "extra\_devices" option [\#18140](https://github.com/netdata/netdata/pull/18140) ([ilyam8](https://github.com/ilyam8))
+- Spawn server fixes 6 [\#18136](https://github.com/netdata/netdata/pull/18136) ([ktsaou](https://github.com/ktsaou))
+- Regenerate integrations.js [\#18135](https://github.com/netdata/netdata/pull/18135) ([netdatabot](https://github.com/netdatabot))
+- docs: go.d mysql: remove unix sockets from auto\_detection [\#18134](https://github.com/netdata/netdata/pull/18134) ([ilyam8](https://github.com/ilyam8))
+- go.d fix url path overwrite [\#18132](https://github.com/netdata/netdata/pull/18132) ([ilyam8](https://github.com/ilyam8))
+- Spawn server improvements 5 [\#18131](https://github.com/netdata/netdata/pull/18131) ([ktsaou](https://github.com/ktsaou))
+- Spawn server fixes No 4 [\#18127](https://github.com/netdata/netdata/pull/18127) ([ktsaou](https://github.com/ktsaou))
+- go.d filecheck fix dir existence chart label [\#18126](https://github.com/netdata/netdata/pull/18126) ([ilyam8](https://github.com/ilyam8))
+- Regenerate integrations.js [\#18124](https://github.com/netdata/netdata/pull/18124) ([netdatabot](https://github.com/netdatabot))
+- go.d whoisquery fix "days until" in config\_schema.json [\#18121](https://github.com/netdata/netdata/pull/18121) ([ilyam8](https://github.com/ilyam8))
 - go.d smartctl: add scsi read/write/verify error rate [\#18119](https://github.com/netdata/netdata/pull/18119) ([ilyam8](https://github.com/ilyam8))
 - log in the same line [\#18118](https://github.com/netdata/netdata/pull/18118) ([ktsaou](https://github.com/ktsaou))
 - spawn server fixes 3 [\#18117](https://github.com/netdata/netdata/pull/18117) ([ktsaou](https://github.com/ktsaou))
@@ -398,21 +413,6 @@
 - Fix DEB package conflict entries. [\#17584](https://github.com/netdata/netdata/pull/17584) ([Ferroin](https://github.com/Ferroin))
 - fix ndsudo setuid bit for static builds [\#17583](https://github.com/netdata/netdata/pull/17583) ([ilyam8](https://github.com/ilyam8))
 - fix table [\#17581](https://github.com/netdata/netdata/pull/17581) ([hugovalente-pm](https://github.com/hugovalente-pm))
-- Fix invalid item in postinst script for Netdata package. [\#17580](https://github.com/netdata/netdata/pull/17580) ([Ferroin](https://github.com/Ferroin))
-- Regenerate integrations.js [\#17578](https://github.com/netdata/netdata/pull/17578) ([netdatabot](https://github.com/netdatabot))
-- Cpack fixes [\#17576](https://github.com/netdata/netdata/pull/17576) ([vkalintiris](https://github.com/vkalintiris))
-- Fix compilation without `dbengine` [\#17575](https://github.com/netdata/netdata/pull/17575) ([thiagoftsm](https://github.com/thiagoftsm))
-- Fix handling of netdata.conf on install in build system. [\#17572](https://github.com/netdata/netdata/pull/17572) ([Ferroin](https://github.com/Ferroin))
-- Update Netdata subscription plans documentation [\#17571](https://github.com/netdata/netdata/pull/17571) ([Ancairon](https://github.com/Ancairon))
-- go.d prometheus remove apostrophe in label values [\#17570](https://github.com/netdata/netdata/pull/17570) ([ilyam8](https://github.com/ilyam8))
-- remove go.d symbol/debug info with RelWithDebInfo [\#17569](https://github.com/netdata/netdata/pull/17569) ([ilyam8](https://github.com/ilyam8))
-- go.d smartctl add meta setup prerequisites [\#17568](https://github.com/netdata/netdata/pull/17568) ([ilyam8](https://github.com/ilyam8))
-- Regenerate integrations.js [\#17567](https://github.com/netdata/netdata/pull/17567) ([netdatabot](https://github.com/netdatabot))
-- Increase the message size to the spawn server [\#17566](https://github.com/netdata/netdata/pull/17566) ([stelfrag](https://github.com/stelfrag))
-- go.d smartctl small improvements [\#17565](https://github.com/netdata/netdata/pull/17565) ([ilyam8](https://github.com/ilyam8))
-- go.d smartctl improve units [\#17564](https://github.com/netdata/netdata/pull/17564) ([ilyam8](https://github.com/ilyam8))
-- Regenerate integrations.js [\#17561](https://github.com/netdata/netdata/pull/17561) ([netdatabot](https://github.com/netdatabot))
-- Regenerate integrations.js [\#17560](https://github.com/netdata/netdata/pull/17560) ([netdatabot](https://github.com/netdatabot))
 
 ## [v1.45.6](https://github.com/netdata/netdata/tree/v1.45.6) (2024-06-05)
 

packaging/version

@@ -1 +1 @@
-v1.46.0-139-nightly
+v1.46.0-157-nightly

src/collectors/plugins.d/ndsudo.c

@@ -45,6 +45,14 @@ struct command {
             [1] = NULL,
         },
     },
+    {
+        .name = "smartctl-json-scan-open",
+        .params = "--json --scan-open",
+        .search = {
+            [0] = "smartctl",
+            [1] = NULL,
+        },
+    },
     {
         .name = "smartctl-json-device-info",
         .params = "--json --all {{deviceName}} --device {{deviceType}} --nocheck {{powerMode}}",

src/daemon/commands.c

@@ -164,7 +164,7 @@ static cmd_status_t cmd_reopen_logs_execute(char *args, char **message)
     (void)message;
 
     nd_log_limits_unlimited();
-    nd_log_reopen_log_files();
+    nd_log_reopen_log_files(true);
     nd_log_limits_reset();
 
     return CMD_STATUS_SUCCESS;

src/go/plugin/go.d/modules/consul/collect.go

@@ -69,12 +69,11 @@ func (c *Consul) isServer() bool {
 }
 
 func (c *Consul) doOKDecode(urlPath string, in interface{}, statusCodes ...int) error {
-	req, err := web.NewHTTPRequest(c.Request.Copy())
+	req, err := web.NewHTTPRequestWithPath(c.Request, urlPath)
 	if err != nil {
 		return fmt.Errorf("error on creating request: %v", err)
 	}
 
-	req.URL.Path = urlPath
 	if c.ACLToken != "" {
 		req.Header.Set("X-Consul-Token", c.ACLToken)
 	}

src/go/plugin/go.d/modules/couchbase/collect.go

@@ -112,11 +112,13 @@ func (cb *Couchbase) addDimToChart(chartID string, dim *module.Dim) {
 }
 
 func (cb *Couchbase) scrapeCouchbase() (*cbMetrics, error) {
-	ms := &cbMetrics{}
-	req, _ := web.NewHTTPRequest(cb.Request)
-	req.URL.Path = urlPathBucketsStats
+	req, err := web.NewHTTPRequestWithPath(cb.Request, urlPathBucketsStats)
+	if err != nil {
+		return nil, err
+	}
 	req.URL.RawQuery = url.Values{"skipMap": []string{"true"}}.Encode()
 
+	ms := &cbMetrics{}
 	if err := cb.doOKDecode(req, &ms.BucketsBasicStats); err != nil {
 		return nil, err
 	}

src/go/plugin/go.d/modules/couchdb/collect.go

@@ -120,8 +120,7 @@ func (cdb *CouchDB) scrapeCouchDB() *cdbMetrics {
 }
 
 func (cdb *CouchDB) scrapeNodeStats(ms *cdbMetrics) {
-	req, _ := web.NewHTTPRequest(cdb.Request)
-	req.URL.Path = fmt.Sprintf(urlPathOverviewStats, cdb.Config.Node)
+	req, _ := web.NewHTTPRequestWithPath(cdb.Request, fmt.Sprintf(urlPathOverviewStats, cdb.Config.Node))
 
 	var stats cdbNodeStats
 	if err := cdb.doOKDecode(req, &stats); err != nil {
@@ -132,8 +131,7 @@ func (cdb *CouchDB) scrapeNodeStats(ms *cdbMetrics) {
 }
 
 func (cdb *CouchDB) scrapeSystemStats(ms *cdbMetrics) {
-	req, _ := web.NewHTTPRequest(cdb.Request)
-	req.URL.Path = fmt.Sprintf(urlPathSystemStats, cdb.Config.Node)
+	req, _ := web.NewHTTPRequestWithPath(cdb.Request, fmt.Sprintf(urlPathSystemStats, cdb.Config.Node))
 
 	var stats cdbNodeSystem
 	if err := cdb.doOKDecode(req, &stats); err != nil {
@@ -144,8 +142,7 @@ func (cdb *CouchDB) scrapeSystemStats(ms *cdbMetrics) {
 }
 
 func (cdb *CouchDB) scrapeActiveTasks(ms *cdbMetrics) {
-	req, _ := web.NewHTTPRequest(cdb.Request)
-	req.URL.Path = urlPathActiveTasks
+	req, _ := web.NewHTTPRequestWithPath(cdb.Request, urlPathActiveTasks)
 
 	var stats []cdbActiveTask
 	if err := cdb.doOKDecode(req, &stats); err != nil {
@@ -156,8 +153,7 @@ func (cdb *CouchDB) scrapeActiveTasks(ms *cdbMetrics) {
 }
 
 func (cdb *CouchDB) scrapeDBStats(ms *cdbMetrics) {
-	req, _ := web.NewHTTPRequest(cdb.Request)
-	req.URL.Path = urlPathDatabases
+	req, _ := web.NewHTTPRequestWithPath(cdb.Request, urlPathDatabases)
 	req.Method = http.MethodPost
 	req.Header.Add("Accept", "application/json")
 	req.Header.Add("Content-Type", "application/json")
@@ -182,18 +178,18 @@ func (cdb *CouchDB) scrapeDBStats(ms *cdbMetrics) {
 }
 
 func findMaxMQSize(MessageQueues map[string]interface{}) int64 {
-	var max float64
+	var maxSize float64
 	for _, mq := range MessageQueues {
 		switch mqSize := mq.(type) {
 		case float64:
-			max = math.Max(max, mqSize)
+			maxSize = math.Max(maxSize, mqSize)
 		case map[string]interface{}:
 			if v, ok := mqSize["count"].(float64); ok {
-				max = math.Max(max, v)
+				maxSize = math.Max(maxSize, v)
 			}
 		}
 	}
-	return int64(max)
+	return int64(maxSize)
 }
 
 func (cdb *CouchDB) pingCouchDB() error {

src/go/plugin/go.d/modules/dnsdist/collect.go

@@ -36,8 +36,10 @@ func (d *DNSdist) collectStatistic(collected map[string]int64, statistics *stati
 }
 
 func (d *DNSdist) scrapeStatistics() (*statisticMetrics, error) {
-	req, _ := web.NewHTTPRequest(d.Request)
-	req.URL.Path = urlPathJSONStat
+	req, err := web.NewHTTPRequestWithPath(d.Request, urlPathJSONStat)
+	if err != nil {
+		return nil, err
+	}
 	req.URL.RawQuery = url.Values{"command": []string{"stats"}}.Encode()
 
 	var statistics statisticMetrics

src/go/plugin/go.d/modules/elasticsearch/collect.go

@@ -158,13 +158,15 @@ func (es *Elasticsearch) scrapeElasticsearch() *esMetrics {
 }
 
 func (es *Elasticsearch) scrapeNodesStats(ms *esMetrics) {
-	req, _ := web.NewHTTPRequest(es.Request)
+	var p string
 	if es.ClusterMode {
-		req.URL.Path = urlPathNodesStats
+		p = urlPathNodesStats
 	} else {
-		req.URL.Path = urlPathLocalNodeStats
+		p = urlPathLocalNodeStats
 	}
 
+	req, _ := web.NewHTTPRequestWithPath(es.Request, p)
+
 	var stats esNodesStats
 	if err := es.doOKDecode(req, &stats); err != nil {
 		es.Warning(err)
@@ -175,8 +177,7 @@ func (es *Elasticsearch) scrapeNodesStats(ms *esMetrics) {
 }
 
 func (es *Elasticsearch) scrapeClusterHealth(ms *esMetrics) {
-	req, _ := web.NewHTTPRequest(es.Request)
-	req.URL.Path = urlPathClusterHealth
+	req, _ := web.NewHTTPRequestWithPath(es.Request, urlPathClusterHealth)
 
 	var health esClusterHealth
 	if err := es.doOKDecode(req, &health); err != nil {
@@ -188,8 +189,7 @@ func (es *Elasticsearch) scrapeClusterHealth(ms *esMetrics) {
 }
 
 func (es *Elasticsearch) scrapeClusterStats(ms *esMetrics) {
-	req, _ := web.NewHTTPRequest(es.Request)
-	req.URL.Path = urlPathClusterStats
+	req, _ := web.NewHTTPRequestWithPath(es.Request, urlPathClusterStats)
 
 	var stats esClusterStats
 	if err := es.doOKDecode(req, &stats); err != nil {
@@ -201,8 +201,7 @@ func (es *Elasticsearch) scrapeClusterStats(ms *esMetrics) {
 }
 
 func (es *Elasticsearch) scrapeLocalIndicesStats(ms *esMetrics) {
-	req, _ := web.NewHTTPRequest(es.Request)
-	req.URL.Path = urlPathIndicesStats
+	req, _ := web.NewHTTPRequestWithPath(es.Request, urlPathIndicesStats)
 	req.URL.RawQuery = "local=true&format=json"
 
 	var stats []esIndexStats

src/go/plugin/go.d/modules/filecheck/charts.go

@@ -153,7 +153,7 @@ func (f *Filecheck) updateDirCharts(infos []*statInfo) {
 
 		if !sd.hasExistenceCharts {
 			sd.hasExistenceCharts = true
-			f.addFileCharts(info.path,
+			f.addDirCharts(info.path,
 				dirExistenceStatusChartTmpl.Copy(),
 			)
 		}

src/go/plugin/go.d/modules/ipfs/collect.go

@@ -125,13 +125,11 @@ func (ip *IPFS) collectPinLs(mx map[string]int64) error {
 }
 
 func (ip *IPFS) queryStatsBandwidth() (*ipfsStatsBw, error) {
-	req, err := web.NewHTTPRequest(ip.Request)
+	req, err := web.NewHTTPRequestWithPath(ip.Request, urlPathStatsBandwidth)
 	if err != nil {
 		return nil, err
 	}
 
-	req.URL.Path = urlPathStatsBandwidth
-
 	var stats ipfsStatsBw
 	if err := ip.doOKDecode(req, &stats); err != nil {
 		return nil, err
@@ -145,13 +143,11 @@ func (ip *IPFS) queryStatsBandwidth() (*ipfsStatsBw, error) {
 }
 
 func (ip *IPFS) querySwarmPeers() (*ipfsSwarmPeers, error) {
-	req, err := web.NewHTTPRequest(ip.Request)
+	req, err := web.NewHTTPRequestWithPath(ip.Request, urlPathSwarmPeers)
 	if err != nil {
 		return nil, err
 	}
 
-	req.URL.Path = urlPathSwarmPeers
-
 	var stats ipfsSwarmPeers
 	if err := ip.doOKDecode(req, &stats); err != nil {
 		return nil, err
@@ -161,13 +157,11 @@ func (ip *IPFS) querySwarmPeers() (*ipfsSwarmPeers, error) {
 }
 
 func (ip *IPFS) queryStatsRepo() (*ipfsStatsRepo, error) {
-	req, err := web.NewHTTPRequest(ip.Request)
+	req, err := web.NewHTTPRequestWithPath(ip.Request, urlPathStatsRepo)
 	if err != nil {
 		return nil, err
 	}
 
-	req.URL.Path = urlPathStatsRepo
-
 	var stats ipfsStatsRepo
 	if err := ip.doOKDecode(req, &stats); err != nil {
 		return nil, err
@@ -177,13 +171,11 @@ func (ip *IPFS) queryStatsRepo() (*ipfsStatsRepo, error) {
 }
 
 func (ip *IPFS) queryPinLs() (*ipfsPinsLs, error) {
-	req, err := web.NewHTTPRequest(ip.Request)
+	req, err := web.NewHTTPRequestWithPath(ip.Request, urlPathPinLs)
 	if err != nil {
 		return nil, err
 	}
 
-	req.URL.Path = urlPathPinLs
-
 	var stats ipfsPinsLs
 	if err := ip.doOKDecode(req, &stats); err != nil {
 		return nil, err

src/go/plugin/go.d/modules/logstash/collect.go

@@ -45,8 +45,10 @@ func (l *Logstash) updateCharts(pipelines map[string]pipelineStats) {
 }
 
 func (l *Logstash) queryNodeStats() (*nodeStats, error) {
-	req, _ := web.NewHTTPRequest(l.Request.Copy())
-	req.URL.Path = urlPathNodeStatsAPI
+	req, err := web.NewHTTPRequestWithPath(l.Request, urlPathNodeStatsAPI)
+	if err != nil {
+		return nil, err
+	}
 
 	var stats nodeStats
 

src/go/plugin/go.d/modules/mysql/integrations/mariadb.md

@@ -45,12 +45,8 @@ This collector supports collecting metrics from multiple instances of this integ
 
 #### Auto-Detection
 
-By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP and UNIX sockets:
+By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP sockets:
 
-- /var/run/mysqld/mysqld.sock
-- /var/run/mysqld/mysql.sock
-- /var/lib/mysql/mysql.sock
-- /tmp/mysql.sock
 - 127.0.0.1:3306
 - "[::1]:3306"
 

src/go/plugin/go.d/modules/mysql/integrations/mysql.md

@@ -45,12 +45,8 @@ This collector supports collecting metrics from multiple instances of this integ
 
 #### Auto-Detection
 
-By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP and UNIX sockets:
+By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP sockets:
 
-- /var/run/mysqld/mysqld.sock
-- /var/run/mysqld/mysql.sock
-- /var/lib/mysql/mysql.sock
-- /tmp/mysql.sock
 - 127.0.0.1:3306
 - "[::1]:3306"
 

src/go/plugin/go.d/modules/mysql/integrations/percona_mysql.md

@@ -45,12 +45,8 @@ This collector supports collecting metrics from multiple instances of this integ
 
 #### Auto-Detection
 
-By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP and UNIX sockets:
+By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP sockets:
 
-- /var/run/mysqld/mysqld.sock
-- /var/run/mysqld/mysql.sock
-- /var/lib/mysql/mysql.sock
-- /tmp/mysql.sock
 - 127.0.0.1:3306
 - "[::1]:3306"
 

src/go/plugin/go.d/modules/mysql/metadata.yaml

@@ -47,12 +47,8 @@ modules:
       default_behavior:
         auto_detection:
           description: |
-            By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP and UNIX sockets:
-
-            - /var/run/mysqld/mysqld.sock
-            - /var/run/mysqld/mysql.sock
-            - /var/lib/mysql/mysql.sock
-            - /tmp/mysql.sock
+            By default, it detects instances running on localhost by trying to connect as root and netdata using known MySQL TCP sockets:
+
             - 127.0.0.1:3306
             - "[::1]:3306"
         limits: