Need help !: Not able to perform influx db export

[M0hanrajp]

[ Issue ]

Not able to send device-virtual metrics data to Influx db !

[ Description ]

• Execute simple scenario of sending device-virtual metrics to influx db and UI(localhost:8086) should indicate metrics.
• Export to influx db is performed by referring metrics-influxdb
  

I have included environment variables, but I don't see device virtual sending any metrics.

device-virtual configurations: (full configurations provided in the attached .yml file)

  device-virtual:
...
...
    environment:
      EDGEX_SECURITY_SECRET_STORE: "false"
      SERVICE_HOST: edgex-device-virtual
      WRITABLE_TELEMETRY_INTERVAL: "3s"
      WRITABLE_TELEMETRY_METRICS_EVENTSSENT: "true"
      WRITABLE_TELEMETRY_METRICS_READINGSSSENT: "true"
      WRITABLE_LOGLEVEL: TRACE
    hostname: edgex-device-virtual
    image: edgexfoundry/device-virtual:3.1.1
...

device-virtual logs:

edgex-device-virtual  | level=INFO ts=2024-08-05T14:05:06.678514465Z app=device-virtual source=messaging.go:104 msg="Connected to redis Message Bus @ redis://edgex-redis:6379 with AuthMode='usernamepassword'"
# Publish 0 metrics to the 'edgex/telemetry/device-virtual
edgex-device-virtual  | level=DEBUG ts=2024-08-05T14:05:36.6855458Z app=device-virtual source=reporter.go:195 msg="Publish 0 metrics to the 'edgex/telemetry/device-virtual' base topic"

According to me, if I'm able to send metric data to app-metrics-influx then I would be sending data to Influx db.

Please help on this !

---

Thanks.
docker-compose.txt

[cloudxxx8]

please use the /api/v3/config to confirm the environment variable setting takes effect.
if you started the services without these envronment variables before, the configs are pushed to the config provider (Consul by default), and they will take priority.
You need to add "-o" flag to the service or modify the config from the config provider.
If it's Consul, try to use its UI to modifyt http://ip-address:8500

[M0hanrajp]

Can you please check below comment ? Thanks

[M0hanrajp]

used /api/v3/config for device-virtual and the values were not set:

      "Telemetry": {
        "Interval": "30s",
        "Metrics": {
          "EventsSent": false,
          "LastConnected": false,
          "ReadingsSent": false,
          "SecurityConsulTokenDuration": false,
          "SecurityConsulTokensRequested": false,
          "SecurityGetSecretDuration": false,
          "SecurityRuntimeSecretTokenDuration": false,
          "SecuritySecretsRequested": false,
          "SecuritySecretsStored": false
        },
        "Tags": null
      }
    }
  },
  "serviceName": "device-virtual"
}

So I added overwrite and cp & config provider for device-virtual in the compose file:

  device-virtual:
    command:
    - cp
    - --overwrite
    container_name: edgex-device-virtual
    depends_on:
      consul:
        condition: service_started
        required: true
      core-data:
        condition: service_started
        required: true
      core-metadata:
        condition: service_started
        required: true
    environment:
      EDGEX_SECURITY_SECRET_STORE: "false"
      SERVICE_HOST: edgex-device-virtual
      WRITABLE_TELEMETRY_INTERVAL: "5s"
      WRITABLE_TELEMETRY_METRICS_EVENTSSENT: "true"
      WRITABLE_TELEMETRY_METRICS_READINGSSSENT: "true"
      WRITABLE_LOGLEVEL: TRACE
      EDGEX_CONFIG_PROVIDER: consul.http://edgex-core-consul:8500

When used /api/v3/config again the values are not set and there is no metrics published.

edgex-device-virtual  | level=DEBUG ts=2024-08-06T10:39:36.031538196Z app=device-virtual source=reporter.go:195 msg="Publish 0 metrics to the 'edgex/telemetry/device-virtual' base topic"

I'm I missing anything ?

[cloudxxx8]

your command setting is wrong. please try:

    command:
      - -cp=consul.http://edgex-core-consul:8500
      - --registry
      - -o

[M0hanrajp]

I used the same configuration:

  device-virtual:
    command:
    - -cp=consul.http://edgex-core-consul:8500
    - --registry
    - -o
    container_name: edgex-device-virtual
    depends_on:
      consul:
        condition: service_started
        required: true
      core-data:
        condition: service_started
        required: true
      core-metadata:
        condition: service_started
        required: true
    environment:
      EDGEX_SECURITY_SECRET_STORE: "false"
      SERVICE_HOST: edgex-device-virtual
      WRITABLE_TELEMETRY_INTERVAL: 5s
      WRITABLE_TELEMETRY_METRICS_EVENTSSENT: TRUE
      WRITABLE_TELEMETRY_METRICS_LASTCONNECTED: TRUE
      WRITABLE_TELEMETRY_METRICS_READINGSSSENT: TRUE
      WRITABLE_LOGLEVEL: TRACE

But there are no metrics published:

edgex-device-virtual  | level=DEBUG ts=2024-08-07T06:58:43.081496296Z app=device-virtual source=reporter.go:195 msg="Publish 0 metrics to the 'edgex/telemetry/device-virtual' base topic"

api/v3/config also returns false for all the settings.

[cloudxxx8]

it looks like -o won't publish the environment variables override.
can it work if you directly modify the config from Consul?

[M0hanrajp]

OK I edited device service metrics by following the documentation: Via Config Provider (after deployment)

I'm able to see device metrics data being published to edgex/telemetry

edgex-device-virtual  | level=DEBUG ts=2024-08-07T09:51:12.383040393Z app=device-virtual source=reporter.go:195 msg="Publish 2 metrics to the 'edgex/telemetry/device-virtual' base topic"
edgex-device-virtual  | level=DEBUG ts=2024-08-07T09:51:12.383076279Z app=device-virtual source=manager.go:123 msg="Reported metrics..."
edgex-device-virtual  | level=DEBUG ts=2024-08-07T09:51:17.387256178Z app=device-virtual source=reporter.go:195 msg="Publish 5 metrics to the 'edgex/telemetry/device-virtual' base topic"
edgex-device-virtual  | level=DEBUG ts=2024-08-07T09:51:17.387333192Z app=device-virtual source=manager.go:123 msg="Reported metrics..."

I have a few more questions here:

Based on documentation on Command Line Options

Two command line options (-o for overwrite and -r for registry) are not overridable by environmental variables.

NOTES: Use of the --overwrite command line option should be used sparingly and with expert knowledge of EdgeX; in particular knowledge of how it operates and where/how it gets its configuration on restarts, etc. Ordinarily, --overwrite is provided as a means to support development needs. Use of --overwrite permanently in production enviroments is highly discouraged.

Since use of of --overwrite permanently in production enviroments is highly discouraged.

Then if I need to enable metrics in my device-virtual I need to clone the source code make changes, build and upload to docker Hub ? so that this particular edit can be used in my custom docker-compose.yml ? Is there any alternative way ?

If I'm planning to make changes to the service, let's say to include a different topic, different registry what is the suggested method to make changes ? Native build and post to docker hub ? environment variable (which don't seem to work) ? or any other ?

---

Edit: Managed to get the influx db UI to work.

[jiekechoo]

Then if I need to enable metrics in my device-virtual I need to clone the source code make changes, build and upload to docker Hub ? so that this particular edit can be used in my custom docker-compose.yml ? Is there any alternative way ?

You just build the docker image of device-service in your host machine.