From f88412290ff970c089ba04ee8207cd2a034c56ce Mon Sep 17 00:00:00 2001 From: atsag Date: Wed, 20 Nov 2024 13:48:05 +0200 Subject: [PATCH 1/9] Debugging message (#32) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --- .../discovery/registration/service/SALRegistrationService.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java index a5b769a..3113101 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java @@ -144,7 +144,7 @@ public void register(Device device) { } //TODO handle the response here Map response = register_device_publisher.publish_for_response(register_device_message_string, Collections.singleton(application_name)); - log.info("The response received while trying to register device " + device_name + " is "+response.toString()); + log.warn("The response received while trying to register device " + device_name + " is "+response.toString()); //} /* This is some realtime information, could be retrieved with a different call to the EMS. From 4f0932026a47882709d96e3c14d1f9976f7cc01e Mon Sep 17 00:00:00 2001 From: atsag Date: Thu, 21 Nov 2024 14:58:06 +0200 Subject: [PATCH 2/9] Logging improvements (#33) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --- .../discovery/ResourceDiscoveryProperties.java | 3 ++- .../broker_communication/SALCommunicator.java | 2 +- .../controller/DeviceManagementController.java | 1 + .../resource/discovery/monitor/model/Device.java | 5 +++-- .../service/UnknownDeviceRegistrationService.java | 2 +- .../RegistrationRequestProcessor.java | 7 +++++++ .../service/SALDeregistrationService.java | 9 ++++----- .../service/SALRegistrationService.java | 15 +++++++++++---- 8 files changed, 30 insertions(+), 14 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/ResourceDiscoveryProperties.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/ResourceDiscoveryProperties.java index 7ff3345..0702c7b 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/ResourceDiscoveryProperties.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/ResourceDiscoveryProperties.java @@ -67,7 +67,8 @@ public class ResourceDiscoveryProperties { private boolean salRegistrationEnabled = true; private long salRegistrationTimeout = 60*1000; private String registration_topic_name = "eu.nebulouscloud.exn.sal.node.create"; - + private String deregistration_topic_prefix = "eu.nebulouscloud.exn.sal.node.delete"; + // Failed devices detection private boolean automaticFailedDetection = true; private long suspectDeviceThreshold = 5; // in minutes diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java index b7a598e..6787185 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java @@ -144,7 +144,7 @@ private static void register_devices(String request_body_file, String sessionID, public static String get_device_deregistration_json(Device device){ JSONObject root_json_object = new JSONObject(); - root_json_object.put("name",device.getRef()); + //root_json_object.put("name",device.getRef()); return root_json_object.toJSONString(); } diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/controller/DeviceManagementController.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/controller/DeviceManagementController.java index 48dbc45..8755e78 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/controller/DeviceManagementController.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/controller/DeviceManagementController.java @@ -91,6 +91,7 @@ public Device getDeviceByIpAddress(@PathVariable String ipAddress) { @PutMapping(value = "/device", consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE) public Device createDevice(@RequestBody Device device) { + log.warn("DeviceManagementController: createDevice: device: {}", device); salRegistrationService.ifPresent(salRegistrationService -> salRegistrationService.register(device)); return deviceService.save(device); diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java index f3df47e..b1eafa8 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java @@ -16,8 +16,9 @@ @NoArgsConstructor @Document(collection = "device") public class Device { - private String id; - private String ref; + private String id; //This is the internal id of the device in the Resource Discovery component + private String sal_id; //This identifier is used by SAL, and is used for deregistration + private String ref; //This identifier is used by the Cloud Fog Service Broker, to ascertain which devices are available for which applications private Double price; private String os; private String name; diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java index b544fcc..65fe627 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java @@ -253,7 +253,7 @@ private void processDeviceDetailsResponses() { log.info("UnknownDeviceRegistrationService: Registered device: {}", newDevice); if (salRegistrationService.isPresent()) { - log.info("Registering the device {} to SAL...", newDevice); + log.info("Registering the device to SAL: {}", newDevice); salRegistrationService.get().register(newDevice); } diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java index df5e583..680059c 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java @@ -240,6 +240,7 @@ private void processResponse(@NonNull Map response) { long timestamp = Long.parseLong(response.getOrDefault("timestamp", "-1").toString().trim()); RegistrationRequest registrationRequest = registrationRequestService.getById(requestId).orElse(null); + log.warn("RegistrationRequestProcessor: processResponse: request: {}", registrationRequest); if (registrationRequest!=null) { RegistrationRequestStatus currStatus = registrationRequest.getStatus(); RegistrationRequestStatus newStatus = switch (currStatus) { @@ -288,6 +289,7 @@ private void processResponse(@NonNull Map response) { boolean doArchive = false; Object obj = response.get("nodeInfo"); + log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: {} {}", obj==null?null:obj.getClass().getTypeName(), obj); if (obj instanceof Map devInfo) { // Update request info registrationRequest.setLastUpdateDate(Instant.ofEpochMilli(timestamp)); @@ -297,12 +299,14 @@ private void processResponse(@NonNull Map response) { boolean allowAllKeys = processorProperties.getAllowedDeviceInfoKeys().contains("*"); final Map processedDevInfo = new LinkedHashMap<>(); devInfo.forEach((key, value) -> { + log.warn("RegistrationRequestProcessor: processResponse: Dev-info pair: {} = {}", key, value); if (key!=null && value!=null) { String k = key.toString().trim(); String v = value.toString().trim(); if (StringUtils.isNotBlank(k) && StringUtils.isNotBlank(v)) { if (allowAllKeys || processorProperties.getAllowedDeviceInfoKeys().contains(k.toUpperCase())) { processedDevInfo.put(k, v); + log.warn("RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: {} = {}", key, value); } else { log.debug("processResponse: Not allowed device info key for request: id={}, key={}", requestId, k); } @@ -364,6 +368,8 @@ private RegistrationRequestStatus getNextStatus(RegistrationRequestStatus currSt } private void copyDeviceToMonitoring(RegistrationRequest registrationRequest) { + log.warn("RegistrationRequestProcessor: copyDeviceToMonitoring: BEGIN: request: {}", registrationRequest); + log.warn("RegistrationRequestProcessor: copyDeviceToMonitoring: request-DEVICE: {}", registrationRequest.getDevice()); Device device = objectMapper.convertValue(registrationRequest.getDevice(), Device.class); // override values device.setId(null); @@ -379,6 +385,7 @@ private void copyDeviceToMonitoring(RegistrationRequest registrationRequest) { device.setRequestId(registrationRequest.getId()); device.setNodeReference(registrationRequest.getNodeReference()); deviceManagementService.save(device); + log.warn("RegistrationRequestProcessor: copyDeviceToMonitoring: COPIED-DEVICE: {}", device); if (processorProperties.isSalRegistrationEnabled()) salRegistrationService.ifPresent(salRegistrationService -> salRegistrationService.queueForRegistration(device)); } diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java index 95099a2..cbad55c 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java @@ -62,7 +62,7 @@ public void deregister(Device device) { if (processorProperties.isDeregistration_emulated()){ return; } - SynchronousBrokerPublisher deregister_device_publisher = new SynchronousBrokerPublisher(get_deregistration_topic_name(application_name), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); + SynchronousBrokerPublisher deregister_device_publisher = new SynchronousBrokerPublisher(get_deregistration_topic_name(device.getSal_id()), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); int sending_attempt = 1; while (deregister_device_publisher.is_publisher_null()) { if (sending_attempt <= 2) { @@ -81,7 +81,7 @@ public void deregister(Device device) { //TODO handle the response here if (deregister_device_message_string!=null && !deregister_device_message_string.isEmpty()) { Map response = deregister_device_publisher.publish_for_response(deregister_device_message_string, Collections.singleton(application_name)); - log.info("The response received while trying to deregister device " + device.getRef() + " is " + response.toString()); + log.warn("The response received while trying to deregister device " + device.getRef() + " is " + response.toString()); }else{ log.warn("Deregistration was to be initiated with an empty deregistration payload"); } @@ -100,9 +100,8 @@ public void deregister(Device device) { } - private String get_deregistration_topic_name(String application_name) { - return processorProperties.getRegistration_topic_name(); - //return ("eu.nebulouscloud.exn.sal.edge." + application_name); + private String get_deregistration_topic_name(String sal_id) { + return processorProperties.getDeregistration_topic_prefix()+"."+sal_id; } @Override diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java index 3113101..c5b5f91 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java @@ -8,6 +8,7 @@ import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; +import org.json.simple.JSONObject; import org.springframework.beans.factory.InitializingBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.core.task.TaskExecutor; @@ -37,8 +38,8 @@ public void queueForRegistration(@NonNull Device device) { queue.add(device); } - public void register(Device device) { - + public String register(Device device) { + log.warn("SALRegistrationService: register: BEGIN: device: {}", device); String application_name = device.getRef().split("\\|")[1]; if (application_name.equals("all_applications")){ @@ -46,11 +47,12 @@ public void register(Device device) { } String public_ip = System.getenv("NEBULOUS_IP"); if (public_ip==null || public_ip.isEmpty()){ - log.warn("Using default IP address ("+processorProperties.getNebulous_server_ip_address()+") to fetch Proactive client jar files from, as the environmental variable was not set or found"); public_ip = processorProperties.getNebulous_server_ip_address(); + log.warn("Using default IP address ({}) to fetch Proactive client jar files from, as the environmental variable was not set or found", public_ip); } Map device_info = device.getDeviceInfo(); + log.warn("SALRegistrationService: register: DEVICE-INFO: {}", device_info); /* Information available from the EMS, based on https://gitlab.com/nebulous-project/ems-main/-/blob/master/ems-core/bin/detect.sh?ref_type=heads echo CPU_SOCKETS=$TMP_NUM_CPUS echo CPU_CORES=$TMP_NUM_CORES @@ -145,6 +147,10 @@ public void register(Device device) { //TODO handle the response here Map response = register_device_publisher.publish_for_response(register_device_message_string, Collections.singleton(application_name)); log.warn("The response received while trying to register device " + device_name + " is "+response.toString()); + JSONObject response_json = new JSONObject(response); + JSONObject response_json_body = (JSONObject) response_json.get("body"); + String device_id = (String) response_json_body.get("id"); + return device_id; //} /* This is some realtime information, could be retrieved with a different call to the EMS. @@ -197,7 +203,8 @@ public void processQueue() { device = queue.take(); log.warn("SALRegistrationService: processQueue(): Will register device: {}", device); lastRegistrationStartTimestamp = System.currentTimeMillis(); - register(device); + String device_sal_id = register(device); + device.setSal_id(device_sal_id); lastRegistrationStartTimestamp = -1L; device.setRegisteredToSAL(true); deviceManagementService.update(device); From d1de53300717796141ef32ed67612890fc30ee06 Mon Sep 17 00:00:00 2001 From: atsag Date: Thu, 21 Nov 2024 15:22:07 +0200 Subject: [PATCH 3/9] Sal response parsing improvement (#34) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --- .../registration/service/SALRegistrationService.java | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java index c5b5f91..a35912b 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java @@ -9,6 +9,8 @@ import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.json.simple.JSONObject; +import org.json.simple.parser.JSONParser; +import org.json.simple.parser.ParseException; import org.springframework.beans.factory.InitializingBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.core.task.TaskExecutor; @@ -148,7 +150,13 @@ public String register(Device device) { Map response = register_device_publisher.publish_for_response(register_device_message_string, Collections.singleton(application_name)); log.warn("The response received while trying to register device " + device_name + " is "+response.toString()); JSONObject response_json = new JSONObject(response); - JSONObject response_json_body = (JSONObject) response_json.get("body"); + JSONParser json_parser = new JSONParser(); + JSONObject response_json_body = null; + try { + response_json_body = (JSONObject) json_parser.parse(String.valueOf(response_json.get("body"))); + } catch (ParseException e) { + throw new RuntimeException(e); + } String device_id = (String) response_json_body.get("id"); return device_id; //} From 77625ca8b5fddaf6b34fce2f007c7d049388ce1d Mon Sep 17 00:00:00 2001 From: atsag Date: Thu, 21 Nov 2024 18:10:53 +0200 Subject: [PATCH 4/9] Avoid publish connector stop (#36) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL * Changes to avoid stopping the connector --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --- .../broker_communication/BrokerPublisher.java | 38 +++++++++++-------- .../discovery/monitor/DeviceProcessor.java | 5 ++- 2 files changed, 26 insertions(+), 17 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java index 6863080..42a59ee 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java @@ -63,7 +63,11 @@ public BrokerPublisher(String topic, String broker_ip, int broker_port, String b throw new RuntimeException(e); } } - active_connector.stop(); + try { + active_connector.stop(); //TODO reassure expected stop() functionality is working here when this is necessary + }catch (Exception e){ + e.printStackTrace(); + } } } @@ -104,23 +108,27 @@ public BrokerPublisher(String topic, String broker_ip, int broker_port, String b //TODO The methods below assume that the only content to be sent is json-like public void publish (String json_string_content, Collection application_names){ - for (String application_name : application_names) { - JSONParser parser = new JSONParser(); - JSONObject json_object = new JSONObject(); - try { - json_object = (JSONObject) parser.parse(json_string_content); - } catch (ParseException p) { - log.warn( "publish: Could not parse the string content to be published to the broker as json, which is the following: "+json_string_content); - } - if (!is_publisher_null()) { - private_publisher_instance.send(json_object); - log.info("Sent new message\n"+json_object.toJSONString()); - } else { - log.error( "Could not send message to AMQP broker, as the publisher instance is null"); - } + publish(json_string_content); } } + + public void publish (String json_string_content){ + JSONParser parser = new JSONParser(); + JSONObject json_object = new JSONObject(); + try { + json_object = (JSONObject) parser.parse(json_string_content); + } catch (ParseException p) { + log.warn( "publish: Could not parse the string content to be published to the broker as json, which is the following: "+json_string_content); + } + if (!is_publisher_null()) { + private_publisher_instance.send(json_object); + log.info("Sent new message\n"+json_object.toJSONString()); + } else { + log.error( "Could not send message to AMQP broker, as the publisher instance is null"); + } + } + public boolean is_publisher_null(){ return (private_publisher_instance == null); } diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java index 2ea34c3..5fbdcb3 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java @@ -49,6 +49,7 @@ public class DeviceProcessor implements InitializingBean { private final TaskScheduler taskScheduler; private final AtomicBoolean isRunning = new AtomicBoolean(false); private final Optional salDeregistrationService; + private BrokerPublisher device_lost_publisher; @Override @@ -145,7 +146,7 @@ private void processFailedDevices() { Clock clock = Clock.systemUTC(); lost_device_message.put("timestamp",(int)(clock.millis()/1000)); log.info("Creating new BrokerPublisher to publish device lost message"); - BrokerPublisher device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); + device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); int sending_attempt = 1; while (device_lost_publisher.is_publisher_null()){ @@ -163,7 +164,7 @@ private void processFailedDevices() { } device_lost_publisher.publish(lost_device_message.toJSONString(), Collections.singleton("")); log.warn("processFailedDevices: Marked as FAILED device with Id: {}", device.getId()); - device_lost_publisher.stop(); + //device_lost_publisher.stop(); } deviceManagementService.update(device); From 1a02765fd9039884e8b5c69d2e3a3dc6135f5d6b Mon Sep 17 00:00:00 2001 From: atsag Date: Fri, 22 Nov 2024 11:58:26 +0200 Subject: [PATCH 5/9] Miscellaneous improvements (#37) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL * Changes to avoid stopping the connector * Avoid starting new connectors, mark device as offboarded When a device has failed and it has been processed (i.e a message has been sent to the device lost topic) then it should be marked as offboarded to be archived Do not create a new connector unless this is required Do not use a redundant environmental variable --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --- .../resource/discovery/monitor/DeviceProcessor.java | 6 +++--- .../registration/service/SALRegistrationService.java | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java index 5fbdcb3..3dd0618 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java @@ -145,8 +145,8 @@ private void processFailedDevices() { lost_device_message.put("device_name",device.getName()); Clock clock = Clock.systemUTC(); lost_device_message.put("timestamp",(int)(clock.millis()/1000)); - log.info("Creating new BrokerPublisher to publish device lost message"); - device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); + log.info("Trying to use existing BrokerPublisher to publish device lost message"); + //device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); int sending_attempt = 1; while (device_lost_publisher.is_publisher_null()){ @@ -166,7 +166,7 @@ private void processFailedDevices() { log.warn("processFailedDevices: Marked as FAILED device with Id: {}", device.getId()); //device_lost_publisher.stop(); } - + device.setStatus(DeviceStatus.OFFBOARDED); deviceManagementService.update(device); } diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java index a35912b..ef28ba3 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java @@ -47,11 +47,11 @@ public String register(Device device) { if (application_name.equals("all_applications")){ application_name=""; } - String public_ip = System.getenv("NEBULOUS_IP"); + /*String public_ip = System.getenv("NEBULOUS_IP"); if (public_ip==null || public_ip.isEmpty()){ public_ip = processorProperties.getNebulous_server_ip_address(); - log.warn("Using default IP address ({}) to fetch Proactive client jar files from, as the environmental variable was not set or found", public_ip); - } + log.warn("Using default IP address ({}) to fetch Proactive client jar files from, as the environmental variable was not set or found", public_ip);}*/ + String public_ip = processorProperties.getNebulous_server_ip_address(); Map device_info = device.getDeviceInfo(); log.warn("SALRegistrationService: register: DEVICE-INFO: {}", device_info); From d79877b325592d549e3cb83f8b125ed329a9cab5 Mon Sep 17 00:00:00 2001 From: ipatini <3739531+ipatini@users.noreply.github.com> Date: Fri, 22 Nov 2024 12:12:36 +0200 Subject: [PATCH 6/9] Fix for missing device info. Also additional log messages (#38) --- .../registration/RegistrationRequestProcessor.java | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java index 680059c..ce67109 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java @@ -290,6 +290,20 @@ private void processResponse(@NonNull Map response) { boolean doArchive = false; Object obj = response.get("nodeInfo"); log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: {} {}", obj==null?null:obj.getClass().getTypeName(), obj); + + // If device info are missing copy them from the registration request + if (obj==null || obj instanceof Map && ((Map) obj).isEmpty()) { + log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: ** NO DEVICE INFO IN RESPONSE **"); + if (! registrationRequest.getDevice().getDeviceInfo().isEmpty()) { + obj = registrationRequest.getDevice().getDeviceInfo(); + log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: ** DEVICE INFO COPIED FROM REGISTRATION REQUEST **"); + log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: {}", obj); + } else { + log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: ** PROBLEM: REGISTRATION REQUEST DOES NOT CONTAIN DEVICE INFO EITHER **"); + } + } + log.warn("RegistrationRequestProcessor: processResponse: CHECK: ** NOT A REAL EXCEPTION **\n", new RuntimeException("NOT A REAL EXCEPTION")); + if (obj instanceof Map devInfo) { // Update request info registrationRequest.setLastUpdateDate(Instant.ofEpochMilli(timestamp)); From 3b8920ee92cd6237acd6827f48c59175c36447b0 Mon Sep 17 00:00:00 2001 From: ipatini <3739531+ipatini@users.noreply.github.com> Date: Fri, 22 Nov 2024 13:07:38 +0200 Subject: [PATCH 7/9] RD: Removed unneeded log messages (#40) --- .../eu/nebulous/resource/discovery/common/BrokerUtil.java | 4 +++- .../discovery/registration/RegistrationRequestProcessor.java | 1 - 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java index d308d64..0848a55 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java @@ -200,9 +200,11 @@ public void onMessage(Message message) { Object obj = objectMapper.readerFor(typeRef).readValue(payload); if (obj instanceof Map dataMap) { + log.warn("BrokerUtil: Received a new message: topic: {}", ((ActiveMQTextMessage) message).getDestination().getPhysicalName()); + log.warn("BrokerUtil: Received a new message: payload: {}", dataMap); handlePayload(((ActiveMQTextMessage) message).getDestination().getPhysicalName(), dataMap); } else { - log.debug("BrokerUtil: Message payload is not recognized. Expected Map but got: type={}, object={}", obj.getClass().getName(), obj); + log.warn("BrokerUtil: Message payload is not recognized. Expected Map but got: type={}, object={}", obj.getClass().getName(), obj); } } else { log.debug("BrokerUtil: Message type is not supported: {}", message); diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java index ce67109..bee88c4 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/RegistrationRequestProcessor.java @@ -302,7 +302,6 @@ private void processResponse(@NonNull Map response) { log.warn("RegistrationRequestProcessor: processResponse: nodeInfo: ** PROBLEM: REGISTRATION REQUEST DOES NOT CONTAIN DEVICE INFO EITHER **"); } } - log.warn("RegistrationRequestProcessor: processResponse: CHECK: ** NOT A REAL EXCEPTION **\n", new RuntimeException("NOT A REAL EXCEPTION")); if (obj instanceof Map devInfo) { // Update request info From d8f0e08346fabc925d32c1bd7ebd02bc7792faec Mon Sep 17 00:00:00 2001 From: ipatini <3739531+ipatini@users.noreply.github.com> Date: Fri, 22 Nov 2024 15:31:14 +0200 Subject: [PATCH 8/9] RD: Change to reduced log messages of BrokerUtil (#42) --- .../resource/discovery/common/BrokerUtil.java | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java index 0848a55..964e966 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/common/BrokerUtil.java @@ -200,9 +200,16 @@ public void onMessage(Message message) { Object obj = objectMapper.readerFor(typeRef).readValue(payload); if (obj instanceof Map dataMap) { - log.warn("BrokerUtil: Received a new message: topic: {}", ((ActiveMQTextMessage) message).getDestination().getPhysicalName()); - log.warn("BrokerUtil: Received a new message: payload: {}", dataMap); - handlePayload(((ActiveMQTextMessage) message).getDestination().getPhysicalName(), dataMap); + String topic = ((ActiveMQTextMessage) message).getDestination().getPhysicalName(); + // Print response messages except the EMS node status reports (_ui_instance_info, _client_metrics) + if (StringUtils.isNotBlank(topic) + && ! topic.equals(properties.getDeviceStatusMonitorTopic()) + && ! topic.equals(properties.getDeviceMetricsMonitorTopic())) + { + log.warn("BrokerUtil: Received a new message: topic: {}", topic); + log.warn("BrokerUtil: Received a new message: payload: {}", dataMap); + } + handlePayload(topic, dataMap); } else { log.warn("BrokerUtil: Message payload is not recognized. Expected Map but got: type={}, object={}", obj.getClass().getName(), obj); } From b3eec9e937ab61744dce94c1c0c27657c202f992 Mon Sep 17 00:00:00 2001 From: atsag Date: Fri, 22 Nov 2024 15:40:13 +0200 Subject: [PATCH 9/9] Uninitialized connector fix (#43) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL * Changes to avoid stopping the connector * Sal response parsing improvement and other improvements (#35) * Debugging message (#32) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Logging improvements (#33) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Sal response parsing improvement (#34) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Avoid publish connector stop (#36) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL * Changes to avoid stopping the connector --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos --------- Co-authored-by: atsag Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Avoid starting new connectors, mark device as offboarded When a device has failed and it has been processed (i.e a message has been sent to the device lost topic) then it should be marked as offboarded to be archived Do not create a new connector unless this is required Do not use a redundant environmental variable * RD: Fix for missing device info * R1 candidate changes (#39) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Debugging message (#32) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Logging improvements (#33) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Fix parsing the json response of SAL * Sal response parsing improvement (#34) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized * Password improvements Removal of plaintext passwords and replacement with environmental variable values * Debugging message commit * Work on Deregistration support * Added more logging statements to debug device registration (work by ipatini) * Fix parsing the json response of SAL --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos * Changes to avoid stopping the connector * Avoid publish connector stop (#36) * RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f6048a8b88b4cfc7410db2146c7d7855c03bd. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c207dd900689c94fe5b608a0a03db8fc80. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae6582de0359a4f33b0d8c066b1b55ecad2227. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd3852 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registra… * Fix uninitialized connector issue --------- Co-authored-by: ipatini Co-authored-by: Andreas Tsagkaropoulos Co-authored-by: robert-sanfeliu Co-authored-by: ipatini <3739531+ipatini@users.noreply.github.com> --- .../nebulous/resource/discovery/monitor/DeviceProcessor.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java index 3dd0618..eaa5c79 100644 --- a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java +++ b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/DeviceProcessor.java @@ -148,10 +148,10 @@ private void processFailedDevices() { log.info("Trying to use existing BrokerPublisher to publish device lost message"); //device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), ""); int sending_attempt = 1; - while (device_lost_publisher.is_publisher_null()){ + while (device_lost_publisher==null || device_lost_publisher.is_publisher_null()){ try { - log.warn("Will now make attempt No "+sending_attempt+" to recreate the BrokerPublisher connector for the lost device topic"); + log.warn("Will now make attempt No "+sending_attempt+" to (re)create the BrokerPublisher connector for the lost device topic"); log.info("The topic name is "+processorProperties.getLost_device_topic()+", the broker ip is "+ processorProperties.getNebulous_broker_ip_address()+", the broker port is "+ processorProperties.getNebulous_broker_port()+", the username is "+ processorProperties.getNebulous_broker_username()+", and the password is "+ processorProperties.getNebulous_broker_password()); device_lost_publisher = new BrokerPublisher(processorProperties.getLost_device_topic(), processorProperties.getNebulous_broker_ip_address(), processorProperties.getNebulous_broker_port(), processorProperties.getNebulous_broker_username(), processorProperties.getNebulous_broker_password(), "");