Merge pull request #25 from Kilemonn/authenticated-sub-queue

Authenticated sub queue

Dockerfile

@@ -5,7 +5,7 @@
 # Push image to remote
 # docker push kilemon/message-queue:0.1.5
 
-FROM gradle:7.5.1-jdk17-alpine as builder
+FROM gradle:8.4.0-jdk17-alpine as builder
 
 WORKDIR /builder
 

README.md

@@ -4,237 +4,22 @@
 ## Overview
 
 A message queue service, which can receive, hold and provide messages that are sent between services.
-The message storage mechanisms supported are:
-- In-memory (default)
-- Redis (stand alone and sentinel support)
-- SQL Database (MySQL, PostgreSQL)
-- No SQL Database (Mongo)
+A storage mechanism can be used to persist messages and sub queues can be restricted so only correctly provided credentials
+can interact with such queues.
 
-## Rest API Documentation
-
-The application provides a REST API to interact with the messages queued within the Multi Queue.
-REST Documentation is provided as Swagger docs from the running application. 
-You can simply run the docker image:
-> docker run -p8080:8080 kilemon/message-queue
-
-Once the image is running you can reach the Swagger documentation from the following endpoint: `http://localhost:8080/swagger-ui/index.html`.
+**More detailed documentation can be found in the Wiki!**
 
 ## Quick Start
 
-## In-Memory
-
-The `In-Memory` configuration is the default and requires no further configuration.
-Steps to run the In-Memory Multi Queue is as follows:
-- `docker run -p8080:8080 kilemon/message-queue`
-- Once running the best endpoint to call at the moment is probably: `http://localhost:8080/queue/keys`
-
-If you really like you can provide an environment variable to the application to explicitly set the application into `In-Memory` mode: `MULTI_QUEUE_TYPE=IN_MEMORY`.
-
-## Redis
-
-The `Redis` setup requires some additional environmental configuration.
-Firstly to set the application into `Redis` mode you need to provide the following environment variable with the appropriate value: `MULTI_QUEUE_TYPE=REDIS`.
-Once this is set you will need to provide further environment variable in order to correctly configure the redis standalone or sentinel configuration (depending on your setup).
-
-### Redis Environment Properties
-
-Below are the optional and required properties for the `Redis` configuration.
-
-#### REDIS_USE_SENTINELS
-
-By default, this property is set to `false` (defaulting to direct connection to a single Redis instance).
-
-This flag indicates whether the `MultiQueue` should connect directly to the redis instance or connect via one or more sentinel instances.
-If set to `true` the `MultiQueue` will create a sentinel pool connection instead of a direct connection to a single redis node.
-
-#### REDIS_PREFIX
-
-By default, this property is set to `""` the application will apply no prefix.
-
-For each defined "sub-queue" or "queue type", the application will create a single set entry into redis. This prefix can be used to remove/reduce the likelihood of any collisions if this is being used in an existing redis instance.
-
-The defined value will be used as a prefix used for all redis entry keys that the application will create.
-E.g. if the initial value for the redis entry key is `my-key` and no prefix is defined the entries would be stored under `my-key`.
-Using the same scenario if the prefix is `prefix` then the prefix and entry key will be concatenated and the resultant key would be `prefixmy-key`.
-
-#### REDIS_ENDPOINT
-
-By default, this property is set to `127.0.0.1` (the application will auto append the default redis port if it is not defined).
-
-The input endpoint string which is used for both standalone and the sentinel redis configurations.
-This supports a comma separated list or single definition of a redis endpoint in the following formats:
-`<endpoint>:<port>,<endpoint2>:<port2>,<endpoint3>`.
-
-If you are using standalone and provide multiple endpoints, only the first will be used.
-
-#### REDIS_MASTER_NAME
-
-By default, this property is set to `mymaster`.
-This is **REQUIRED** when `REDIS_USE_SENTINELS` is set to `true`. Is used to indicate the name of the redis master instance.
-
-### Example:
-
-An example of `Redis` configuration environment variables is below:
-```yaml
-environment:
- - MULTI_QUEUE_TYPE=REDIS
- - REDIS_USE_SENTINELS=true
- - REDIS_PREFIX=my-prefix
- - REDIS_ENDPOINT=sentinel1.com:5545,sentinel2.org:9980
- - REDIS_MASTER_NAME=not-my-master
-```
-
-## SQL Database
-
-The `SQL` mechanism requires some configuration too similarly to `Redis`.
-To set the application into `SQL` mode you need to provide the following environment variable with the appropriate value: `MULTI_QUEUE_TYPE=SQL`.
-
-### SQL Environment Properties
-
-Below are the required properties for `SQL` configuration.
-
-#### spring.jpa.hibernate.ddl-auto
-
-Depending on your setup this is probably required initially but may change based on your usage needs. Since the application uses Hibernate to create and generate the database structure.
-If the structure is not initialised, I recommend setting this property to `create` (E.g. `spring.jpa.hibernate.ddl-auto=create`).
-
-Please refer to https://docs.spring.io/spring-boot/docs/1.1.0.M1/reference/html/howto-database-initialization.html#howto-initialize-a-database-using-hibernate
-
-#### spring.autoconfigure.exclude
-
-***This property is required***.
-
-Just providing `spring.autoconfigure.exclude=` as one of the environment variables is required to force JPA to initialise correctly. 
-By default, it is suppressed to allow of more stream lined configuration of the other mechanisms.
-
-#### spring.datasource.url
-
-***This property is required***.
-
-This defines the database connection string that the application should connect to. E.g: `jdbc:mysql://localhost:3306/message-queue`
+By default, the application will be store messages in memory and no queue restriction will be available.
+To start the application you can use the following command to pull and run the latest version of the image:
 
-#### spring.datasource.username
+`docker run -p8080:8080 kilemon/message-queue`
 
-***This property is required***.
+Once running the best endpoint to call at the moment is probably: `http://localhost:8080/queue/healthcheck`
 
-This is the username/account name used to access the database at the configured endpoint.
+The application provides REST APIs to interact with the messages queued within the MultiQueue.
 
-#### spring.datasource.password
-
-***This property is required***.
-
-This is the password used to access the database at the configured endpoint.
-
-### Example:
-```yaml
-environment:
- - MULTI_QUEUE_TYPE=SQL
- - spring.jpa.hibernate.ddl-auto=create
- - spring.autoconfigure.exclude=
- - spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/postgres
- - spring.datasource.username=postgres
- - spring.datasource.password=5up3r5tR0nG!
-```
-
-## NO SQL (Mongo DB)
-
-The application can be set into `MONGO` mode to interface with a NoSQL database. Similarly to the others you can set the type with `MULTI_QUEUE_TYPE=MONGO`. 
-
-### NoSQL Environment Properties
-
-You can either specify all properties individually via, `spring.data.mongodb.host`, `spring.data.mongodb.port`, `spring.data.mongodb.database`, `spring.data.mongodb.username`, `spring.data.mongodb.password`.
-Or you can provide all together in a single property: `spring.data.mongodb.uri`.
-
-#### spring.data.mongodb.host
-
-***This property is required unless `spring.data.mongodb.uri` is provided***.
-
-This is the host that the mongo DB is accessible from.
-
-#### spring.data.mongodb.database
-
-***This property is required unless `spring.data.mongodb.uri` is provided***.
-
-This is the database that should be connected to and where the related documents will be created.
-
-#### spring.data.mongodb.username
-
-***This property is required unless `spring.data.mongodb.uri` is provided***.
-
-This is the username/account name used to access the database at the configured endpoint.
-
-#### spring.data.mongodb.password
-
-***This property is required unless `spring.data.mongodb.uri` is provided***.
-
-This is the password used to access the database at the configured endpoint.
-
-#### spring.data.mongodb.port
-
-***This property is required unless `spring.data.mongodb.uri` is provided***.
-
-The port that the mongo db has exposed.
-
-#### spring.data.mongodb.uri
-
-***This property is required unless the above properties are already provided***.
-
-The whole url can be provided in the following format: `mongodb://<username>:<password>@<host>:<port>/<database>` for example: `mongodb://root:password@localhost:27107/messagequeue`.
-
-### Example:
-***Note:** the use of `?authSource=admin` is to allow you to get up and running quickly, properly secured credentials and a non-admin account should always be used.*
-
-```yaml
-environment:
-- MULTI_QUEUE_TYPE=MONGO
-- spring.data.mongodb.uri=mongodb://root:password@mongo:27017/messagequeue?authSource=admin
-```
-
----
-
-## HTTPS
-
-By default the `MessageQueue` does not have HTTPS enabled and is exposed on port `8080`.
-To enable HTTPS you'll need to provide your own SSL certificate and extend the current version of the image hosted at: https://hub.docker.com/r/kilemon/message-queue. When extending this image you want to add your own SSL certificate into the container and take note of the generated file location as you'll need to reference it in the environment properties you provide to the `MessageQueue`.
-**NOTE: You need to use version 0.1.9 or above of the `MessageQueue` image.**
-
-Below is an example Dockerfile that you could use to generate a self signed certificate.
-Dockerfile:
-```
-FROM kilemon/message-queue:latest
-
-# The generated cert will be placed at /messagequeue/keystore.p12 in the container (refer to path in docker compose file).
-RUN ["keytool", "-genkeypair", "-alias", "sslcert", "-keyalg", "RSA", "-keysize", "4096", "-validity", "3650", "-dname", "CN=message-queue", "-keypass", "changeit", "-keystore", "keystore.p12", "-storeType", "PKCS12", "-storepass", "changeit"]
-
-EXPOSE 8443
-
-ENTRYPOINT ["java", "-jar", "messagequeue.jar"]
-```
-
-Using docker compose you can reference and build this Dockerfile and pass in the appropriate parameters to enable HTTP on the `MessageQueue` application:
-
-docker-compose.yml:
-```yaml
-version: "3.9"
-services:
-  queue:
-    container_name: queue
-    build: .
-    ports:
-      - "8443:8443"
-    environment:
-      MULTI_QUEUE_TYPE: IN_MEMORY
-      server.port: 8443 # The port set here must match the health check port below and the exposed port from the Dockerfile
-      server.ssl.enabled: true
-      server.ssl.key-store-type: PKCS12
-      server.ssl.key-store: keystore.p12 # This path is relative to the `messagequeue.jar` location. The full location is /messagequeue/keystore.p12 for this example
-      server.ssl.key-store-password: changeit
-    healthcheck: # Example simple health check, disabling cert check for this example since it is self-signed
-      test: wget --no-check-certificate https://localhost:8443/queue/healthcheck
-      start_period: 3s
-      interval: 3s
-      timeout: 3s
-      retries: 5
-```
+## Rest API Documentation
 
-Once this starts up you should be able to access the application using HTTPS on the exposed port `8443`.
+REST Documentation is provided as Swagger docs from the running application. Once the image is running you can reach the Swagger documentation from the following endpoint: `http://localhost:8080/swagger-ui/index.html`.

build.gradle.kts

@@ -10,7 +10,7 @@ plugins {
 
 group = "au.kilemon"
 // Make sure version matches version defined in MessageQueueApplication
-version = "0.2.1"
+version = "0.3.0"
 java.sourceCompatibility = JavaVersion.VERSION_17
 
 repositories {
@@ -46,12 +46,16 @@ dependencies {
     // https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-data-mongodb
     implementation("org.springframework.boot:spring-boot-starter-data-mongodb:3.1.3")
 
+    // JWT token
+    // https://mvnrepository.com/artifact/com.auth0/java-jwt
+    implementation("com.auth0:java-jwt:4.4.0")
+
     // Test dependencies
     testImplementation("org.springframework.boot:spring-boot-starter-test:3.0.6")
     // Required to mock MultiQueue objects since they apparently override a final 'remove(Object)' method.
     testImplementation("org.mockito:mockito-inline:5.1.0")
     testImplementation("org.junit.jupiter:junit-jupiter-api:5.9.0")
-    testImplementation("org.testcontainers:testcontainers:1.17.5")
+    testImplementation("org.testcontainers:testcontainers:1.19.2")
     testImplementation("org.testcontainers:junit-jupiter:1.17.5")
     testImplementation(kotlin("test"))
 }

src/main/kotlin/au/kilemon/messagequeue/MessageQueueApplication.kt

@@ -17,7 +17,7 @@ open class MessageQueueApplication
         /**
          * Application version number, make sure this matches what is defined in `build.gradle.kts`.
          */
-        const val VERSION: String = "0.2.1"
+        const val VERSION: String = "0.3.0"
     }
 }
 

src/main/kotlin/au/kilemon/messagequeue/authentication/AuthenticationMatrix.kt

@@ -0,0 +1,56 @@
+package au.kilemon.messagequeue.authentication
+
+import com.fasterxml.jackson.annotation.JsonIgnore
+import java.io.Serializable
+import javax.persistence.*
+
+/**
+ * An object that holds subqueue authentication information.
+ * If a specific sub-queue is in restricted mode it will have a matching [AuthenticationMatrix] created which will
+ * be checked to verify if a specific sub-queue can be operated on.
+ * This object is used for `In-memory`, `SQL` and `Redis`.
+ *
+ * @author github.com/Kilemonn
+ */
+@Entity
+@Table(name = AuthenticationMatrix.TABLE_NAME)
+class AuthenticationMatrix(@Column(name = "subqueue", nullable = false) var subQueue: String): Serializable
+{
+    companion object
+    {
+        const val TABLE_NAME: String = "multiqueueauthenticationmatrix"
+    }
+
+    @JsonIgnore
+    @Id
+    @GeneratedValue(strategy = GenerationType.IDENTITY)
+    var id: Long? = null
+
+    /**
+     * Required for MySQL.
+     */
+    constructor() : this("")
+
+    /**
+     * Overriding to only include specific properties when checking if messages are equal.
+     * This checks the following are equal in order to return `true`:
+     * - subQueue
+     */
+    override fun equals(other: Any?): Boolean
+    {
+        if (other == null || other !is AuthenticationMatrix)
+        {
+            return false
+        }
+
+        return other.subQueue == this.subQueue
+    }
+
+    /**
+     * Only performs a hashcode on the properties checked in [AuthenticationMatrix.equals].
+     */
+    override fun hashCode(): Int
+    {
+        return subQueue.hashCode()
+    }
+}

src/main/kotlin/au/kilemon/messagequeue/authentication/AuthenticationMatrixDocument.kt

@@ -0,0 +1,24 @@
+package au.kilemon.messagequeue.authentication
+
+import com.fasterxml.jackson.annotation.JsonIgnore
+import org.springframework.data.annotation.Id
+import org.springframework.data.mongodb.core.mapping.Document
+
+/**
+ * A holder object for the restricted sub-queues. Refer to [AuthenticationMatrix].
+ * This is used only for `Mongo`.
+ *
+ * @author github.com/Kilemonn
+ */
+@Document(value = AuthenticationMatrixDocument.DOCUMENT_NAME)
+class AuthenticationMatrixDocument(var subQueue: String)
+{
+    companion object
+    {
+        const val DOCUMENT_NAME: String = "multiqueueauthenticationmatrix"
+    }
+
+    @JsonIgnore
+    @Id
+    var id: Long? = null
+}