Analytics supporting changes and infrastructure.

.env.example

@@ -1,4 +1,16 @@
 SCHEME=http
 HOSTNAME=localhost:3000
 RATELIMIT_WINDOWTIME=900000
-RATELIMIT_WINDOWSIZE=10
+RATELIMIT_WINDOWSIZE=10
+
+# Analytics Configuration:
+# ========================
+#
+# Analytics Provider Name
+ANALYTICS_ENABLED=false
+ANALYTICS_PROVIDER=ConsoleProvider
+# Configuration for the Plausible Analytics Engine
+PLAUSIBLE_CONFIGURATION={"baseUrl":"<base for example : http://mockedapi.local:3000>","loggingURL": "<Address of the plausible server - for example : http://localhost:8000>"
+# Configuration for the Console Logging.
+CONSOLELOG_CONFIGURATION={"logEmoji":"🔍"}
+

.github/ISSUE_TEMPLATE/bug_report.md

@@ -13,9 +13,9 @@ A clear and concise description of what the bug is.
 **To Reproduce**
 Steps to reproduce the behavior:
 1. Go to '...'
-2. Click on '....'
-3. Scroll down to '....'
-4. See error
+1. Click on '....'
+1. Scroll down to '....'
+1. See error
 
 **Expected behavior**
 A clear and concise description of what you expected to happen.
@@ -24,15 +24,15 @@ A clear and concise description of what you expected to happen.
 If applicable, add screenshots to help explain your problem.
 
 **Desktop (please complete the following information):**
- - OS: [e.g. iOS]
- - Browser [e.g. chrome, safari]
- - Version [e.g. 22]
+- OS: [e.g. iOS]
+- Browser [e.g. chrome, safari]
+- Version [e.g. 22]
 
 **Smartphone (please complete the following information):**
- - Device: [e.g. iPhone6]
- - OS: [e.g. iOS8.1]
- - Browser [e.g. stock browser, safari]
- - Version [e.g. 22]
+- Device: [e.g. iPhone6]
+- OS: [e.g. iOS8.1]
+- Browser [e.g. stock browser, safari]
+- Version [e.g. 22]
 
 **Additional context**
 Add any other context about the problem here.

.gitignore

@@ -6,4 +6,10 @@ npm-debug.log
 .DS_Store
 .idea
 coverage
-.vscode
+.vscode
+
+# ignoring mapped volumes for docker compose / plausible
+plausible-conf.env
+db-data
+event-data
+clickhouse

.remarkrc

@@ -0,0 +1,20 @@
+{   
+  "settings": {
+    "incrementListMarker": false
+  },
+  "plugins": [
+    "preset-lint-consistent",
+    "preset-lint-recommended",
+    "lint-heading-increment",
+    ["lint-no-undefined-references",false],
+    [
+      "lint-list-item-indent","space"
+    ],
+    [
+      "lint-unordered-list-marker-style", "consistent"
+    ],
+    [
+      "lint-ordered-list-marker-value", "one"
+    ]
+  ]
+}

CONTRIBUTING.md

@@ -1,3 +1,4 @@
+<!-- markdownlint-disable -->
 <!-- omit in toc -->
 # Contributing to Mocked-API
 
@@ -14,15 +15,15 @@ All types of contributions are encouraged and valued. See the [Table of Contents
 <!-- omit in toc -->
 ## Table of Contents
 
-    - [I Have a Question](#i-have-a-question)
-    - [I Want To Contribute](#i-want-to-contribute)
-    - [Reporting Bugs](#reporting-bugs)
-    - [Suggesting Enhancements](#suggesting-enhancements)
-    - [Your First Code Contribution](#your-first-code-contribution)
-    - [Improving The Documentation](#improving-the-documentation)
-    - [Coding Conventions](#Coding-Conventions)
-    - [Commit Messages](#commit-messages)
-    - [Join The Project Team](#join-the-project-team)
+- [I Have a Question](#i-have-a-question)
+- [I Want To Contribute](#i-want-to-contribute)
+- [Reporting Bugs](#reporting-bugs)
+- [Suggesting Enhancements](#suggesting-enhancements)
+- [Your First Code Contribution](#your-first-code-contribution)
+- [Improving The Documentation](#improving-the-documentation)
+- [Coding Conventions](#coding-conventions)
+- [Commit Messages](#commit-messages)
+- [Join The Project Team](#join-the-project-team)
 
 
 
@@ -130,8 +131,8 @@ Enhancement suggestions are tracked as [GitHub issues](https://github.com/agedde
 include Setup of env, IDE and typical getting started instructions?
 -->
 1. When adding new API endpoints, please add OpenAPI comments
-2. All code must have corresponding unit tests.
-3. All API endpoints must have unit tests for all return codes.
+1. All code must have corresponding unit tests.
+1. All API endpoints must have unit tests for all return codes.
 
 ### Improving The Documentation
 <!-- TODO
@@ -153,31 +154,31 @@ All of the folders and files should use `kebab-case`.
 ### File Extensions
 
 This is all about the files and the clear definition of their usage :
-
-    - APIs
-        - `*.routes.ts`
-        - This file located right inside the feature's folder
-    - Data
-        - `*.ts`
-        - This file located under `data` folder of the feature's folder (e.g. `data/*.ts`)
-        - This `data` folder is used to store the static data used in APIs (not your mock data)
-    - Interfaces/Types
-        - `*.types.ts`
-        - This file located under `models` folder of the feature's folder (e.g. `models/*.types.ts`)
-    - Utilities
-        - `*.ts`
-        - This file located under `utils` folder of the feature's folder (e.g. `utils/*.ts`)
-    - Tests
-        - `*.test.ts`
-        - This file located under `tests` folder of the feature's folder (e.g. `tests/*.test.ts`)
-        - This `tests` folder should follow the feature's folder structure, e.g. :
-            - `tests/*.test.ts` -> test file for the APIs
-            - `tests/utils/*.test.ts` -> test file for the APIs
-
+```none
+- APIs
+    - `*.routes.ts`
+    - This file located right inside the feature's folder
+- Data
+    - `*.ts`
+    - This file located under `data` folder of the feature's folder (e.g. `data/*.ts`)
+    - This `data` folder is used to store the static data used in APIs (not your mock data)
+- Interfaces/Types
+    - `*.types.ts`
+    - This file located under `models` folder of the feature's folder (e.g. `models/*.types.ts`)
+- Utilities
+    - `*.ts`
+    - This file located under `utils` folder of the feature's folder (e.g. `utils/*.ts`)
+- Tests
+    - `*.test.ts`
+    - This file located under `tests` folder of the feature's folder (e.g. `tests/*.test.ts`)
+    - This `tests` folder should follow the feature's folder structure, e.g. :
+        - `tests/*.test.ts` -> test file for the APIs
+        - `tests/utils/*.test.ts` -> test file for the APIs
+```
 ### File Structures
 
 The file structures in this repository should look like this :
-
+```bash
     Mocked-API
     │   README.md
     │   package.json
@@ -275,7 +276,7 @@ The file structures in this repository should look like this :
     │       │   ...
     │
     └───another-folders
-
+```
 ### Commit Messages
 
 We use semantic commit messages

README.md

@@ -23,9 +23,9 @@ If you would like to contribute during Hacktoberfest please following our contri
 
 1. Find a ticket and ask to be assigned, once assigned start work. :) This is so we don't get multiple PR's for the same issue.
 1. Please branch from our active next release *dev*
-2. Please make sure your new endpoints all have OpenAPI Comments as they will be used to generate our swagger documentation.
-3. Please make sure you have added unit tests for end points and util functions.
-4. MOST importantly, have fun.
+1. Please make sure your new endpoints all have OpenAPI Comments as they will be used to generate our swagger documentation.
+1. Please make sure you have added unit tests for end points and util functions.
+1. MOST importantly, have fun.
 
 If you need help, please leave a comment on the ticket you are assigned to our join our discord by clicking the link below.
 
@@ -50,12 +50,12 @@ Once you've cloned this repository, running Mocked-API is quite easy, there are
 ```bash
 npm install
 ```
-2. Configure the application
+1. Configure the application
 An example configuration file `.env.example` is provided.
 ```bash
 npm run config
 ```
-3. Serve the application
+1. Serve the application
 ```bash
 npm run serve
 ```

app.ts

@@ -3,26 +3,36 @@ import express, { Request, Response } from 'express';
 import { swaggerSpec } from './src/utils/swagger';
 import swag from './swagger.json';
 import { applicationRateLimiter } from './middleware/rate-limiter/RateLimiter';
+import { AnalyticsLoader } from './middleware/analytics/AnalyticsLoader';
+import {IAnalytics} from './middleware/analytics/IAnalytics';
 import path from 'path';
 const morgan = require('morgan');
 const cors = require('cors');
-
 var fs = require('fs');
-
 const app = express();
+const ENVIRONMENT = process.env.NODE_ENV;
+const ANALYTICS_ENABLED: boolean = process.env.ANALYTICS_ENABLED ? JSON.parse(process.env.ANALYTICS_ENABLED.toLowerCase()) : false;
 
 // Rate limit middleware
 app.use(applicationRateLimiter); // rate-limit applied to all the routes by default
-
 var constantPath = './src/modules/';
-var routes = {};
 
+const APIrouter = express.Router();
+var Analytics : IAnalytics = null;
+Analytics = AnalyticsLoader(process.env.ANALYTICS_PROVIDER,ANALYTICS_ENABLED, ENVIRONMENT);
+if (Analytics){
+    APIrouter.use(Analytics.middleware(JSON.parse(process.env[Analytics.name])))
+}
+
+var routes = {};
 fs.readdirSync(constantPath).forEach((module) => {
     const apiRoutePath = `${constantPath}${module}/api/`;
     fs.readdirSync(apiRoutePath).forEach((route) => {
-        routes[module] = require(`${apiRoutePath}${route}`)(app);
+        routes[module] = require(`${apiRoutePath}${route}`)(APIrouter);
     });
 });
+app.use('/', APIrouter)
+
 
 // Add an healthcheck endpoint
 // Shows amount of API Categories and their endpoints
@@ -51,10 +61,11 @@ app.get('/docs.json', (req: Request, res: Response) => {
     res.setHeader('Content-Type', 'application/json');
     res.send(swaggerSpec);
 });
-
+ 
 app.use(express.static(path.join(__dirname,'public')))
 
 const schemaOptions = {
+    customJsStr: Analytics ? Analytics.swaggerRegistration(JSON.parse(process.env[Analytics.name])):undefined,
     swaggerOptions: {
         dom_id: '#swagger-ui',
         tagsSorter: 'alpha',
@@ -63,6 +74,10 @@ const schemaOptions = {
     },
 };
 
+
+
+
+
 // Setup Swagger API Documentation
 const swaggerUi = require('swagger-ui-express');
 app.use('/', swaggerUi.serve, swaggerUi.setup(swaggerSpec, schemaOptions));

docker-compose.yml

@@ -8,4 +8,11 @@ services:
     ports:
       - 3000:3000
     volumes:
-      - "./:/usr/src/app/"
+      - "./:/usr/src/app/"
+
+  db-data:
+    driver: local
+  event-data:
+    driver: local
+  geoip:
+    driver: local

infrastructure/analytics/Plausible/docker-compose.yml

@@ -0,0 +1,52 @@
+version: '3.7'
+
+services:
+  mail:
+    image: mailhog/mailhog
+    logging:
+      driver: 'none'  # disable saving logs
+    ports:
+      - 1025:1025 # smtp server
+      - 8025:8025 # web ui
+
+  plausible_db:
+    # supported versions are 12, 13, and 14
+    image: postgres:14-alpine
+    restart: always
+    volumes:
+      - db-data:/var/lib/postgresql/data
+    environment:
+      - POSTGRES_PASSWORD=postgres
+
+  plausible_events_db:
+    image: clickhouse/clickhouse-server:22.6-alpine
+    restart: always
+    volumes:
+      - event-data:/var/lib/clickhouse
+      - ./clickhouse/clickhouse-config.xml:/etc/clickhouse-server/config.d/logging.xml:ro
+      - ./clickhouse/clickhouse-user-config.xml:/etc/clickhouse-server/users.d/logging.xml:ro
+    ulimits:
+      nofile:
+        soft: 262144
+        hard: 262144
+
+  plausible:
+    image: plausible/analytics:latest
+    restart: always
+    command: sh -c "sleep 10 && /entrypoint.sh db createdb && /entrypoint.sh db migrate && /entrypoint.sh db init-admin && /entrypoint.sh run"
+    depends_on:
+      - plausible_db
+      - plausible_events_db
+      - mail
+    ports:
+      - 8000:8000
+    env_file:
+      - plausible-conf.env
+
+volumes:
+  db-data:
+    driver: local
+  event-data:
+    driver: local
+  geoip:
+    driver: local

infrastructure/analytics/Plausible/plausible-conf.env.example

@@ -0,0 +1,5 @@
+[email protected]
+ADMIN_USER_NAME=bob
+ADMIN_USER_PWD=changeme
+BASE_URL=http://localhost:8000  #Port 8000 by default without reverse proxy
+SECRET_KEY_BASE="xxxxREAD INSTRUCTIONS TO CHANGExxxx"

middleware/analytics/AnalyticsLoader.ts

@@ -0,0 +1,24 @@
+import { IAnalytics } from "./IAnalytics";
+import {consoleLogger} from "./console-logger";
+import {plausible} from "./plausible";
+
+/**
+ * Analytics provider loader
+ * @param ProviderName string: Name of the provider to load
+ * @param enabled bool : Is analytics enabled?
+ * @param environment string: The environment executing against (no analytics provider will be loaded when testing)
+ * @returns 
+ */
+export const AnalyticsLoader = function(ProviderName: string,enabled:boolean, environment:string) :IAnalytics {
+    if (environment==="test" || enabled === false)
+        return null;
+    var providers =[consoleLogger,plausible]
+    var result = providers.filter(x=>{
+        var provider:IAnalytics = new x();
+        return provider.providerName === ProviderName
+    })
+    if (result.length === 1){
+        return new result[0]() as IAnalytics; 
+    }
+    return null;
+}