Merge pull request #53 from ICTU/rules-update

Rules update
ICTU · Sep 5, 2023 · c0bc383 · c0bc383
2 parents 59ac0e5 + 46f7729
commit c0bc383
Show file tree

Hide file tree

Showing 13 changed files with 158 additions and 176 deletions.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1 @@
+/.idea
diff --git a/MAINTENANCE.md b/MAINTENANCE.md
@@ -0,0 +1,67 @@
+# Maintenance related tasks
+
+
+## Version upgrade workflow
+
+1. Update `Dockerfile`s with the new version of SonarQube
+1. Update [external plugins](https://github.com/ICTU/sonar/blob/master/plugins/plugin-list)
+1. Create profiles based on the internal plugin versions in [start-with-profile.sh](https://github.com/ICTU/sonar/blob/rules-update/start-with-profile.sh)
+    1. Obtain the base version numbers from the vanilla SonarQube image directory `/opt/sonarqube/lib/extensions`, excluding build number
+    1. Update the profile version number `RULES_VERSION` if the rules have been changed
+1. Create new version tags on github
+    1. `MAJOR.MINOR.PATCH`
+    1. `MAJOR.MINOR.PATCH-developer`
+1. Build and push new images to docker hub with [CircleCI](https://app.circleci.com/pipelines/github/ICTU/sonar)
+
+
+## Adding plugins
+
+Add the url of the plugin jar-file to be installed to `plugins/plugin-list`.
+
+
+## Creating a new quality profile
+
+Modify `start-with-profile.sh` and add a statement to the end of the script, such as:
+
+    createProfile "ictu-cs-profile-v6.6" "Sonar%20way" "cs"
+
+The parameters are:
+* Profile name
+* Base profile name
+* Language (internal SonarQube language identifier)
+
+
+## Create rules txt file from SonarQubes quality profile backup (xml)
+
+In order to make the importing of existing profiles easier, use the transformation `profile_backup_transform.xslt`.
+Go to the profiles page in your SonarQube instance, backup a profile to an xml file and transform it.
+
+
+## Activating or deactivating individual rules in the quality profiles
+
+Modify the corresponding `rules/(language).txt` file.
+Each line represents a rule to be activated or deactivated and has the following syntax: `(operation)(ruleId)#(comment)`
+Please ensure each file ends with a new line character, otherwise the rule will not be added to the profile
+
+* **operation**: `+` activates a rule; `-` deactivates a rule
+* **ruleId**: SonarQube rule identifier
+
+Example:
+
+    +csharpsquid:S104               # NCSS; used by Quality-time
+
+
+## Activating or deactivating rule types in the quality profiles
+
+To (de)activate groups of rules by type use this syntax:
+`(operation)types=(comma,delimited,list,of,types)#(comment)`
+
+The following types are available:
+- `CODE_SMELL`
+- `BUG`
+- `VULNERABILITY`
+- `SECURITY_HOTSPOT`
+
+Example:
+
+    +types=SECURITY_HOTSPOT,VULNERABILITY        # Enable these types by default
diff --git a/README.md b/README.md
@@ -1,118 +1,18 @@
 # ICTU SonarQube Docker image
-A sonar image containing plugins and quality profiles used at ICTU
-
-## Running from docker hub
-
-    docker run -it -p 9000:9000 ictu/sonar
-
-## Building and running locally
-
-    docker build -f Dockerfile-community-edition -t ictusonar .
-    docker run -it -p 9000:9000 ictusonar
-    browse to http://localhost:9000
 
-## Running with PostgreSQL via a docker composition
-
-Example docker-compose file:
-
-    version: '3'
-    services:
-
-      www:
-        image: ictu/sonar:8.6
-        environment:
-          - SONAR_JDBC_URL=jdbc:postgresql://db:5432/sonar
-          - SONAR_JDBC_USERNAME=sonar
-          - SONAR_JDBC_PASSWORD=sonar
-        ports:
-          - 9000:9000
-        links:
-          - db
-
-      db:
-        image: postgres:10.9
-        environment:
-          - POSTGRES_USER=sonar
-          - POSTGRES_PASSWORD=sonar
-        volumes:
-          - /db/postgresql:/var/lib/postgresql
-          # This needs explicit mapping due to https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
-          - /db/postgresql_data:/var/lib/postgresql/data
-
-> Note: Change the passwords above to your own secret value
-
-> Note: Use the environment variables below to provide admin credentials.
-If the default Sonarqube admin password has not yet been changed and SONARQUBE_PASSWORD is provided the startup script will try to change the Sonarqube default password to the one provided.
-Otherwise if incorrect credentials are provided Sonarqube will exit.
-
-      - SONARQUBE_TOKEN=<sonarqube_token>
-
-  or
-
-      - SONARQUBE_USERNAME=<username>
-      - SONARQUBE_PASSWORD=<password>
-
-> Note: The environment variables below can be used to set additional Java options, for instance to set the timezone use:
-
-      - SONAR_WEB_JAVAADDITIONALOPTS=-Duser.timezone=Europe/Amsterdam
-      - SONAR_CE_JAVAADDITIONALOPTS=-Duser.timezone=Europe/Amsterdam
-      - SONAR_SEARCH_JAVAADDITIONALOPTS=-Duser.timezone=Europe/Amsterdam
-
-> Note: The Sonar start script waits for the database to become available (only when using PostgreSQL). DB_START_TIMEOUT (default: 60 seconds) defines how long the script will wait for the database to become available before exiting. Similarly SONAR_START_TIMEOUT (default: 600 seconds) defines how long the script should wait for Sonar to start up.
-
-> Note: The docker images are built automatically with circleci and pushed to docker hub when a tag is created.
-
-## Adding plugins
-Add the url of the plugin to be installed to ```plugins/plugin-list```
+A sonar image containing plugins and quality profiles used at ICTU
 
 
 ## Creating a new quality profile
 
-Modify start-with-profile.sh and add a statement to the end of the script, such as:
+When starting the SonarQube image, new quality profiles will be automatically created for [supported languages](https://github.com/ICTU/sonar/blob/master/rules).
+These newly created profiles are set to be the default profile, but can also be [extended with your own custom rules](https://docs.sonarsource.com/sonarqube/latest/instance-administration/quality-profiles/#extending-a-quality-profile).
 
-    createProfile "ictu-cs-profile-v6.6" "Sonar%20way" "cs"
+Extending the default can be done by ensuring that the current profile has a name ending with `EXTENDED` (or `extended`).
+Alternatively, the automatic overriding of default profile can be avoided by ensuring that the current profile has a name ending with `DEFAULT` (or `default`).
 
-The parameters are:
-Profile name
-Base profile name
-Language (internal SonarQube language identifier)
 
-**The newly created profile will be set to default unless the current default profile has a name ending with "DEFAULT" (or "default")**
-**OR unless the current default profile has a name ending with "EXTENDED" (or "extended"). In the latter case the parent of the current default profile will be changed to the newly created profile.**
-
-## Activating or deactivating individual rules in the quality profiles
-
-Modify the corresponding ```rules/(language).txt``` file.
-Each line represents a rule to be activated or deactivated and has the following syntax:
-```(operation)(ruleId)#(comment)```
-
-Please ensure each file ends with a new line character, otherwise the rule will not be added to the profile!
-
-**operation**:
-    + activates a rule; - deactivates a rule
-
-**ruleId**: SonarQube rule identifier
-
-Example:
-
-    +csharpsquid:S104               # NCSS; used by Quality-time
-
-## Activating or deactivating rule types in the quality profiles
-
-To (de)activate groups of rules by type use this syntax:
-```(operation)types=(comma,delimited,list,of,types)#(comment)```
-
-The following types are available:
-- CODE_SMELL
-- BUG
-- VULNERABILITY
-- SECURITY_HOTSPOT
-
-Example:
-
-    +types=SECURITY_HOTSPOT,VULNERABILITY        # Enable these types by default
-
-## Overriding the standard quality profiles
+## Overriding the ICTU standard quality profiles
 
 Add the project code (it will be used as a prefix for the quality profile name) to the environment variable PROJECT_CODE.
 Add a list of semicolon separated rule ids to be enabled or disabled to the environment variable PROJECT_RULES.
@@ -133,36 +33,36 @@ And change severity:
     PROJECT_RULES=-squid:S4274;+csharpsquid:S110|max=7&severity=INFO;+csharpsquid:S3925&severity=INFO
 
 
-## Analysing projects
-
-### Typescript
-
-Create a file named "sonar-project.properties", on the same location as packages.json. Example:
-
-    sonar.host.url=http://mysonarqubeserver:9000
-    sonar.projectKey=myproject:master
-    sonar.projectName=myproject master
-    sonar.projectVersion=master-version
-    sonar.sourceEncoding=UTF-8
-    sonar.sources=src
-    sonar.tests=src
-    sonar.exclusions=**/node_modules/**,**/*.spec.ts,**/keycloak.js
-    sonar.test.inclusions=**/*.spec.*
-    sonar.typescript.lcov.reportPaths=coverage/lcov.info
-    sonar.scm.disabled=true
-
-Create the unit tests coverage file on the location specified at *sonar.typescript.lcov.reportPaths*. If you are using a standard Angular CLI project, you  can do that by executing:
-
-    ng test --single-run --code-coverage
+## Running with PostgreSQL via a docker composition
 
-Execute:
+Example docker-compose file:
 
-    npm i typescript
-    npm i sonar-scanner
-    ./node_modules/sonar-scanner/bin/sonar-scanner -Dproject.settings=sonar-project.properties
+    version: '3.7'
+    services:
+      www:
+        image: ictu/sonar:9.9.1
+        environment:
+          - SONAR_JDBC_URL=jdbc:postgresql://db:5432/sonar
+          - SONAR_JDBC_USERNAME=sonar
+          - SONAR_JDBC_PASSWORD=sonar
+        ports:
+          - 9000:9000
+        depends_on:
+          - db
+      db:
+        image: postgres:15.3
+        environment:
+          - POSTGRES_USER=sonar
+          - POSTGRES_PASSWORD=sonar
+          - POSTGRES_HOST_AUTH_METHOD=scram-sha-256
+          - POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256
+        volumes:
+          - /db/postgresql_data:/var/lib/postgresql/data
 
-## Create rules txt file from SonarQubes quality profile backup (xml)
 
-In order to make import of existing profiles easier, there is an XSLT transformation file provided: profile_backup_transform.xslt
+If the default SonarQube admin password has not yet been changed and `SONARQUBE_PASSWORD` is provided, the startup script will try to change the SonarQube default password to the one provided.
+Alternatively, the `SONARQUBE_TOKEN` can be used as admin credential instead of the `SONARQUBE_USERNAME` / `SONARQUBE_PASSWORD` combination.
 
-Go to profiles page in your SonarQube, backup a profile to an xml file and transform it.
+The Sonar start script waits for the database to become available (only when using PostgreSQL).
+`DB_START_TIMEOUT` (default: 60 seconds) defines how long the script will wait for the database to become available before exiting.
+Similarly `SONAR_START_TIMEOUT` (default: 600 seconds) defines how long the script should wait for Sonar to start up.
diff --git a/rules/cs.txt b/rules/cs.txt
@@ -1,9 +1,11 @@
 +types=SECURITY_HOTSPOT,VULNERABILITY # Enable these types by default
-+csharpsquid:S104            # NCSS; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#long-units-from-sonarqube)
-+csharpsquid:S107|max=5      # Too many parameters; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#many-parameters-from-sonarqube)
-+csharpsquid:S125            # Commented code; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#commented-out-code-from-sonarqube)
++csharpsquid:S104            # NCSS; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++csharpsquid:S107|max=5      # Too many parameters; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#many-parameters)
++csharpsquid:S125            # Commented code; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#commented-out-code)
 +csharpsquid:S134            # Depth of nesting; NOT used by Quality-time
 +csharpsquid:S1067           # Too complex expression; NOT used by Quality-time
-+csharpsquid:S1309           # Violation suppression; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#suppressed-violations-from-sonarqube)
-+csharpsquid:S1541|max=10    # Methods and properties too complex; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#complex-units-from-sonarqube)
++csharpsquid:S109            # Magic numbers; NOT used by Quality-time
++csharpsquid:S138|max=20     # Methods with too many lines; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++csharpsquid:S1309           # Violation suppression; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
++csharpsquid:S1541|maximumFunctionComplexityThreshold=10    # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#complex-units)
 #end please ensure every rule ends with a new line character
diff --git a/rules/java.txt b/rules/java.txt
@@ -1,12 +1,12 @@
 +types=SECURITY_HOTSPOT,VULNERABILITY # Enable these types by default
-+java:S1541|Threshold=10  # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#complex-units-from-sonarqube)
-+java:NoSonar             # Used by Quality-time to report on suppressed violations (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#suppressed-violations-from-sonarqube)
-+java:S1309               # Used by Quality-time to report on suppressed violations (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#suppressed-violations-from-sonarqube)
-+java:S1310               # Used by Quality-time to report on suppressed violations (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#suppressed-violations-from-sonarqube)
-+java:S1315               # Used by Quality-time to report on suppressed violations (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#suppressed-violations-from-sonarqube)
++java:S1541|Threshold=10  # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#complex-units)
++java:NoSonar             # Used by Quality-time to report on suppressed violations (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
++java:S1309               # Used by Quality-time to report on suppressed violations (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
++java:S1310               # Used by Quality-time to report on suppressed violations (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
++java:S1315               # Used by Quality-time to report on suppressed violations (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
 +java:S1067               # Expression too complex; NOT used by Quality-time
 +java:S109                # Magic numbers; NOT used by Quality-time
-+java:S138|max=20         # Methods with too many lines; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#long-units-from-sonarqube)
-+java:S107|max=5          # Too many parameters; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#many-parameters-from-sonarqube)
-+java:S125                # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#commented-out-code-from-sonarqube)
++java:S138|max=20         # Methods with too many lines; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++java:S107|max=5          # Too many parameters; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#many-parameters)
++java:S125                # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#commented-out-code)
 #end please ensure every rule ends with a new line character
diff --git a/rules/js.txt b/rules/js.txt
@@ -1,9 +1,11 @@
 +types=SECURITY_HOTSPOT,VULNERABILITY                    # Enable these types by default
-+javascript:S1541|maximumFunctionComplexityThreshold=10  # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#complex-units-from-sonarqube)
++javascript:S1541|maximumFunctionComplexityThreshold=10  # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#complex-units)
 +javascript:S134                                         # NOT used by Quality-time
 +javascript:S1067                                        # Expression too complex; NOT used by Quality-time
 +javascript:S106                                         # Console logging should not be used; NOT used by Quality-time
-+javascript:S107|maximumFunctionParameters=5             # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#many-parameters-from-sonarqube)
-+javascript:S125                                         # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#commented-out-code-from-sonarqube)
++javascript:S107|maximumFunctionParameters=5             # Too many parameters; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#many-parameters)
++javascript:S109                                         # Magic numbers; NOT used by Quality-time
++javascript:S138|max=20                                  # Methods with too many lines; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++javascript:S125                                         # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#commented-out-code)
 # Missing: NoSonar, NCSS, Parameters
 #end please ensure every rule ends with a new line character
diff --git a/rules/kotlin.txt b/rules/kotlin.txt
@@ -1 +1,4 @@
-+types=SECURITY_HOTSPOT,VULNERABILITY # Enable these types by default
++types=SECURITY_HOTSPOT,VULNERABILITY                    # Enable these types by default
++kotlin:S1067                                            # Expression too complex; NOT used by Quality-time
++kotlin:S138|max=20                                      # Methods with too many lines; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++kotlin:S107|Max=5                                       # Too many parameters; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#many-parameters)
diff --git a/rules/py.txt b/rules/py.txt
@@ -1,7 +1,9 @@
 +types=SECURITY_HOTSPOT,VULNERABILITY                             # Enable these types by default
 +python:S104                                                      # Too many lines of code in file; NOT used by Quality-time
-+python:S107|max=5                                                # Too many parameters; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#many-parameters-from-sonarqube)
-+python:S125                                                      # Commented code; used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#commented-out-code-from-sonarqube)
++python:S107|max=5                                                # Too many parameters; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#many-parameters)
++python:NoSonar                                                   # Used by Quality-time to report on suppressed violations (https://quality-time.readthedocs.io/en/latest/reference.html#suppressed-violations)
++python:S125                                                      # Commented code; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#commented-out-code)
 +python:S134                                                      # Too deep nesting; NOT used by Quality-time
-+python:FunctionComplexity|maximumFunctionComplexityThreshold=10  # Used by Quality-time (https://github.com/ICTU/quality-time/blob/master/docs/METRICS_AND_SOURCES.md#complex-units-from-sonarqube)
++python:S138|max=20                                               # Methods with too many lines; used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#long-units)
++python:FunctionComplexity|maximumFunctionComplexityThreshold=10  # Used by Quality-time (https://quality-time.readthedocs.io/en/latest/reference.html#complex-units)
 #end please ensure every rule ends with a new line character