Skip to content

Latest commit

 

History

History
180 lines (161 loc) · 6.29 KB

README.md

File metadata and controls

180 lines (161 loc) · 6.29 KB

The Guardian Project

An Open Source Software (OSS) Security Inspector made with ❤️ by

Live Instance: theguardianproject.ml

Features

  • Trust Score
    EDF probability distribution applied on data scraped from GIT API to generate an estimate trust score of a repository.
  • Git Analysis
    Recursively walk and analyze dependencies of a GIT repository to check for vulnerabilities and outdated dependencies.
  • PyPI Analysis
    Interact with PyPI API to fetch dependencies of a package and analyze for CVEs.
  • NPM Analysis
    Interact with NPM API to fetch dependencies of a package and identify vulnerabilities and outdated dependencies.
  • Sensitive Info
    Identify hardcoded secrets, tokens, passwords, emails from a repository.

Pre-Requisites

You need a LINUX ENVIRONMENT to run this project. The web application, however, can be accessed from any device.

The following should be installed for this project to work:

  • Python3
  • Node
  • NPM
  • pm2

GitHub Tokens

  • This project needs GitHub Tokens to work.
  • In the .env file in the project's folder, you can enter as many tokens as you want in the format: 
    GITHUB_TOKEN$i=<ENTER_TOKEN_HERE>
    where $i is a number
  • You can start the project with zero tokens as well, but for using the "Trust Score" feature for repositories with a lot of stars, you will need tokens.
  • There is no upper limit to the number of tokens you can add.

NOTE: For production mode, you need to build your client atleast once, before running.

Auto Start

A bash script has been provided for automatically running the project.

  • Development Mode

    To run in development mode, in the project folder, run:

    chmod +x run.sh
./run.sh dev

    The server will start on port 5000, and the client on port 3000. You can access the website by visiting: http://localhost:3000

  • Production Mode

    To run in production mode, in the project folder, run:

    chmod +x run.sh

    If you want to build the client folder as well, run:

    ./run.sh prod

    If you do not wish to build the client folder, run:

    ./run.sh prod --no-build

    Access the website from http://localhost:5000

    You can also specify a custom port using:

    ./run.sh prod 1025
./run.sh prod --no-build 1025

    This will start the server on port 1025 (http://localhost:1025).
    (The specified port must be greater than or equal to 1024)
    By default, the port is 5000 if you do not specify anything.

Manual Start

  • Development Mode

    To start the project in the development mode manually, follow these steps:

    • Go to the project folder, and run: 
      pip3 install -r requirements.txt
npm install
    • Go the the client folder, and run: 
      npm install
    • In the project folder, run: 
      NODE_ENV=development npm run dev
      (You need not set the NODE_ENV environment variable to 'development', just make sure that it is not set to 'production')

    The server will start on port 5000, and the client on port 3000.
    You can access the website by visiting: http://localhost:3000

  • Production Mode

    To start the project in production mode, follow these steps:

    • Go the project folder, and run: 
      pip3 install -r requirements.txt
npm install
    • Go to the client folder:
      If you want to build the client, first delete any existing build folder: 
      rm -rf ./client/build
      and then run: 
      npm install
npm run build
      Else, you can skip this part.
    • In the project folder, run: 
      NODE_ENV=production pm2 start server.js --name "TheGuardianProject"

    Access the website from http://localhost:5000

    If you wish to specify a port as well, then run:

    NODE_ENV=production PORT=1025 pm2 start server.js --name "TheGuardianProject"

    Access the website from http://localhost:1025
    (The specified port must be greater than or equal to 1024)

Stop Production Mode

To stop the project running in production mode, run:

chmod +x run.sh
./run.sh stop

To stop it manually, enter these two commands:

pm2 stop TheGuardianProject
pm2 delete TheGuardianProject

Common Issues

  • The first time you use GIT or NPM in the website, it may take a long time to get the results or it might give an error. This is only a one-time thing because the corresponding python module (js2py) requires some time to initialise. In order to not face this issue, after starting the project, run this command:

    python3 -c "import js2py; js2py.require('compare-versions')"

    This will initialise the module.
    (If you are using the run.sh script, then you need not do this. The script automatically initialises this module.)

  • Sometimes, when running in development mode, you might get an error like:

    options.allowedHosts[0] should be a non-empty string

    To fix this, set the environment variable DANGEROUSLY_DISABLE_HOST_CHECK to true by running:

    export DANGEROUSLY_DISABLE_HOST_CHECK=true

Dataset Collection

The script used for generating the dataset for trust score is also included. To create your own dataset:

  • Head over to the create_dataset folder inside the modules directory.
  • In the files folder, in the links.txt file, enter the URLs of the repository which you want to include in the dataset.
  • In the project's home folder, run: 
    python3 main.py create_dataset
    or you can also use the run.sh script: 
    chmod +x run.sh
./run.sh dataset

Note that you need to have plenty of tokens in the .env file for this function.

Clean

The run.sh script contains a clean feature as well, for cleaning any remaining cloned repositories. To use this, run:

chmod +x run.sh
./run.sh clean