Skip to content

Latest commit

 

History

History
49 lines (38 loc) · 2.74 KB

README.md

File metadata and controls

49 lines (38 loc) · 2.74 KB
Raw

matomo_import

Purpose

The aim of the script provided is to pull data from a Matomo instance into a Postgres database, database that will then be usable in a data analysis tool. In our case, the software used after is Metabase, but it can suit all software accepting that format of database.

Getting started

Requirements

  • Python >= 3.8
  • Poetry
  • Strongly recommended : a virtual environment manager (the one used for this project is pew).
  • An admin access to your Matomo instance to retrieve the API key
  • A Postgres instance(either your local machine, or a remote installation)

How does it work

  • A python script will run API calls toward your Matomo instance, retrieving relevant data and fill the Postgres database.
  • You can export that file into your Metabase instance (or any other data analysis tool), setup the database, and write your questions.

Setup

  1. Optional : modify the config.yml file containing informations related to data you want to pull. The current one contains already many different informations that can be pulled out Matomo.

  2. Create a .env file containing all informations concerning parameters needed for the script to run. You can copy the .env.example file given in this repository.

    • base_url : the URL of your Matomo instance (e.g. : https://matomo.yoursite.com/, ⚠️ the ending slash is mandatory)
    • db_name : the name of the Postgres database that will contain all data
    • id_site : id of the site you want to pull information from. Available via Matomo
    • start_date : date from which you want to import data. Format YYYY-MM-DD, like 2021-11-23 as instance.
    • token_auth : the API token provided on your Matomo instance.
    • end_date (optional) : same format as start date. When set, the database will only contain data within the date range given.

  3. Run the following comand : poetry install to install relevant packages

  4. Run the following command to obtain your data : python -c "import main; main.exec()"

  5. Gather the database file/ database informations created and use it for your data analysis

Improve/contribute

Any help to improve this script is welcome ! Do not hesitate to write issues/PRs to improve that script, see the CONTRIBUTE.adoc file.

TL;DR:

  • write issues if you got problems or want to improve the project
  • do not hesitate to make PRs
  • validation workflow: be sure that test and syntax workflow works before asking for review

Next steps

  • config.yml file configuration
  • Add requirements.txt
  • Add minimal tests
  • Env variables for critical parameters
  • Adapt to obtain a postgres-data format
  • Update README for container-use