Skip to content

Latest commit

 

History

History
137 lines (111 loc) · 8.83 KB

ARCHITECTURE.md

File metadata and controls

137 lines (111 loc) · 8.83 KB
Raw

📐 Project architecture

Following diagram explain how metrics code is structured:

Architecture

🗂️ Project structure

This section explain how metrics is structured.

  • source/app/metrics/ contains metrics engine files
  • source/app/action/ contains GitHub action files
    • index.mjs contains GitHub action entry point
    • action.yml contains GitHub action templated descriptor
  • source/app/web/ contains web instance files
    • index.mjs contains web instance entry point
    • instance.mjs contains web instance source code
    • settings.example.json contains web instance settings example
    • statics/ contains web instance static files
      • app.js contains web instance client source code
      • app.placeholder.js contains web instance placeholder mocked data
  • source/plugins/*/ contains source code of plugins
    • README.md contains plugin documentation
    • metadata.yml contains plugin metadata
    • examples.yml contains plugin workflow examples
    • index.mjs contains plugin source code
    • queries/ contains plugin GraphQL queries
  • source/templates/*/ contains templates files
    • README.md contains template documentation
    • metadata.yml contains template metadata
    • examples.yml contains template workflow examples
    • image.svg contains template image used to render metrics
    • style.css contains style used to render metrics
    • fonts.css contains additional fonts used to render metrics
    • template.mjs contains template source code
  • tests/ contains tests
    • metrics.test.js contains metrics testers
    • source/app/mocks/ contains mocked data files
    • api/ contains mocked api data
      • axios/ contains external REST APIs mocked data
      • github/ contains mocked GitHub api data
    • index.mjs contains mockers
  • Dockerfile contains docker instructions used to build metrics image
  • package.json contains dependencies and command line aliases

🎬 Behind the scenes

This section explore some topics which explain globally how metrics was designed and how it works.

💬 Creating SVGs images on-the-fly

metrics actually exploit the possibility of integrating HTML and CSS into SVGs, so basically creating these images is as simple as designing static web pages. It can even handle animations and transparency.

Metrics are html

SVGs are templated through EJS framework to make the whole rendering process easier thanks to variables, conditional and loop statements. Only drawback is that it tends to make syntax coloration a bit confused because templates are often misinterpreted as HTML tags markers (<%= "EJS templating syntax" %>).

Images (and custom fonts) are encoded into base64 to prevent cross-origin requests, while also removing any external dependencies, although it tends to increase files sizes.

Since SVG renders differently depending on OS and browsers (system fonts, CSS support, ...), it's pretty hard to compute dynamically height. Previously, it was computed with ugly formulas, but as it wasn't scaling really well (especially since the introduction of variable content length plugins). It was often resulting in large empty blank spaces or really badly cropped image.

To solve this, metrics now spawns a puppeteer instance and directly render SVG in a browser environment (with all animations disabled). An hidden "marker" element is placed at the end of the image, and is used to resize image through its Y-offset.

Metrics marker

Additional bonus of using puppeteer is that it can take screenshots, making it easy to convert SVGs to PNG output.

💬 Gathering external data from GitHub APIs and Third-Party services

metrics mostly use GitHub APIs since it is its primary target. Most of the time, data are retrieved through GraphQL to save APIs requests, but it sometimes fallback on REST for other features. Octokit SDKs are used to make it easier.

As for other external services (Twitter, Spotify, PageSpeed, ...), metrics use their respective APIs, usually making https requests through axios and by following their documentation. It would be overkill to install entire SDKs for these since plugins rarely uses more than 2/3 calls.

In last resort, puppeteer is seldom used to scrap websites, though its use tends to make things slow and unstable (as it'll break upon HTML structural changes).

💬 Web instance and GitHub action similarities

Historically, metrics used to be only a web service without any customization possible. The single input was a GitHub username, and was composed of what is now base content (along with languages and followup plugin, which is why they can be computed without any additional queries). That's why base content is handled a bit differently from plugins.

As it gathered more and more plugins over time, generating a single user's metrics was becoming costly both in terms of resources but also in APIs requests. It was thus decided to switch to GitHub Action. At first, it was just a way to explore possibilities of this GitHub feature, but now it's basically the full-experience of metrics (unless you use your own self-hosted instance).

Both web instance and Action actually use the same entrypoint so they basically have the same features. Action just format inputs into a query-like object (similarly to when url params are parsed by web instance), from which metrics compute the rendered image. It also makes testing easier, as test cases can be reused since only inputs differs.

📦 Packages reference

Below is a list of used packages.