Feature - CSV Download Button

[sbelten]

Pull Request

This pull request implements the changes proposed on this issue

Please see the commits for detailed code. The code serves some comments, which help you understand the more technical & complex part of the implementation

Frontend Changes

The new design adds a download button on the right side of the search & filter bar

The size of the search bar has been decreased. You can see the change in this line

This was necessary for the following reason:

On smaller screens, the download button would be way off to the side, which makes the whole search bar look like a mess. You don't necessarily see any issues with this on the image above, as it was taken on a 27' inch monitor, in fact you can only see the whole search tools bar getting smaller.

The following picture was taken on a 14 inch screen laptop

The difference is very visible

I hope this change meets the standard, it's not something to break ones head over, but it should be left unmentioned

Backend changes - CSV Download feature

The feature was directly implemented inside scripts/index.js

The following functions have been added

async function fetch_repos(url)

function generate_schema

with inner function collect_unique_keys()
and inner function key_finalizer()

async function repos_to_csv()

arrow function flatten()

function description_formatting(entry)

function download_csv_file(csv, file_name)

They will be explained in enough detail here. Some more technical information is inside the source code

Fetch Repos

found here

let response = await fetch(url);
let data = await response.json();
return data;

This is very short, it's only used to retrieve the repos.json and basically serves as a wrapper function
It's only used once.

await fetch_repos(`${window.location.origin}/repos.json`);

The exact location of the repos.json (or any other json file) can vary. It may not even be a file stored on a server but rather an API call, so you may need to adjust this to your own needs

Generate Schema

found here

The tricky part of this feature was to generate any csv file for any json. What I noticed is that the entries inside repos.json vary one another. Some entries have keys inside _InnerSourceMetadata that other entries don't have

This made it hard to generate the headers for the CSV file, a data structure which absolutely needs to have the same keys / columns for each and every single entry.

I solved this with the help of the 2 functions collect_unique_keys and key_finalizer

You can read the more technical part inside the code itself, but here is a brief summary:
collect_unique_keys does exactly what the name suggests, it stores every single key and nested key that has not appeared before while iterating through all entries. By running key_finalizer at the end, we can return a JSON object, which serves as a schema of the repos with all possible keys.

Repos To CSV

found here

This is like the main function of the csv feature, as it's the one being called when clicking on the button and every other function listed here is called inside this function too.

It's not long either, but a pseudo code explanation is best fitted here

	fetch repo data to variable A
	generate the json template to variable B
	flatten the template, extract the keys and seperate it with semicolon

	iterate through the data ->
		flatten current iteration
		map each value to correct column, if data doesn't exist assign default value
		add to variable csv_rows
	iteration end

	seperate rows through new lines
	ddownload csv

Flatten

found here

This method flattens any object (here, the JSON object and seperates it with the .DOT notation
It's basically the json to csv Converter
Source Code was found on this github gist

I changed the behaviour on arrays -> The array is stored as a single value and not comma seperated by each element

Format Description

found here

This method is not used right now but should there be the rare case, where inside the description (or anything else actually) the semicolon character appears, then this method can help to replace it with a normal comma.
Leaving the semicolon would basically mess up the structure of the csv row and the data integrity

Download CSV File

found here

This is a basic function used to initiate the download process. It is straight forward and easy to understand.