Code

Parser script for the chatbot input#675

Code

Open

EwDa291 wants to merge 161 commits intohpcugent:mainhpcugent/vsc_user_docs:mainfrom EwDa291:chatbot_parser_codeEwDa291/vsc_user_docs:chatbot_parser_codeCopy head branch name to clipboard

+2,343-1

Commits

Commits on Aug 8, 2024

• initial commit

  EwDa291

  committed
  1ebc363

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  34df842

• some cleanup

  EwDa291

  committed
  10edb20

Commits on Aug 9, 2024

• used jinja to replace macros

  EwDa291

  committed
  85a93ec

• adapt if-mangler to accommodate for nested if-clauses

  EwDa291

  committed
  dfff5fa

• adapt the parser to take all files as input, not all files get parsed successfully yet

  EwDa291

  committed
  649ddec

• adapt the parser to take all files as input, not all files get parsed successfully yet

  EwDa291

  committed
  2116d6e

• small update, not important

  EwDa291

  committed
  159aa62

• change to the templates

  EwDa291

  committed
  75765e5

• change to accommodate for more nested if-clauses

  EwDa291

  committed
  57d9cfe

• Delete scripts/HPC chatbot preprocessor/start_checker.py

  Show description for 75d345b
  EwDa291

  authored
  75d345b

Commits on Aug 12, 2024

• make sure files with duplicate names between normal files and linux-tutorial are both read and saved properly

  EwDa291

  committed
  ff7a9fc

• Merge branch 'chatbot_parser' of https://github.com/EwDa291/vsc_user_docs into chatbot_parser

  EwDa291

  committed
  47a33b7

• fixed the problem of some files being written in reST instead of markdown

  EwDa291

  committed
  7d279d6

• some small fixes

  EwDa291

  committed
  8047572

Commits on Aug 13, 2024

• remove try-except-structure

  EwDa291

  committed
  7d1c5ed

• collapse all code into one file

  EwDa291

  committed
  984b0cd

• Rename file

  EwDa291

  committed
  8f5eeaa

• cleanup repository

  EwDa291

  committed
  2b97b7a

• Rename directory

  EwDa291

  committed
  b595301

• add a main function

  EwDa291

  committed
  90c8ab7

• make file paths non os-specific

  EwDa291

  committed
  b8ae706

• use docstrings to document the functions

  EwDa291

  committed
  b751497

• rewrite the if-mangler to make it more readable

  EwDa291

  committed
  0f8eb5d

• got rid of most global variables

  EwDa291

  committed
  9938e92

• fixed some issues with if statements

  EwDa291

  committed
  508b22c

• fixed some issues with if statements

  EwDa291

  committed
  a25ce2d

• got rid of all global variables

  EwDa291

  committed
  80d0535

Commits on Aug 14, 2024

• small changes to make file more readable

  EwDa291

  committed
  9163a75

• codeblocks, tips, warnings and info reformatted

  EwDa291

  committed
  1dcffc1

• small optimisations

  EwDa291

  committed
  4d7fbdb

• small optimisations

  EwDa291

  committed
  671f7f3

• initial commit

  EwDa291

  committed
  e5c39bd

• added requirements

  EwDa291

  committed
  c6492fc

• added requirements and usage info

  EwDa291

  committed
  aff8198

• minor changes to the print statements

  EwDa291

  committed
  a981002

Commits on Aug 16, 2024

• reworked function to take care of html structures

  EwDa291

  committed
  1f3b343

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  b6388d3

• filter out images

  EwDa291

  committed
  48cad97

• get rid of backquotes, asterisks, pluses and underscores used for formatting

  EwDa291

  committed
  df58f23

• dump to json files instead of txt files

  EwDa291

  committed
  c423e07

• cleaned up parser with macros

  EwDa291

  committed
  2c333fe

• cleaned up parser with macros

  EwDa291

  committed
  ce52352

• cleaned up parser with macros

  EwDa291

  committed
  5db34af

Commits on Aug 19, 2024

• Update README.md

  EwDa291

  authored
  4226d28

• Update README.md

  EwDa291

  authored
  d730a26

• added section about restrictions on input files

  EwDa291

  committed
  f3182e3

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  aee54de

• adapted section about restrictions on input files

  EwDa291

  committed
  675bec5

• adapted section about restrictions on input files

  EwDa291

  committed
  f1e58ef

• Merge branch 'chatbot_parser' of https://github.com/EwDa291/vsc_user_docs into chatbot_parser

  EwDa291

  committed
  2bf1075

• change variables to be lowercase

  EwDa291

  committed
  a168509

• take out some copy pasting

  EwDa291

  committed
  09b86c9

• added warning about long filepaths

  EwDa291

  committed
  f95b99e

• fixing typos

  EwDa291

  committed
  06bb7b9

• take out copy pasting

  EwDa291

  committed
  2f3e5b3

Commits on Aug 20, 2024

• first draft version of the restructured script to accommodate for the new file format

  EwDa291

  committed
  0c4dbe8

• added support to filter out collapsable admonitions

  EwDa291

  committed
  38c4572

• attempt at fix for problems with jinja include, not working yet

  EwDa291

  committed
  5cbd653

Commits on Aug 21, 2024

• fixed an issue with jinja templates

  EwDa291

  committed
  0e6f8b2

• added docstrings to new functions

  EwDa291

  committed
  cd77837

• only add necessary if-statements in front of non-if-complete sections

  EwDa291

  committed
  98eb695

• fixed some more jinja problems

  EwDa291

  committed
  27457e3

• implemented extra test to make sure generic files dont accidentally get flagged as os-specific

  EwDa291

  committed
  bb72287

• make sure empty os-specific files are not saved

  EwDa291

  committed
  67cb19e

• clean up unused code

  EwDa291

  committed
  cf9834a

• introduce more macros

  EwDa291

  committed
  da32459

• reintroduce logic to remove unnecessary directories

  EwDa291

  committed
  093200b

• added functionality to include links or leave them out

  EwDa291

  committed
  5d0ffe9

• added functionality to include links or leave them out

  EwDa291

  committed
  a3e34a9

• adapt filenames to allow for splitting on something other than subtitles

  EwDa291

  committed
  7c6154b

• making some changes to prepare to add paragraph level splitting tomorrow

  EwDa291

  committed
  8d5b50d

• making some changes to prepare to add paragraph level splitting tomorrow

  EwDa291

  committed
  0c10376

• making some changes to prepare to add paragraph level splitting tomorrow

  EwDa291

  committed
  f8ee860

• adapted the parsing script to allow for testing in a semi-efficient way

  EwDa291

  committed
  6533733

• added test for make_valid_title

  EwDa291

  committed
  2e7a00f

• removed useless lines from testscript

  EwDa291

  committed
  f5e0579

Commits on Aug 22, 2024

• First attempt at splitting in paragraphs (need for other fixes for title-based-split first

  EwDa291

  committed
  6757b4f

• make two functions for different ways of dividing the text

  EwDa291

  committed
  6d9558d

• added docstrings to new functions

  EwDa291

  committed
  2c7025a

• update test for valid titles

  EwDa291

  committed
  ae99bb9

• fixed problem with splitting os-specific text (metadata not fixed yet)

  EwDa291

  committed
  084b421

• fix for metadata of os-specific sections

  EwDa291

  committed
  cf7f5f0

• clean up temporary version

  EwDa291

  committed
  b7c10d3

• added command line options for custom macros

  EwDa291

  committed
  4a441f3

• small fix to macros

  EwDa291

  committed
  662134f

• clean up test for valid title

  EwDa291

  committed
  05eab4a

• add a test for write_metadata

  EwDa291

  committed
  b85a8fb

Commits on Aug 23, 2024

• added functionality to split on paragraphs

  EwDa291

  committed
  39a3c99

• clean up

  EwDa291

  committed
  af9e6cc

• clean up

  EwDa291

  committed
  f4163a7

• further clean up and added shebang

  EwDa291

  committed
  833f964

• clean up

  EwDa291

  committed
  79b1a56

• added test for if mangler

  EwDa291

  committed
  cec154c

• clean up

  EwDa291

  committed
  2f4a277

• clean up customizable options

  EwDa291

  committed
  cd0c8eb

Commits on Aug 26, 2024

• further adapt the script to be able to test it

  EwDa291

  committed
  3be262a

• make changes to usage in command line to be more intuitive

  EwDa291

  committed
  1d32aab

• first revised version of the README

  EwDa291

  committed
  5902c96

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  6f97d5f

• added docstring to main function

  EwDa291

  committed
  6e48800

• include chatbot_prepprocessor

  EwDa291

  committed
  0bc440b

• added options for source and destination directories

  EwDa291

  committed
  e6e6023

• cleanup

  EwDa291

  committed
  a6d99d9

• cleanup

  EwDa291

  committed
  2be834f

• cleanup

  EwDa291

  committed
  532543a

• relocate test files

  EwDa291

  committed
  107464e

• update arguments of if mangler

  EwDa291

  committed
  dd64381

• relocate full test files

  EwDa291

  committed
  ef3fd58

• Revert "update arguments of if mangler"

  Show description for 4d7db8f
  EwDa291

  committed
  4d7db8f

• Revert "relocate full test files"

  Show description for df9bac5
  EwDa291

  committed
  df9bac5

• update test to adapt to new arguments in if mangler

  EwDa291

  committed
  631d9e9

• relocated full test files

  EwDa291

  committed
  c6e600d

• Rename test_paragraph_split_1.md to test_paragraph_split_1_input.md

  EwDa291

  authored
  d1c6194

• Rename test_title_split_1.md to test_title_split_1_input.md

  EwDa291

  authored
  695ffd6

• smal fix

  EwDa291

  committed
  af4832b

• test text for paragraph split

  EwDa291

  committed
  8805c8c

• start of a fix for double title problem, not done yet

  EwDa291

  committed
  a265ffd

Commits on Aug 27, 2024

• Fix for double title bug when splitting on paragraph

  EwDa291

  committed
  6c2a61c

• Fix bug for empty linklist in metadata

  EwDa291

  committed
  ed08879

• fix bug where too many directories were sometimes created

  EwDa291

  committed
  176af13

• test of full script, test files not ready to be pushed yet

  EwDa291

  committed
  d4ceac8

• updated requirements.txt

  EwDa291

  committed
  815a863

• updated docstring in main function

  EwDa291

  committed
  d15469f

• add support for comments for the bot to be included in the source files

  EwDa291

  committed
  daa6b36

• changed the default for min paragraph length

  EwDa291

  committed
  4c19f44

• added test files for full script test

  EwDa291

  committed
  9a6ff58

• small fix for double title bug

  EwDa291

  committed
  56543f0

• added examples of output of the script when splitting on paragraphs with a min_paragraph_length of 683

  EwDa291

  committed
  52a3861

• fix for issue with html links

  EwDa291

  committed
  692e77b

• fix for issue with html links

  EwDa291

  committed
  7f493a1

• fix for issue with relative links to the same document

  EwDa291

  committed
  0e34396

• added test for replace_markdown_markers

  EwDa291

  committed
  fa00044

• fix to small inconsistency in metadata

  EwDa291

  committed
  b3952b2

• added test for insert_links

  EwDa291

  committed
  73072bf

Commits on Aug 28, 2024

• make sure paragraphs only include full lists

  EwDa291

  committed
  3161309

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  7d4d7f9

• adapted to the new source files

  EwDa291

  committed
  3407be3

• add source-directory to metadata and verbose mode

  EwDa291

  committed
  6d04bbc

• added verbose mode

  EwDa291

  committed
  f33cfb3

• Merge branch 'hpcugent:main' into chatbot_parser

  EwDa291

  authored
  1c389d7

Commits on Aug 29, 2024

• Added limitation on lists

  EwDa291

  committed
  3227f19

• fix for non os-specific if-statement not being recognised

  EwDa291

  committed
  67aed53

• new test for links

  EwDa291

  committed
  9e297b1

• new test to make sure lists are kept as one section

  EwDa291

  committed
  b6b8610

• updated test_file for list test

  EwDa291

  committed
  57a2139

Commits on Aug 30, 2024

• dropped <> around links and started new function to calculate length of paragraphs

  EwDa291

  committed
  170a10c

• removed parsed mds

  EwDa291

  committed
  04efff6

• Changed paragraphs to decide length based on tokens instead of characters

  EwDa291

  committed
  1ef1f10

• Changed paragraphs to decide length based on tokens instead of characters

  EwDa291

  committed
  621c0a3

• Changed paragraphs to decide length based on tokens instead of characters

  EwDa291

  committed
  adf364d

• Merge branch 'hpcugent:main' into chatbot_parser_code

  EwDa291

  authored
  2c6e3c4

• removing unnecessary files

  EwDa291

  committed
  32b8b74

• Revert "removing unnecessary files"

  Show description for 445f7ee
  EwDa291

  committed
  445f7ee

• change filenames

  EwDa291

  committed
  d96aeba

• Updated all tests to incorporate the latest changes

  EwDa291

  committed
  bec9a63

• change working directory of test lists

  EwDa291

  committed
  886da19

• change directory in metadata to linux version

  EwDa291

  committed
  b354bb2

• change directory in metadata to linux version

  EwDa291

  committed
  87e5575

• add pytest to requirements

  EwDa291

  committed
  8f39e08

• change test for write_metadata

  EwDa291

  committed
  d6c33e7