Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a test suite of HTML files #7

Open
0xedward opened this issue Jan 3, 2021 · 4 comments
Open

Create a test suite of HTML files #7

0xedward opened this issue Jan 3, 2021 · 4 comments
Assignees
@0xedward @kelvinlin97
Labels
testing Testing existing features
Milestone
Initial Release (...

Comments

@0xedward
Copy link
Owner

0xedward commented Jan 3, 2021 •
edited
Loading

@0xedward 0xedward added the testing Testing existing features label Jan 9, 2021
@kelvinlin97
Copy link
Collaborator

https://docs.google.com/document/d/1WcbUEw5iyeHAOrs2M7SXS0F6CEVtk5Z6q80Y8cl7uN4/edit?usp=sharing
Comparison of different testing frameworks with sample code on setup

@0xedward
Copy link
Owner Author

0xedward commented Feb 1, 2021

Great work! So we need to have our testing framework run a browser in the background (not just executing a javascript function and checking the result), because DOMParser is a browser API.

Also another requirement is that we need to consider support for testing on multiple browsers (Edge, Chromium, Chrome, Safari, Firefox, etc.), because DOMParser may behave differently on each browser and we don't want to have to use multiple testing frameworks to test the browsers we are planning on supporting.

Ideally we use something fast and lightweight, with just the features we need and nothing more, to write end-to-end tests with.

https://blog.checklyhq.com/cypress-vs-selenium-vs-playwright-vs-puppeteer-speed-comparison/

@kelvinlin97
Copy link
Collaborator

kelvinlin97 commented Feb 3, 2021 •
edited
Loading

Gotcha, my bad I misunderstood which testing frameworks to look at. Here's my writeup of the two blogs comparing Selenium, Playwright, Puppeteer, and Cypress. I wrote this at the top of the doc but for our use case, I think Selenium would be the best option. Below is a quick summary of each testing frameworks strengths.

Browser-support:

  1. Cypress: Chrome family browsers (including Chromium) and Firefox
  2. Selenium: IE, Google Chrome (12+), Safari, Opera, Firefox **Most users use Selenium
  3. Playwright: Webkit (Safari), Chrome, and Firefox
  4. Puppeteer: Specific to Chrome

Speed:

  1. Less back-end: Puppeteer
  2. More back-end + full live website: Playwright
  3. Selenium comes third (1.5x slower than other testing frameworks on average)

Local testing focus:

Cypress (considerably slower on live websites e2e testing)

@0xedward 0xedward added this to the Initial Release (1.0.0) milestone Feb 21, 2021
@0xedward
Copy link
Owner Author

Test cases for use to include:

  • comment: <!-- This is a comment-->
  • doctype: <!DOCTYPE html>
  • missing closing tags: <html><script>alert(1)</script><body><p>text
  • multiple head tags: <html><head></head><head></head></html>
  • root: <html></html>
  • tags with attributes: <html><head><meta description="test attribute"></head></html>

We should also include MathML, SVG and table tags in our tests because of weird side effects they cause with the HTML parser

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@0xedward 0xedward

@kelvinlin97 kelvinlin97

Labels
testing Testing existing features
Projects
None yet
Milestone
Initial Release (1.0.0)
Development

No branches or pull requests
2 participants
@0xedward @kelvinlin97