Skip to content

Test Machines

Jump to bottom
Frédéric Wang edited this page Oct 15, 2012 · 4 revisions

Test Machines

Overview

It is highly recommended to read the MathJax-test documentation to understand how the Test Framework is working. Our test infrastructure contains several installations of the test framework and they are managed the same way. For each installation we have a "main" machine, that can be used to control the test on machines of the same local network (including the main machine itself). The test results are stored stored on the main machine and can be made public. Here is the how our infrastructure is organized:

  • Public test results are stored on Amazon S3.
  • A test framework is installed on Amazon EC2 and is only used during the test period. A "micro" Linux instance is continuously running during that period and contains an Apache installation to serve public Web pages with the MathJax test suite, QA tools and documentation. More powerful Linux or Windows instances can be executed when we want to run the tests. The "micro" instance controls the execution on the other machines, collects the results and can be used to publish these results on Amazon S3.
  • A DSI Mac machine located in Long Beach, where the test framework is installed.
  • A DSI Mac machine located in St. Paul. It contains two partitions (Mac and Windows) with a test framework installed on each of them. The Windows partition also contains a virtual Linux machine which can be controlled by the Windows machine.
  • The member of the MathJax team may also install the test framework on their own machine.

As a rule of thumb, use the test framework on your local machine for simple testing: you can either verify the tests manually or automatically. For intensive testing, use the Amazon EC2 instance whose remote connection is more reliable, where you can run several machines of different CPU powers and from which you can directly publish the results to Amazon S3. Some drawbacks are that the instances do not have fixed IPs and we have to pay the service (running instances, server communications etc). Finally, use the DSI machine for Mac testing or when we are outside the test period. Note that the remote interface is less convenient and slower and the machines may not always be available. Also, the tests results won't be gathered at the same place.

Maintenance

MathJax, the browsers, Selenium, the test framework etc are always Ideally, we would like to use only EC2 instances created from a few disk images so that we only need to maintain these disk images. That's not currently the case so we have to keep each test machine up-to-date. Fortunately, the browsers, operating systems and test framework generally have tools to do this task more or less easily. Here is an overview:

  1. Updating the programs on the operating system
  • On Linux, use the command "sudo apt-get update && sudo apt-get upgrade"
  • On Mac, use the command "sudo port selfupdate" to update the MacPorts packages. You may also need to use the graphical package managers of the system or of third-party tools.
  • On Windows use Windows update or other graphical package managers of third-party tools.
  1. Updating programs used by test machines
  • Browsers: This is done by the system package manager on Linux and on Windows Most browsers now have their own integrated automatic updates.
  • Selenium Server: if the test machine has the test framework installed, you can use the "make downloadSeleniumServer" command (you may need to override the version specified in custom.cfg). Otherwise, you must get it manually from the Selenium download page.
  • Browser drivers: Only for Internet Explorer and Chrome. The Internet Explorer driver is available on the Selenium download page while the Chrome driver can be found on the ChromeDriver download page
  • Java: This is done by the system package manager on Linux and there should be a graphical updater on other platforms.
  • STIX and MathJax fonts: the STIX fonts should be kept up-to-date by the system package manager. Otherwise you can find package and installers on Mozilla MathML Fonts page
  • MathPlayer: Updates should be checked automatically when you open a page with MathML in Internet Explorer.
  1. Updating the test framework
  • Open a UNIX terminal and go in MathJax-test/
  • Backup the default configuration: "cp default.cfg default-.cfg"
  • Get the update: "git pull"
  • Verify whether the format of the configuration file has changed: "diff default.cfg default-.cfg". If so, then you will have to add/remove configuration options in custom.cfg accordingly. Hopefully, this format won't change too often.
  • "make config"
  • You may also want to update the documentation ("make doc"), the selenium driver ("sudo make updateSeleniumDriver") or the MathJax branches ("make updateMathJaxBranches"). If necessary, execute "make config" again.
  1. Saving the AMI. For Amazon EC2 machines, your changes will be lost if you terminate an instance. To preserve the state of the Linux or Windows machines, do Instances Actions => Create an AMI file. After a moment, a new AMI image and Snapshot will be created. You can now terminate the instance and delete the former AMI and Snapshot. Be careful not to delete the wrong Snapshot!

See also

(Need to be updated and expanded)

Amazon-EC2 and DSI-test-machine

MathJax Wiki

Clone this wiki locally