EEG Speller is an application desgined to give users the ability to spell words without a physical keyboard. Brain signals are read from an EEG headset and are then decoded using machine-learning to classify the particular character the user wants to select. This application specifically focuses on motor-imagery based speller and aims to deliver a novel graphical-user interface that would improve the performance and speed of character selection.
The GUI consists of an inner and outer circle that has three layers. Each layer is spliced into 4 parts or subsections (tubes). The inner layers have 5 letters in each subsection resulting in a total of 20 letters in the inner layer. The outer layer also consists of 5 letters in each subsection. The purpose of the outer layer is to provide the users easier access to characters that they will most likely select. These characters are generated based on our text-generation NLP model.
- GUI Starts from the upper right subsection and rotates every 5 seconds
- Rotation will continue between the subsections until the user chooses a particular subsection through a right-imagined movement (motor-imagery task)
- When the subsection is selected, the GUI will now focus on the characters along the wedge and rotate through the characters one by one
- The user will then pick the character through a right imagined movement (motor-imagery task)
- After the user has picked a character, 5 characters will appear in the outer layer of the same subsection. These 5 characters are the most likely characters to be selected based on our NLP model. The user can then easily move to the outer layer in one step as opposed to using multiple steps to pick a character in the inner layer.
- Note that a left-imagined movement is dedicated to reverting a decision
Characters selected are saved in a typing window next to GUI window.
This GUI consists of four 3x3 character matrices, totaling 36 characters. Each submatrix flashes at its own frequency, recognized by the SSVEP signal, while there is identical pattern of these flashing characters through each individual matrix, recognized by the P300 potential signal.
The GUI follows this pattern:
- The top left character begins flashing at its submatrix's respective frequency.
- After X ms, the next character begins to flash at the same frequency as the previous. There will be two characters flashing at once per submatrix.
- This pattern continues until the end of the round.
The user's instruction is to locate their desired character on the supermatrix and notice when it begins to flash. It should then get selected once that round is over.
Chao-Li Wei, Eugene Kim, Andrew Clinton Wong, Joelle Faybishenko, Nova Lee, Quinton Ramaswamy, Simon Fei, Bailey Man