-
Notifications
You must be signed in to change notification settings - Fork 14
Keyboarding
Keyboard events are a system provided by Apple that allows you to
simulate keyboard input. The API for this in the ApplicationServices
framework, but there is an analogue in the Acessibility APIs which
has the additional option of directing the input to a specific application.
Using accessibility actions and setting attributes you can already perform most of the interactions that would be possible with the keyboard simulation. However, there are some things that you will need to, or it will just make more sense to, simulate keyboard input. For example, to make use of hot keys you would have to add extra actions or attributes to a control in the application; that would be more work, possibly prone to error, than simply simulating the hot key from outside the application. In other situations you may specifically want to test out keyboard navigation and so actions would not be a good substitute. It may be that the APIs that AXElements provides for typing just make more sense when writing tests or scripts.
The {Accessibility::DSL} mix-in exposes keyboard events through the
type method. A simple example would look like this:
type "Hello, #{ENV['USER']}! How are you today?\n"
And watch your computer come to life! The type command takes an
additional optional parameter that we'll get to later. The first
parameter is just a string that you want AXElements to type out. How
to format the string should be obvious for the most part, but some
things like the command key and arrows might not be so obvious.
Letters and numbers should be written just as you would for any other string. Any of the standard symbols can also be plainly added to a string that you want to have typed. Here are some examples:
type "UPPER CASE LETTERS"
type "lower case letters"
type "1337 message @/\/|) 57|_||=|="
type "A proper sentence can be typed out (all at once)."
Things like newlines and tabs should be formatted just like they would in a regular string. That is, normal string escape sequences should "just work" with AXElements. Here are some more examples:
type "Have a bad \b\b\b\b\b good day!"
type "First line.\nSecond line."
type "I \t like \t to \t use \t tabs \t a \t lot."
type "Explicit\sSpaces."
Unfortunately, there is no built in escape sequence for deleting to
the right or pressing command keys like F1. AXElements defines some
extra escape sequences in order to easily represent the remaining
keys.
These custom escape sequences shoud start with two \ characters,
as in this example:
type "\\F1"
A custom escape sequence should terminate with a space or the end of the string, as in this example:
type "\\PAGEDOWN notice the space afterwards\\PAGEUP but not before"
The full list of supported custom escape sequences is listed in
{Accessibility::StringParser::CUSTOM}. Some escapes have an alias,
such as the right arrow key which can be escaped as "\\RIGHT" or as
"\\->".
To support pressing multiple keys at the same time (i.e. hot keys), you
must start with the custom escape sequence for the combination and
instead of ending with a space you should put a + character to chain
the next key. The entire sequence should be ended with a space or
nil. Some common examples are opening a file or quitting an
application:
type "\\COMMAND+o"
type "\\CONTROL+a Typing at the start of the line"
type "\\COMMAND+\\SHIFT+s"
You might also note that CMD+SHIFT+s could also be:
type "\\COMMAND+S"
Since a capital S will cause the shift key to be held down.
One caveat with hot keys is that you cannot use "\\COMMAND+ " to
represent command and space combination, you will need to use
"\\COMMAND+\s" instead.
In order make sure that certain sequences of characters are properly escaped, it is recommended to simply always use double quoted strings.
The second argument to the type command can be an {AX::Application}
object. If you do not include the argument, the events will be posted
to the system, which usually means the application that currently is
active. Note that you cannot be more specific than the application
that you want to send the events to, within the application, the
control that has keyboard focus will receive the events.
You can set the typing speed at load time by setting the environment
variable KEY_RATE. See {Accessibility::Core::KEY_RATE} for details on
possible values. An example of using it would be:
KEY_RATE=SLOW irb -rubygems -rax_elements
KEY_RATE=0.25 rspec gui_spec.rb