Hello World

Let's begin the tutorial with a customary Hello World example!

You will learn how to capture a screenshot of a GUI element and write a Sikuli Script to:

  1. Click on the element
  2. Type a string into the element

The goal of the Hello World script is to automatically type "Hello World" into the spotlight search box, like this:

http://script.sikuli.org/tutorial/spotlight.png

Now, open the Sikuli IDE. We begin by taking the screenshot of our target, the Spotlight symbol http://script.sikuli.org/tutorial/spotlight-symbol.png in the upper-right corner so that we can tell Sikuli Script what to look for and click on.

Sikuli Script provides two methods to switch to the screen capture mode. The first method is to click on the camera button in the toolbar.

The second method is to press the hot-key (Command + Shift + 2 Macs). This hot-key can be triggered even if the script editor is minimized.

In the screen capture mode, the screen will look darker and freeze momentarily. The entire desktop becomes like a canvas. We can draw a rectangle around the target, the spotlight symbol.

http://script.sikuli.org/tutorial/capture-spotlight.png

The image within the rectangle will be captured and inserted into the editor.

http://script.sikuli.org/tutorial/ed1.png

Next, we are going to use the click function to simulate a mouse click on the given target. Type click() and put the image inside of the pair of parenthesizes as the argument to the function.

http://script.sikuli.org/tutorial/ed2.png

The next action is to type "Hello World" in to the search box. This is very straightforward.

http://script.sikuli.org/tutorial/ed3.png

The type function will type the string given in the argument into whichever input control that has the focus. After clicking on the spotlight symbol, we can expect the spotlight search box will be the input that has the focus.

Congratulations! You have completed your first Sikuli Script. You can press the run button to see this script in action!

Make It Simpler

The API provided by Sikuli Script intends to simplify and minimize the code to perform GUI automation. For example, the type function provides 2 types of interfaces: one accepts the string to type, and another one requires an additional image pattern for specifying the target to click on before typing in the string. (Actually, each type of interface can have an optional parameter "key modifiers." They are neglected here because we want to keep this tutorial simple.) In other words, the above 2-line hello world example can be reduced to the following 1 line, which has the same functions: 

type(, "hello world")

Make It More Robust

Some of you may find the hello world example does not work on your machine because of the lag of the spotlight search box. You may usually find that some programs respond very slowly. This delay may break Sikuli Script's automation, because Sikuli Script does not know how long it should wait between each action.

In the hello world example, the spotlight search box may delay to appear for several seconds. However, our script just types "hello world" anyway without waiting the search box appeared. This could cause unexpected results that another program with keyboard focus got the "hello world".

To resolve this issue, a more robust way is to make sure the appearance of the spotlight search box. Thus, we can write a wait function to ask Sikuli Script for waiting the appearance of the search box. The wait() function can take an optional parameter timeout that specifies the maximum waiting time in milliseconds. If it is not specified, the default timeout is 3 seconds.

A robust version of hello world with wait() is illustrated as following: 

click()
wait()
type("hello world")

Consider a long script of GUI interaction, every action may need a wait() beforehand to make sure the GUI program has responded the previous action and entered the expected state. To simplify these redundant wait() functions, Sikuli Script provides a function setAutoWaitTimeout function to let every action that needs an image target automatically wait the appearance of the target. If the auto waiting timeout is set to a non-zero value, Sikuli Script will keep looking for the image on the screen as many times as possible until the target is located or timeout. By default, Sikuli Script sets the AutoWaitTimeout to 3 seconds so that users do not need to call wait() before each action.

Attachments