Automating Calculator application using Java and Sikuli.

Below program demonstrates automating calculator application using Java & Sikuli.

Prerequisites:
1. Java 1.6 or higher version installed.
2. Sikulli installed, click link for Sikuli Installation.
    For simple java program using Sikuli click here.

Capture all buttons of Calculator:

Set build path for Sikuli:

Executing script:

Download the source code from repository.

Program for automating calculator application:
import java.awt.Toolkit;

import java.awt.datatransfer.Clipboard;
import java.awt.datatransfer.DataFlavor;
import java.awt.datatransfer.Transferable;
import java.awt.datatransfer.UnsupportedFlavorException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.sikuli.basics.Settings;
import org.sikuli.script.App;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Key;
import org.sikuli.script.Screen;

/** Automating Calculator Application.
 *
 * @author http://automation-home.blogspot.in/
 */
public class CalcTest {
Runtime rs = null;
Screen screen = null;
static CalcTest ct = null;
Map<String, String> calcButtonMap = null;
final String APPLICATION = "calc.exe";

/** Initialization block
*
*/
{
calcButtonMap = new HashMap<String, String>();

String imagePath = "\\img\\";

calcButtonMap.put("0", imagePath + "0.PNG");
calcButtonMap.put("1", imagePath + "1.PNG");
calcButtonMap.put("2", imagePath + "2.PNG");
calcButtonMap.put("3", imagePath + "3.PNG");
calcButtonMap.put("4", imagePath + "4.PNG");
calcButtonMap.put("5", imagePath + "5.PNG");
calcButtonMap.put("6", imagePath + "6.PNG");
calcButtonMap.put("7", imagePath + "7.PNG");
calcButtonMap.put("8", imagePath + "8.PNG");
calcButtonMap.put("9", imagePath + "9.PNG");

//Operators
calcButtonMap.put("Add", imagePath + "Add.PNG");
calcButtonMap.put("Mul", imagePath + "Mul.PNG");
calcButtonMap.put("Div", imagePath + "Div.PNG");
calcButtonMap.put("Sub", imagePath + "Sub.PNG");

calcButtonMap.put("C", imagePath + "C.PNG");
calcButtonMap.put("Equals", imagePath + "Equals.PNG");
}

/** main method.
*
* @param args
* @throws IOException
* @throws InterruptedException
* @throws FindFailed
* @throws UnsupportedFlavorException
*/
public static void main(String[] args) throws IOException, InterruptedException, FindFailed, UnsupportedFlavorException {
 
ct = new CalcTest();

Settings.ClickDelay = 20;

//disabling the sikuli logs.
Settings.ActionLogs=false;

ct.screen = new Screen();
ct.rs = Runtime.getRuntime();

ct.launchCalculator();
ct.calculate();
ct.closeCalculator();
}

/** Perform's calculations and validates the results.
*
* @throws NumberFormatException
* @throws FindFailed
* @throws InterruptedException
* @throws UnsupportedFlavorException
* @throws IOException
*/
public void calculate() throws NumberFormatException, FindFailed, InterruptedException, UnsupportedFlavorException, IOException{
int results = 0;

// 4 + 5 =9
List<String> calcListAdd = new ArrayList<String>();
calcListAdd.add("4");
calcListAdd.add("Add");
calcListAdd.add("5");
calcListAdd.add("Equals");
results =Integer.parseInt(performOperation(calcListAdd));
if(results == 9){
System.out.println("Results Sucess: 4 + 5 = "+results);
}else{
System.out.println("Results Failed: 4 + 5 = "+results);
}

// 8 * 2 =16
List<String> calcListMul = new ArrayList<String>();
calcListMul.add("8");
calcListMul.add("Mul");
calcListMul.add("2");
calcListMul.add("Equals");
results = Integer.parseInt(performOperation(calcListMul));
if(results == 16){
System.out.println("Results Sucess: 8 * 2 = "+results);
}else{
System.out.println("Results Failed: 8 * 2 = "+results);
}

// 71 - 11 = 60
List<String> calcListSub = new ArrayList<String>();
calcListSub.add("7");
calcListSub.add("1");
calcListSub.add("Sub");
calcListSub.add("1");
calcListSub.add("1");
calcListSub.add("Equals");
results = Integer.parseInt(performOperation(calcListSub));
if(results == 60){
System.out.println("Results Sucess: 71 - 1 = "+results);
}else{
System.out.println("Results Failed: 71 - 1 = "+results);
}

// 18 / 2 = 9
List<String> calcListDiv = new ArrayList<String>();
calcListDiv.add("1");
calcListDiv.add("8");
calcListDiv.add("Div");
calcListDiv.add("2");
calcListDiv.add("Equals");
results = Integer.parseInt(performOperation(calcListDiv));
if(results == 9){
System.out.println("Results Sucess: 18 / 2 = "+results);
}else{
System.out.println("Results Failed: 18 / 2 = "+results);
}

// 4 + 5 + 1 + 10 = 20
List<String> calcListAddMoreNums = new ArrayList<String>();
calcListAddMoreNums.add("4");
calcListAddMoreNums.add("Add");
calcListAddMoreNums.add("5");
calcListAddMoreNums.add("Add");
calcListAddMoreNums.add("1");
calcListAddMoreNums.add("Add");
calcListAddMoreNums.add("1");
calcListAddMoreNums.add("0");
calcListAddMoreNums.add("Equals");
results = Integer.parseInt(performOperation(calcListAddMoreNums));
if(results == 20){
System.out.println("Results Sucess: 4 + 5 + 1 + 10 = "+results);
}else{
System.out.println("Results Failed: 4 + 5 + 1 + 10 = "+results);
}
}

/** Perform operations on calculator application.
*
* @param calcList
* @return ct.getResults() as String
* @throws FindFailed
* @throws InterruptedException
* @throws UnsupportedFlavorException
* @throws IOException
*/
public String performOperation(List<String> calcList) throws FindFailed, InterruptedException, UnsupportedFlavorException, IOException{
screen.click(calcButtonMap.get("C"));
Thread.sleep(100);

for(String calElement : calcList){
screen.click(calcButtonMap.get(calElement));
}

return ct.getResults();
}

/** Fetch the results after the operations are done.
*
* @return results as String
* @throws FindFailed
* @throws UnsupportedFlavorException
* @throws IOException
* @throws InterruptedException
*/
public String getResults() throws FindFailed, UnsupportedFlavorException, IOException, InterruptedException{
Thread.sleep(100);
screen.click("\\img\\ResultPanel.PNG");
screen.type("c",Key.CTRL);

Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = clipboard.getContents(DataFlavor.stringFlavor);
String results = (String) contents.getTransferData(DataFlavor.stringFlavor);

System.out.println("results >> "+results);
return results;
}

/** Launch the calculator application.
*
* @throws IOException
* @throws InterruptedException
*/
public void launchCalculator() throws IOException, InterruptedException{
closeCalculator();
Thread.sleep(500);
App.open(APPLICATION);
Thread.sleep(500);
}

/** Quit the calculator application.
*
* @throws IOException
*/
public void closeCalculator() throws IOException{
App.close(APPLICATION);

//Unable to close the calculator application using "App.close("calc.exe")"
rs.exec("taskkill /F /IM "+APPLICATION);
System.out.println("Calculator Application is closed.");
}
}

Watch Demo

Note:

Since Sikuli works on the concept of comparing the image supplied to the program with the Desktop Screen

Above program works with only Windows XP calculator application

Ex: Calculator application User Interface changes from Windows XP to Windows 7.

If you want to run the above program/script in other operating systems, then you need re-capture the calculator images and place them in the "img"(i.e. images folder) with the already existing file names(i.e overwriting the images). .

Extending above script:
Above program can be integrated with TestNG and DataDrivenFramework.

Pros of Sikuli:

1. Sikuli work's with both browser based and non-browser based applications.

2. While automating the web based applications(i.e. using browser), operations which are not supported by Webdriver API can be performed by the Sikuli API.

3. Sikuli can be integrated with the webdriver API(i.e. Automation Framework with Webdriver & Sikuli API)

Cons of Sikuli:

1. As Sikuli works on the concept of image recolonization(i.e. Sikuli click's on the screen based on the input image supplied to the program), if the User Interface components of the application are updated or changed then you need to re-capture the images.

Ex: Calculator buttons color is changed from white to grey color.

2. While the script(i.e. script developed using Sikuli API) is executing, user should not take the control of keyboard or mouse (i.e. similar to that for java.awt.robot API)

3. When the script(i.e. developed using Sikuli API) is executing system should not be locked.

4. Before executing the script(i.e. developed using Sikuli API) make sure that no other applications/folder's are present(i.e. maximized), close all other application and folders (or) minimize them.

Ex: Before executing the script for calculator application, if folder is maximized containing all the calculator images. While script is executing Sikul click's few calculator application buttons and  clicks other images in the maximized folder(since the folder contains the calculator application images). 

Related Links

Java and AutoIt - Automating the calculator application using autoitx4java
http://automation-home.blogspot.in/2015/06/java-and-autoit-automating-calculator-application.html

Java and Twin Automation Tool - Automating windows calculator application.
http://automation-home.blogspot.in/2015/12/java-twin-automation-tool-automate-windows-calc-application.html