This overview provides step-by-step instructions for setting up calabash-android on Windows and explaining the basic components of a calabash-android test suite, with links to relevant resources provided throughout the document.
What are Cucumber and Calabash?
Cucumber is a software tool for writing and running automated acceptance tests using a behavior-driven development style. Test scenarios are written in a domain specific language called Gherkin, which enables the step-by-step description of scenarios in a plain language format intended to be readable by individuals without any programming knowledge. Regular expressions are used to match test steps described in Gherkin to step definitions written in Ruby. These step definitions call methods from an automation API suitable for the target application. Calabash provides these APIs for interacting with native iOS and Android apps.
To create a Calabash test suite for an Android application with calabash-android on Windows, you need a few things in place.
- Java SDK
- Android SDK from Google
- Ruby DevKit
- Apache Ant
- Calabash Android ruby gem
- An Android device connected via USB, or the Genymotion Android emulator which is much more stable with Calabash than the emulator from the Android SDK
- APK for the application under test
Calabash-Android Setup for Windows
- Download and install Oracle JDK: http://www.oracle.com/technetwork/java/javase/downloads/index.html
- Download and install the Android SDK from https://developer.android.com/sdk/installing/index.html. Then add the tools and platform-tools directories from the SDK to your PATH environment variable.
- Download and install Ruby 1.9.3 (check the option to add Ruby to PATH during installation) http://rubyinstaller.org/downloads/
- Download and install apache ant (You can use the self-extractor or WinAnt, but if you use the self-extractor, create an environment variable called ANT_HOME and set it to the directory you extracted it to). https://code.google.com/p/winant/
- Download Ruby DevKit from http://rubyinstaller.org/downloads/ (tdm-32-4.5.2 for Ruby 1.9.3), cd to directory you installed it in and run: ruby dk.rb init
- Check to make sure the config.yml file contains the path to Ruby 1.9.3 and run: ruby dk.rb install
- Install Calabash-Android by running: gem install calabash-android
- Install the Genymotion Android emulator, available from: http://www.genymotion.com.
To generate a cucumber skeleton for your Calabash-Android project:
- Create a directory for your calabash project files and cd to it
- From the directory you created, run: calabash-android gen
Using Calabash Console
The Calabash console allows you to interact with the application under test directly using the calabash API. This is useful for exploring and examining user interface elements and gathering the identifying characteristics necessary to write automation code.
- calabash-android resign <path_to_apk>
- calabash-android console <path_to_apk>
- Start Genymotion Android emulator
- In calabash console, run:
Now the calabash-console can be used interactively.
Running a Calabash-Android test suite
To run a Calabash-Android test suite, run:
calabash-android run <path_to_apk>
For more detailed output, you can use the –verbose option.
For colored output from cucumber, install ANSICON:
- Unzip the file from https://github.com/adoxa/ansicon/downloads
- Navigate to the unzipped directory from a command prompt
- Type ansicon –i to install ANSICON
Now you should see colored output from your cucumber tests. If you get an error message when you run ansicon, try installing an older version.
Components of Calabash-Android:
Gherkin syntax is concisely described here:
Note the Feature Introduction and Given-When-Then links on the bottom of the page.
Cucumber step definitions are described here:
The Calabash-Android Ruby API is described here:
Other Ruby methods that are useful can be found by examining the source code found in operations.rb:
Note that undocumented methods are more likely to change than the documented methods. You may find it instructive to see how the methods described in operations.rb are built using the documented API methods as building blocks.
Xamarin Test Cloud is a cloud-based service that allows you to run Calabash tests against hundreds of different devices: http://developer.xamarin.com/guides/testcloud/introduction-to-test-cloud/
Xamarin Calabash guide: http://developer.xamarin.com/guides/testcloud/calabash/
Calabash-Android Ruby API: https://github.com/calabash/calabash-android/blob/master/documentation/ruby_api.md
Calabash-Android Canned Steps: https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/canned_steps.md
Calabash-Android API code: https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/operations.rb
Calabash-Android Query Syntax: https://github.com/calabash/calabash-android/wiki/05-Query-Syntax
Gherkin Syntax: https://github.com/cucumber/cucumber/wiki/Gherkin
Gherkin Scenario Outlines and tables: https://github.com/cucumber/cucumber/wiki/Scenario-Outlines
Criticism of Cucumber: http://www.jimmycuadra.com/posts/please-don-t-use-cucumber
Cucumber Step Definitions: https://github.com/cucumber/cucumber/wiki/Step-Definitions
Cucumber RegEx Cheat Sheet: http://35qk152ejao6mi5pan29erbr9.wpengine.netdna-cdn.com/wp-content/uploads/2011/08/Cucumber-Regular-Expressions-Cheat-Sheet.pdf
Calabash-iOS setup (using Xcode):
Calabash-iOS setup with Xamarin Studio:
Calabash-iOS setup with Xamarin Test Cloud Agent:
Calabash-iOS Ruby API:
Calabash-iOS predefined steps:
Calabash-iOS query syntax: