Software development methods are constantly evolving, with related tools and technologies not being an exception. Such improvement is necessary to increase the productivity and quality of the product being created, reduce the time required for development, and enhance customer satisfaction with the result. Ultimately, testing plays a vital role in the development of a quality product.
The list of open source automation testing services out there is rather diverse: for mobile apps and load testing, for Android and iOS, behavior driven and keyword driven. One feature they share is high reliability.
1. Selenium
That is one of the most popular open source test automation frameworks for web applications. Selenium is highly extendable through a wide range of libraries and APIs to meet the desired requirements and needs. It also allows writing advanced test scripts of high complexity.
Pros
- Free;
- Integrated with Agile, DevOps, Continuous Delivery workflow;
- Supports mobile testing;
- Supports a wide range of languages, platforms, and browsers;
- Large library of plugins;
- Huge community.
Cons
- Doesn’t allow for codeless testing;
- No built-in image comparison;
- No tech support;
- No reporting capabilities.
2. Cucumber
That is a behavior driven development (BDD) tool which is used to write acceptance tests for web applications. BDD implies close cooperation among the business, the customer and the technical team, which requires specific requirements that can be understood in simple language by all interested parties. Cucumber is a tool that takes specifications and binds them to automated tests, effectively creating the current executable documentation. It facilitates collaboration among team members in determining requirements, developing guides, creating automated tests, and system description.
Pros
- Specification in natural language, readable tests and user stories;
- Allows using another testing framework.
Cons
- Extra code for converting the textual specification to Ruby code;
- Extra layer between test description and the code (you need to make sure the code matches the description);
- Test descriptions should describe the high-level features (usually the user interaction) and not describe the low-level implementation details.
3. Watir
That is a robust tool for web automation testing based on a family of Ruby libraries. Watir supports cross-browser testing, data-driven testing, and mimics the behavior of a user interacting with a website. The tool also integrates with BBD tools like Cucumber, Test/Unit, etc, and is extendable through a set of different APIs.
Pros
- Easy to learn Ruby;
- Has Watir Recorder for easy recording;
- Captures modal dialogues;
Cons
- Doesn’t handle Ajax applications well enough.
4. Appium
That is an open source test automation platform for use with native, hybrid, and mobile web applications. Appium manages applications for iOS and Android using the WebDriver protocol. Appium is “cross-platform”, which means that it allows you to write tests on several platforms (iOS, Android) using the same API. Ultimately, you can reuse the code between tests for iOS and Android.
Pros
- Support for both platforms iOS and Android;
- Support for continuous integration;
- Doesn’t require access to your source code or library;
- Support for various frameworks;
Cons
- Doesn’t support image comparison.
5. RobotFramework
That is an open source automated testing framework that implements a keyword-based testing approach through acceptance test driven development (ATDD). Robot Framework provides the ability to solve various test automation tasks. However, its capabilities can be expanded through the introduction of additional libraries using Python and Java. For example, Selenium WebDriver is a popular external library used in Robot Framework.
Testing engineers can use the Robot Framework as a framework to automatically test not only web applications, but also Android and iOS applications. The Robot Framework will seem easy to learn for testers who are already familiar with keyword-based testing.
Pros
- Keyword Driven Testing;
- High extensibility, in python, java, and/or any .NET language;
- Can be used for REST and SOAP service tests and database tests.
Cons
- IDE’s are not very helpful;
- Hard to debug the code;
- Parallel testing is impossible.
6. Robotium
That is a test automation framework used to simplify the writing of powerful and solid automatic black-box UI tests for Android. With Robotium, developers can write system, function, and user acceptance test scenarios covering various Android activities. Generally, the tool greatly improves the readability of test cases, compared to standard instrumentation tests as it requires minimal time to write solid test cases.
Pros
- Flexible and convenient for results analysis;
- Handles multiple Android activities automatically;
- Requires minimal time to write solid test cases;
- Improved readability of test cases compared to standard instrumentation tests;
- Robust test cases due to the run-time binding to GUI components;
- Automatic timing and delays;
- Fast test case execution.
Cons
- Tied to one app process;
- Tied to JUnit 3 Instrumentation on the device;
- Tied to one app process;
- Can’t work with different applications in on the test.
The above list is just a part of the many available open-source test automation frameworks. To choose the right one for you, decide what needs are paramount in your particular case. Prior to writing the code, check out these automation frameworks first.