| Feature | QTP(UFT) | Selenium | Ranorex | Coded UI | Test Complete | Sahi |
| Language Support | VB Script | Java,
C#, Ruby, Python, Perl PHP , Javascript |
C#, Vb.Net | C#, VB.Net | VBScript, C#Script, Jscript,C++Script,DelphiScript | TBD |
| Windows (Non-browser) based Application support |
Yes | No | Yes | Yes | Yes | TBD |
| Browser support | Google Chrome (uptill ver 23), Internet Explorer , Firefox ( ver 21) | Google Chrome , Internet Explorer , Firefox , Opera , HtmlUnit | Google Chrome , Internet Explorer , Firefox , Safari | IE (For building OR), Google Chrole, Firefox | IE,Firefox,Chrome,Safari, Opera | TBD |
| Environment Support | Only Windows | Windows
, Linux , Solaris OS X , Others (If brower & JVM or Javascript support exists) |
Windows | Only Windows | Only Windows | TBD |
| Technologies Support | a)
Web based applications b) Windows based applications c) .Net applications d) Swing Applications e) SAP f) Oracle Applications g) Siebel h) ActiveX applications i) Peoplesoft j) Mainframe terminal emulators k) Web Services l) Terminal Emulator m) Delphi n) Power Builder o) Windows Mobile p) SOAP |
TBD | Windows Applications a) Web based applications b) Windows based applications c) .Net applications d)WPF e)Qt f)Java g) Delphi h)VB6 i)ActiveX j)Microsoft Visual Fox Pro k)Power Builder l)Microsoft Office GUI m)Microsoft Access n)Microsoft Dynamics AX o)Microsoft Dynamics CRM p)Microsoft Dynamics NAV q) AIr r)Infragistics s)Dev Express t)Janus u)Syncfunction v)Telrik w)Skinsoft Web Applications 1)HTML5 2)Ajax 3)Silverlight 4)Ext Js 5)Adobe Flash/Flex/Air 6)JQuery 7)ASP.Net 8)Google Web Tool Kit 9)YUI Library 10)Java Applets 11)Sencha GXT 12)DHTMLX 13)SweetDevRia 14)MochiKit 15)MooTools 16)Pyjs 17)Rico |
Web
based applications Windows Applications |
Supported-Windows,
Web, Mobile, Flash, .net, Java, PowerBuilder,Delphi,Flex, Air,WebServices(SOAP) Not Sure about ERP |
TBD |
| Mobile (Phones & Tablets) support |
Different
commercial product i.e. HP UFT Mobile (formerly known as MobileCloud for QTP) |
Android
, iPhone & iPad , Blackberry , Headless WebKit |
Android , iOS | Windows phone, windows phone apps. | Android & iOS | TBD |
| Framework | Easily
integrated with HP Quality Center or HP ALM (separate commercial products) |
Selenium
+ Eclipse + Maven / ANT + Jenkins / Hudson & its plugins / Cruise Control + TestNG + SVN |
Visual Studio +TFS+SVN+Jenkins | Easily integrated with MTM. | 1. QAComplete & ALMComplete from
SmartBear 2. QC from HP |
TBD |
| Continuous Integration | Possible
through Quality Center / ALM or Jenkins |
Possible
through Jenkins / Hudson / Cruise Contro |
Possible
through Jenkins / Hudson / Cruise Contro |
Possible through Cruise Control | Possbile through Jenkins | TBD |
| Object Recognition / Storage |
Inbuilt
Object Repository (storing Element Id, multiple attributes) along with weightage that gives flexibility on deviation acceptance in control recognition |
UI
Maps and different object location strategy such as -XPath Element ID or attribute DOM |
UI
Maps and different object location strategy such as -XPath Element ID or attribute DOM |
UI Maps and identification of objects by unique properties. Also support Xpath based object identification. | Built-in Namemapping mechanism | TBD |
| Image based Tests | Easily possible | Possible but not easy | Easily possible | Not possible | Possible | TBD |
| Reports | Quality
Center has in-built awesome dashboards |
Integration
with Jenkins can give good reporting & dashboard capabilities |
Integration
with Jenkins can give good reporting & dashboard capabilities |
MSTest has built in reports. We can also create custom reports. | Built-in reports Of TestComplete is informative and detailed reports also generated when integrated wth test management tools QAComplete, ALMComplete and QC | TBD |
| Software Cost | License
& Annual maintenance fees |
Zero | License
& Annual maintenance fees |
License | License
& Annual maintenance fees |
TBD |
| Coding Experience of Engineer |
Not Much | Should
be very good along with technical capabilities of integrating different pieces of framework |
Should
be very good along with technical capabilities of integrating different pieces of framework |
Should be very good at C# and object oriented concepts. | Hands on scripting is madatory | TBD |
| Script Creation Time | Less | High | High | Medium | Medium | TBD |
| Hardware resource (CPU + RAM) consumption during script execution |
High | Low | Low | High | High | TBD |
| Product Support | Dedicate
HP support along with support forums |
Open Source Community | Dedicate
Ranorex support along with support forums |
Microsoft support along with forums. | Smartbear support with open forums | TBD |
Tuesday, May 10, 2016
Test Automation Tool Comparision Matrix
Monday, April 25, 2016
Integration of Selenium with TestComplete
Integration of
Selenium with TestComplete
Selenium is a free testing tool for web
applications. Using the Selenium WebDriver API, you can create web tests in any
programming language.
You can include your Selenium WebDriver tests
into your TestComplete test project, run them as part of your automated testing
process and analyze test results
Preparing
TestComplete for Integration With Selenium Tests
» Before you run your Selenium WebDriver tests
in TestComplete, you need to prepare your test computer and TestComplete for
Selenium integration.
» The CPI Cards Automation framework must be
installed and configured on your computer and have Selenium WebDriver modules
installed and configured.
» To use
Maven to run Selenium tests created by using TestNG, you must have Maven
modules installed and configured on your computer.
» The PATH environment variable on your
computer must specify the path to Maven modules.
» The JAVA_HOME environment variable must
specify the path to your Java Developer Kit software location.
» For more information on how to install and
configure Maven, see the Maven documentation.
» If your Selenium tests use a specific web
browser driver, that driver modules must be installed and configured on your
computer.
Steps to Integrate Selenium Tests created using TestNG
to TestComplete
1.
Create a New Project suite with a valid name
2.
Create a new Project folder under the newly created Project suite
with a valid name
3.
Right Click on Project folder and mouse over on Add menu item and
select New Item
4.
In the Create Project Item window, select Selenium or Unit Testing
as the Item type and provide a valid name
5.
Right click on newly created Item and mouse over on Add menu item
and select Nes s
6.
In the create Project Item window, select TESTNG as the Item type as the current CPI cards Selenium scripts
are built using Java with TESTNG and provide a valid name
7.
Select the newly created Project Item(TESTNG) and a new tab is
opened in the Workspace requesting for Maven Project and Classpath details
8.
Download latest Maven file from maven.apache.org and click on mvn
windows command file located in bin folder under maven x.x.x folder
9.
Since the Project is built on Maven, select the option Use Maven
Project and provide the valid path of the project created say …\…\pom.xml
10.
Select the option Run all
tests in the opened tab and save it
11.
Ensure Java is installed on
the machine and configured in Environment variables with JAVA_HOME as
Environment variable name
12.
Click on Tools menu and
select Options menu Item
13.
On selection, Options window is opened. Select Engines folder and
select Selenium Item which is located under it and provide the java.exe path
14.
Now right click on TESTNG Item located under Selenium unit Tests
and select Run
15.
Now TestComplete engine
compiles, builds and starts executing the Maven project
16. Once on execution find the execution results in results folder
located under Selenium Framework as TestComplete only provides details about
maven build success
Mobile Tool Comparision Matrix
| Mobile Tool Comparision Matrix | ||||||||
| Criteria | SeeTest | Jamo Solutions | UFT+PerfectoMobile | Eggplant | Appium | Selenium WebDriver | Ranorex | Xcode UIA |
| Native Object Recognition | Yes | Yes | Yes | No | Yes | No | Yes | Yes |
| Image Based Recognition | Yes | No | Yes | Yes | No | No | No | No |
| OCR Text Based Recognition | Yes | No | Yes | Yes | No | No | No | No |
| Web Object Recognition | Yes | No | Yes | No | Yes | Yes | No | Yes (Simulator) |
| Object Spy | Yes | Yes | Yes | Yes | Yes | No | No | Yes (Simulator) |
| Browser Support | ||||||||
| Safari Browser | Yes | Yes | Yes | Yes (Limited) | Yes | Yes | No | Yes |
| Firefox Browser | Yes | Yes | Yes (Limited) | No | No | No | No | |
| Chrome Browser | Yes | Yes | Yes (Limited) | No | No | No | No | |
| Android Browser | Yes | Yes | Yes | Yes (Limited) | Yes | Yes | No | No |
| Opera Browser | Yes | Yes | Yes (Limited) | No | No | No | No | |
| Internet Explorer Browser | Yes | Yes | Yes | Yes (Limited) | No | No | No | No |
| Hybrid Application Support | Yes | Yes | Yes | Yes (Limited) | Yes | No | No | No |
| Record and Playback capability | Yes | Yes | Yes | Yes | No | No | Yes | Yes (Limited) |
| Device OS Support | ||||||||
| Android <4 .1="" td=""> | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
| Android >=4.1 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
| iOS <7 .0="" td=""> | Yes | Yes | Yes | Yes | Yes | Yes | Yes (Limited) | Yes |
| iOS 7.0 | Yes (Limited) | Yes | Yes (Limited) | Yes | Yes (Limited) | Yes | Yes | Yes |
| Windows 6.5 | Yes | Yes | Yes | Yes | No | No | No | No |
| Windows 7.0 | Yes | Yes | Yes | Yes | No | No | No | No |
| Windows 8.0 | Yes | Yes | Yes | Yes | No | No | No | No |
| Symbian | Yes | No | Yes | Yes | No | No | No | No |
| Blackberry RIM | Yes | Yes | Yes | Yes | No | No | No | No |
| Orientation Support | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes |
| Hardware Control support | Yes (Limited) | Yes (Limited) | Yes (Limited) | No | Yes (Limited) | No | No | Yes |
| Gesture support(Swipe, Pinch, Scroll) | Yes (Limited) | Yes (Limited) | Yes (Limited) | No | Yes (Limited) | Yes (Limited) | Yes | Yes |
| Cloud Based Solution | Yes | No | Yes | No | Yes** | Yes** | No | No |
| Interrupt Testing | Yes | Yes | Limited | No | No | No | No | Yes |
| Multi Language Support | Yes | Yes | No | No | Yes | Yes | Yes (Limited) | No |
| Functional Tool Integration Support | Yes | Yes | Yes (Limited) | No | Yes (Limited) | No | Yes (Limited) | No |
| App Instrumentation requirement | No | No | No | No | No | No | Yes | Yes |
| Capture Device Monitors | Yes (Limited) | Yes | No | No | No | No | No | Yes |
| Access to Device Logs | Yes (Limited) | Yes (Limited) | No | No | No | No | No | Yes |
| Support to PC OS | ||||||||
| Windows XP and above | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
| Mac OSX | No | Yes | No | No | Yes | Yes | No | Yes |
| Linux | No | Yes | No | No | Yes | Yes | No | Yes |
| Parallel Execution on Multiple Devices | Yes* | No | No | No | Yes | Yes | Yes | Yes |
| Remote Device Execution | Yes* | Yes | Yes | No | Yes | Yes | No | No |
| Real Device Execution | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes |
| Support for CI | Yes | No | No | No | No | Yes | Yes | No |
Total Score - Yes |
36 | 31 | 26 | 23 | 21 | 17 | 15 | 18 |
| Legend | ||||||||
| Yes* - Additional Licence Cost | ||||||||
| Yes** - External Vendors Service | ||||||||
| Yes (Limited) - Limited Support | ||||||||
Thursday, July 9, 2015
Tool Comparision Matrix - QTP/UFT, Selenium, TestComplete
Tool Comparision Matrix
| Feature | QTP/UFT | TestComplete | Selenium |
| Language Support | VB Script | VBScript, JSScript, DelphiScript, C++Script and C#Script |
Java, C#, Ruby, Python, Perl PHP , Javascript |
| Application Support | A client server application Only (Web+Windows). It also supports add-ons, but user needs to purchase license for them. | All of this included right out of the box there are no plug-ins or add-ons to buy. You can install Test Complete and immediately create any test against any application. | Web Only, It will not support window apps |
| Cross Browser Testing | Yes | Yes | Google Chrome , Internet Explorer , Firefox , Opera , Html Unit |
| API Testing | Yes | Yes | Integrate with external tools |
| Web Service Testing | Yes | Yes | No |
| Database Testing | Yes | Yes | Yes |
| Web Load/Performance Testing | No, Loadrunner separate tool | Yes | No |
| Mobile (Phones & Tablets) support | Different commercial product i.e. HP UFT Mobile (formerly known as MobileCloud for QTP) | Yes (ios,Android,Blackberry..) | Selenium + Appium (blackberry is not possible) |
| Test Management | No, need QC/ALM | Yes, Built-in (QA Complete) | No |
| Jira Integration | No | Yes | Yes |
| Manual Testing | via QC/ALM only | Yes | No |
| Separate Test Execution Module | No, Must use QTP full install (HP is planning to release LeanFT, UFT 12.5 in month of Jul 2015 – there we can have separate test execution engine for less price) | Yes – 399 $ Test Execute Float License | No |
| Version Control Integration | via QC/ALM only | Yes | No |
| Unit Testing Integration | via QC/ALM only | DUnit, JUnit, NUnit, MSTest | Yes |
| Framework | Supports multiple frameworks like Keyword Driven, hybrid driven, modular driven and BPT (ALM) | Integrate with QA Complete,we can create keyword driven, data driven frameworks etc. | Selenium + Eclipse + Maven / ANT + Jenkins / Hudson & its plugins / Cruise Control + TestNG + SVN |
| Object Recognition / Storage | Inbuilt Object Repository | NameSpace mapping | UI Maps and different object location strategy such as -XPath Element ID or attribute DOM |
| Image based Tests | Easily possible | Yes | Possible but not easy |
| Reports | Quality Center has in-built awesome dashboards | Yes | Integration with Jenkins can give good reporting & dashboard capabilities |
| Software Cost | Licensed and very Expensive, Ten user license costs approx. 60L (8000 $ - 15000 $ approx varies from country to country) |
Node Locked – Enterprise – 1999 $, Floating Enterprise – 4499$, Node locked Standard – 999$, Floating Standard – 2999$ | It is open source. So, there's no licensing or renewal cost for this tool. It's free of cost. |
| Coding Experience of Engineer | Not Much | Programming background is highly recommended | Should be very good along with technical capabilities of integrating different pieces of framework |
| Script Creation Time | Less | Less | High |
| Hardware resource (CPU + RAM) consumption during script execution | High | Moderate | Low |
| Product Support | Dedicate HP support along with support forums | Yes | Open Source Community |
Subscribe to:
Posts (Atom)