Search This Blog

Wednesday, August 18, 2010

Automated Regression Testing Challenges in Agile Environment

Recently, when I wanted to start my new Automated Testing Project with four resources, I thought of applying any one of the Agile methodologies. But I was not able to continue because, a series of questions were raised inside my mind. The questions are like “Is it possible to use Agile methodologies in Automated Testing?”, “Can I use traditional tools”, “Should I have to go for open-source tools”, “What are the challenges I have to face if I am implementing automation in Agile Environment”. In this article let us analyze some of challenges we face while implementing Automation with Agile methodologies. Automated testing in the Agile environment stands a risk of becoming chaotic, unstructured and uncontrolled.

Agile Projects present their own challenges to the Automation team; Unclear project scope, Multiple iterations, Minimal documentation, early and frequent Automation needs and active stakeholder involvement all demand lot of challenges from the Automation Team. Some of these challenges are:

Challenge 1: Requirement Phase
Test Automation developer captures requirements in the form of “user stories”, which are brief descriptions of customer-relevant functionality.

Each requirement has to be prioritized as follows:

High: These are mission critical requirements that absolutely have to be done in the first release
Medium: These are requirements that are important but can be worked around until implemented.
Low: These are requirements that are nice-to-have but not critical to the operation of the software.

Once priories are established, the release “iterations” are planned. Normally, each Agile release iteration takes between 1 to 3 months to deliver. Customers/software folks take liberty to make too many changes to the requirements. Sometimes, these changes are so volatile that the iterations are bumped off. These changes are greater challenges in implementing Agile Automation testing process.

Challenge 2: Selecting the Right Tools

Traditional, test-last tools with record-and-playback-features force teams to wait until after the software is done. More over, traditional test automation tools don’t work for an Agile context because they solve traditional problems, and those are different from the challenges facing Agile Automation teams. Automation in the early stages of an agile project is usually very tough, but as the system grows and evolves, some aspects settle and it becomes appropriate to deploy automation. So the choice of testing tools becomes critical for reaping the efficiency and quality benefits of agile.

Challenge 3: Script Development Phase

The Automation testers, developers, business analysts and project stakeholders all contribute to kick-off meetings where “user-stories” are selected to next sprint. Once the “user-stories” are selected for the sprint, they are used as the basis for a set of tests.

As functionality grows with each iteration, regression testing must be performed to ensure that existing functionality has not been impacted by the introduction of new functionality in each iteration cycle. The scale of the regression testing grows with each sprint and ensures that this remains a manageable task the test team use the test automation for the regression suite.

Challenge 4: Resource Management

The Agile approach requires a mixture of testing skills, that is, test resource will be required to define unclear scenarios and test cases, conduct manual testing alongside developers, write automated regression tests and execute the automated regression packages. As the project progresses, specialist skills will also be required to cover further test areas that might include integration and performance testing. There should be an appropriate mix of domain specialist who plan and gather requirements. The challenging part in the Resource management is to find out test resources with multiple skills and
allocate them.

Challenge 5: Communication

Good communication must exist among Automation testing team, developers, business analysts and stake holders. There must be highly collaborative interaction between client and the delivery teams. More client involvement implies more suggestions or changes from the client. It implies more bandwidth for communication. The key challenge is that the process should be able to capture and effectively implement all the changes and data integrity needs to be retained. In traditional testing, developers and testers are like oil and water, but in agile environment, the challenging task is that they both must work together to achieve the target.

Challenge 6: Daily Scrum Meeting

Daily Scrum Meeting is one of the key activities in Agile Process. Teams do meet for 15 minutes stand up sessions. What is the effectiveness of these meetings? How far these meetings help Automation practice Developers?

Challenge 7: Release Phase

The aim of Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. This means that there is no single release phase for a product. The challenging part lies in integration testing and acceptance testing of the product.

If we can meet these challenges in a well optimized manner, then Automated Regression Testing in Agile environment is an excellent opportunity for QA to take leadership of the agile processes. It is better placed to bridge the gap between users and developers, understand both what is required, how it can be achieved and how it can be assured prior to deployment. Automation practice should have a vested interest in both the how and the result, as well as continuing to assure that the whole evolving system meets business objectives and is fit for purpose.

How to Manage Automation Expectations:

How to Manage Automation Expectations:
In order to implement a successful automation effort, testers need to educate management on a number of different issues. Educating management on these issues could mean the difference between a successful automation effort and a failed automation attempt.

One such important issue addresses integrating test automation into the entire development process. Testers need to educate senior management as well as script writers on how a specific tool will fit into their software development environment and software development life cycle. It is equally important to remind everyone that manual testing does not end when automation is implemented.

Another issue concerns the purchasing of tools and their impact on test planning. Test planning, planning what to test and how to test, becomes much more complex and more important when a tool is purchased. For example, test automation is rarely justified for all testing. The tester needs to determine what tasks make sense to automate and what does not make sense to automate.

Moreover, the organization needs to understand that automation does not eliminate manual efforts on the part of testers. Testers need to maintain automation scripts and verify the execution of automated scripts. This is manual work that needs to be factored into the test team's testing activities.

In addition, in order to automate successfully, testers need training and time to master the test tools. Many managers view the test tools as simple capture / playback programs which a tester can learn and implement in his spare time. This is a misconception. Successful test automation frequently requires the use of complex tools — tools which are much more complex than the just capture / replay tools. In order to use these tools effectively, testers need considerable training. Management needs to be committed to training testers on these tools and to establishing the infrastructure in which the tool operates.

Also successful test automation requires the involvement of the software development team as well as testers. In the past, developers and testers had minimal contact. Frequently, the philosophy was to "throw code over the wall and see if it works". This type of interaction does not work when automating the testing process. Developers and testers need to work much more closely together. Developers have to provide support personnel and technical information on their development methods. They can be asked to use the automation tools during their unit and integration testing. More troubling to the developer will be that test automation may raise concerns about the testability of their code. This especially will be the case if standards are not followed, or if developers use odd, homegrown or even very new libraries / objects.

The organization’s project management team must have a clear understanding about the types of roles and responsibilities required for a successful automation effort. The creation of the test environment starts when an organization purchases hardware and installs a tool. Then the test team and development team need to work together to build and maintain a test automation environment that may include dedicated servers, workstations, databases and the like. Management needs to include development of the test environment in the overall project plan. They should also budget for the resources needed to successfully automate testing.

Successful tool automation depends not only on the test tools but also on a standard testing process and the right test team roles, duties and skills. Tools, process and test teams are the three essential legs of the test automation stool.

Moreover, the automation test team needs to have a blend of testing, programming, and tool knowledge. If an organization wants to reap the automation benefits promised by tool vendors, it needs to use the tool as a complement to manual testing. It also means adopting a strong test methodology and training the test team on the ins and outs of the selected tool in an organization's unique development environment.

How to Generate Excel Reports in QC 10.0

To Create Excel reports -We need to have access for DashBoard module in QC 10.0
For QC 9.2 the reports option is under Tools - Home Page
and For QC 9.0 there is no option to generate Excel Reports

Navigation - QC 10.0 -
1) Go to DashBoard module
2) Select Excel Report in top left corner
3) Enter Report Name
4) Write Query under Query Tab
5) Add macro under Post Processing Script
6) Set the Status to "Ready"
7) For test purpose we can use Query Generator
8) For viewing of Tables we can use Query Generator
9) Click Generate Query
10)There you go report