What is test coverage in software testing? It’s advantages and disadvantages

Test coverage measures the amount of testing performed by a set of test. Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage and is known as test coverage.

The basic coverage measure is where the ‘coverage item’ is whatever we have been able to count and see whether a test has exercised or used this item.

There is danger in using a coverage measure. But, 100% coverage does not mean 100% tested. Coverage techniques measure only one dimension of a multi-dimensional concept. Two different test cases may achieve exactly the same coverage but the input data of one may find an error that the input data of the other doesn’t.

There are many different types of coverages which we will look at in detail, in subsequent topic, some of them are:

1. Statement coverage
2. Decision coverage
3. Condition coverage

Benefit of code coverage measurement:

• It creates additional test cases to increase coverage
• It helps in finding areas of a program not exercised by a set of test cases
• It helps in determining a quantitative measure of code coverage, which indirectly measure the quality of the application or product.

Drawback of code coverage measurement:

• One drawback of code coverage measurement is that it measures coverage of what has been written, i.e. the code itself; it cannot say anything about the software that has not been written.
• If a specified function has not been implemented or a function was omitted from the specification, then structure-based techniques cannot say anything about them it only looks at a structure which is already there.

Test Architect

Test Architect is the senior position who looks after solutions for problems faced while testing. The role seeks deep technical knowledge and up-to-date knowledge about latest tools and technologies.  This role does not asks for people/team management skills.

Test Architect is not a common role. It is only found in organizations that focus heavily on the use of automation and technology in testing.

Test Architects must generally fulfill the following criteria

• Significant years of experience (minimum of 8 years)
• Must be a technical specialist

Skills required

1. Experience in creating test automation framework, using latest and relevant tools.
2. Sound knowledge of different technologies and approach to automation.
3. Deep understanding about which test cases should be selected for automation.
4. Strong technical leadership abilities

Roles and responsibilities of a Test Architect

Reports to : Test Manager / Quality head

Role and Responsibilities

1. Helps in defining automation approach for testing.
2. Supports test manager in fulfilling strategic goals by providing technical support and help in terms automation.
3. Identifies effective technologies and tools aligned with what being already in use.
4. Helps and designs test automation framework based on past experience and demand of current project.
5. Works in identifying best suitable test cases for automation.
6. Helps in creating Test environment and test data.
7. Co-ordinates for testing via automation.
8. Monitors, enhances, improves automation activities as per demand of project.
9. Collaborates with team in mentoring and training team members for automation.

Senior Software Tester / Senior QA Engineer

This role may also be called as Senior QA Engineer in some organizations. Every company has different criteria when it comes to defining a designation. Most senior positions are decided in terms of years of experience.

Senior software tester is the role that comes with

• Few years of experience (typically 4-5 years)
• Multiple achievements (multiple projects, strict timelines, identification of critical bugs)
• Constant learning (different tools, processes, methods)

Roles and responsibilities of a Senior Software Tester

Reports to : Test Lead

Role and Responsibilities

1. Participation in Test planning, designing and estimation – After gaining few years of experience, the tester is expected to participate in Test planning meeting and contribute in same.
   • Test planning meeting is the meeting where high level test scenarios, challenges, risks, resources etc. are discussed and the Senior Tester can provide his inputs.
   • Test designing is the process where high level test scenarios are broken into medium/minute level test cases. Which kind of test cases to write, what to focus on, what risk factors to be considered etc. are the points where Senior Tester is expected to help.
   • Test estimation is a very important part of project planning. After years of experience, the Senior Tester can easily gauge how much time a particular task might take, considering all relevant factors.

2. Review of test artifacts – Junior testers document test cases and submit it for review to Senior Tester.

3. Test automation – The Senior Tester is expected to be good at one test automation tool.
4. identify test cases to be automated.
5. Automate them
6. Provide results on timely basis
7. Train junior team members for the same
8. Collaboration with development team – With experience, the Senior Tester is expected to know communication tactics, to deal with Development team.
   • From reproducing the reported issue to emphasizing on fix for critical bugs to understand how the bug had been fixed and to know the change requests and their impacts, the tester has to work closely with developers.
   • As a Senior Tester, one is supposed to be able to work with development team comfortably, in any situation and under huge pressure.

5. Reporting and tracking defects – With experience, what remains constant for a tester is to identify defects in software, report them and track them till they are satisfactorily fixed.
6. Training need identification – The Senior tester knows the weaknesses of Junior testers in team. He knows what is stopping testers in performing their task efficiently. He identifies training needs for the team and conveys it to Test Lead.
7. Communication training
8. Process training
9. Effective reporting training
10. Tools training

How to utilize experience as Senior Software Tester

1. Encouraging test automation
   • After gaining some experience in field, the main target for every tester is to learn required automation tool and implement it.
   • As an experienced tester, you know that executing each test case manually is not a good idea and not advisable too.
   • Automating regression test cases saves time for testers to execute functional test cases and do thorough exploratory testing.
   • As a senior tester, you are expected to come up with suggestions about which tools can be used to make the process smooth and effective.
   • The senior tester is supposed to help/start implementing automation and should train/mentor junior team members for the same.
2. Identifying loopholes in processes being followed
   • Industry experience enables a tester to understand different processes and their effectiveness.
   • As a senior team member, you are expected to share your experience while following different processes and data about which process is not effective and why.
   • Also, as a senior, you are supposed to come up with a solution about the loopholes/problems you are seeing in current processes.
   • Transparent discussion about process improvement, with Leads and management is expected from senior testers.
3. Mentoring and Training junior team members
   
   • You gained experience because someone helped you to grow during initial phase of career. As a senior now, it’s your responsibility to train junior team members.
   • The senior tester works as a communication bridge between junior team members and lead.
   • The senior tester is expected to train junior team members efficiently so that best output can be generated.
4. Data analysis skills
   
   • The senior tester is expected to analyze the test results and should be able to provide insight about product quality in different terms like performance, functional, user’s perspective etc.
   • For non-reproducible bugs, senior tester should be able to help to junior testers as well as developers, by analyzing test environment, test data, sequence of steps taken and timing etc.
   • For automation, the senior tester should be able to analyze the results which the automation script execution has generated.
5. Flexible to support
   • As a senior team member, you know the criticality of bugs and how soon they should be resolved. In this context, you are supposed to be available to help with reproducing the bug and to verify the bug fixes.
   • Due to tight timelines for testing, if the task is not completed, the senior member should be flexible enough to stretch out and complete the task.
6. Self organized with no monitoring required
   
   • As a junior, you worked under constant monitoring and learned how things work. Now, as a senior tester, you know what is the priority of particular task and relevant timeline and how you should approach.
   • The senior team member is aware of processes being followed and therefore he aligns his working style to match with the processes.
   • Understanding the importance of timely reporting comes with experience. As a senior tester, you are skilled about how to report, what to include in report, whom to report etc.