Software Quality Assurance

Abstract

This chapter discusses software quality assurance and the importance of process quality. It is a premise in the quality field that good processes and conformance to them is essential for the delivery of high quality product, and this chapter discusses audits, and describes how they are carried out.

13.1 Introduction

The purpose of software quality assurance is to provide visibility to management on the processes being followed and the work products being produced in the organization. It is a systematic enquiry into the way that things are done in the organization, and involves conducting audits of projects, suppliers, and departments. It provides:

• Visibility into the extent of compliance to the defined processes and standards.

• Visibility into the processes and standards in use in the organization.

• Visibility into the effectiveness of the defined processes.

• Visibility into the fitness for use of the work products produced.

Software quality assurance involves planning and conducting audits; reporting the results to the affected groups; tracking the assigned audit actions to completion; and conducting follow up audits, as appropriate. It is generally conducted by the SQA group,¹ and this group is independent of the groups being audited. The activities involved include (Table 13.1).

Table 13.1 Auditing activities

All involved in the audit process need to receive appropriate training. This includes the participants in the audit who receive appropriate orientation on the purpose of audits and their role in it. The auditor needs to be trained in interview techniques, including asking open and closed questions, as well as possessing effective documentation skills in report writing, in order to record the results of the audit. The auditor needs to be able to deal with any conflicts that might arise during an audit.²

The flow of activities in a typical audit process is sketched in Fig. 13.1, and they are described in more detail in the following sections.

Fig. 13.1

Sample audit process

13.2 Audit Planning

Organizations vary in size and complexity and so the planning required for audits will vary. In a large organization the quality manager or auditor is responsible for planning and scheduling the audits. In a small organization the quality assurance activities may be performed by a part time auditor who plans and schedules the audits.

A representative sample of projects/areas in the organization will be audited, and the number and types of audits conducted will depend on the current maturity of the organization. Mature organizations with a strong process culture will require fewer audits, whereas immature organizations may need a larger number of audits to ensure that the process is ingrained in the way that work is done.

It is essential that the auditor is independent of the area being audited. That is, the auditor should not be reporting to the manager whose area is being audited, as otherwise important findings in the audit could be omitted from the report. The independence of the auditor helps to ensure that the findings are fair and objective, as the auditor may state the facts as they are without fear of negative consequences.

The auditor needs to be familiar with the process, and in a position to judge the extent to which the standards have been followed. The audit report needs to be accurate, as incorrect statements made in the report will damage the credibility of the auditor. The planning and scheduling activities will include:

• Project/Area to be audited,

• Planned Date of Audit,

• Scope of Audit,

• Checklist to be used,

• Documentation required,

• Auditor,

• Attendees.

The auditor may receive orientation on the project/area to be audited prior to the meeting and may review relevant documentation in advance. A checklist may be employed by the auditor as an aid to structure the interview.

The role requires good verbal and documentation skills, as well as the ability to deal with any conflicts that may arise during the audit. The auditor needs to be fair and objective, and audit criteria will be employed to establish the facts in a non-judgmental manner.

Software quality assurance requires that an independent group (e.g., the SQA group) be set up. This may be a part time group of one person in a small organization or a team of auditors in a large organization. The auditors must be appropriately trained to carry out their roles. The individuals being audited need to receive orientation on the purpose of audits and their role in the audit.

13.3 Audit Meeting

An audit consists of interviews and document reviews and involves a structured interview of the various team members. The goal is to give the auditor an understanding of the work done, the processes employed, and the extent to which they are followed and effective. A checklist tailored to the audit being conducted is often employed. This will assist in determining relevant facts to judge whether the process is followed and effective. Table 13.2 gives a small selection of questions that may be part of an audit checklist.

Table 13.2 Sample auditing checklist

The audit is an enquiry into to the role of each attendee, the activities performed, the output produced, the standards followed, and so on. The auditor needs to be familiar with the process and in a position to judge the extent to which it has been followed.

The auditor opens the meeting with an explanation of the purpose and scope of the audit, and usually starts with one or more open questions to get the participants to describe their role. Each attendee is asked to describe their specific role, the activities performed, the deliverables produced, and the standards followed. Closed questions are employed to obtain specific information when required.

The auditor will take notes during the meeting, and these are reviewed and revised after the audit. There may be a need to review additional documentation after the meeting or to schedule follow up meetings.

13.4 Audit Reporting

Once the audit meeting and follow up activities have been completed, the auditor will need to prepare an audit report to communicate the findings from the audit. A draft audit report is prepared and circulated to the attendees, and the auditor reviews any comments received, and makes final changes to address any valid feedback.³ The approved audit report is then circulated to the attendees and management.

The audit report will include audit actions that need to be addressed by groups and individuals, and the auditor will track these actions to completion. In rare cases the auditor may need to escalate the audit actions to management to ensure resolution.

The audit report generally includes three parts namely the overview, the detailed findings, and an action plan. This is described in Table 13.3:

Table 13.3 Sample audit report

13.5 Follow Up Activity

Once the auditor has circulated the audit report to the affected groups, the focus then moves to closure of the assigned audit actions. The auditor will follow up with the affected individuals to monitor closure of the actions by the agreed date, and where appropriate a time extension may be granted. The auditor will update the status of an audit action to closed once it has been completed correctly. In rare cases the auditor may need to escalate the audit action to management for resolution. This may happen when an assigned action has not been dealt with despite one or more-time extensions. Once all audit actions have been closed the audit is closed.

13.6 Audit Escalation

In rare cases the auditor may encounter resistance from one or more individuals in completing the agreed audit actions. The auditor will remind the individual(s) of the audit process and their responsibilities in the process. In rare cases, where the individual(s) fail to address their assigned action(s) in a reasonable time frame, the auditor will escalate the non-compliance to management. The escalation may involve:

• Escalation of actions to Middle Management,

• Escalation to Senior Management.

Escalation is generally a rare occurrence, especially if good software engineering practices are embedded in the organization.

13.7 Review of Audit Activities

The results of the audit activities will be reviewed with management on a periodic basis. Audits provide important information to management on the processes being used in the organization; the extent to which they are followed; and the extent to which they are effective.

An independent audit (usually a third party or separate internal audit function) of SQA activities may be conducted to ensure that the SQA function is effective. Any non-compliance issues identified and assigned to the auditor and quality manager for resolution.

13.8 Other Audits

The audit process that we discussed has been focused on process audits conducted during a project. Other audits that may be conducted include supplier audits, where the auditor visits the supplier to determine the extent to which they are following the agreed processes and standards for the outsourced work.

The SQA team is often the point of contact to facilitate customer audits, where an audit team from the customer visits the organization to determine the extent to which they are following processes and standards.

13.9 Review Questions

• 1. What is the purpose of an audit?

• 2. What planning is done prior to the audit?

• 3. Explain why the auditor needs to be independent?

• 4. Describe the activities in the audit process.

• 5. What happens at an audit meeting?

• 6. What happens after an audit meeting?

• 7. How will the auditor deal with a situation where the audit actions are still open after the due date?

13.10 Summary

The purpose of software quality assurance is to provide visibility to management on the processes being followed and the work products being produced in the organization. It is a systematic enquiry into the way that things are done in the organization, and it involves conducting audits of projects, suppliers, and departments.

It provides visibility into the processes and standards in use, their effectiveness, and the extent of compliance to them. It involves planning and conducting audits; reporting the results to the affected groups; tracking the assigned audit actions to completion; and conducting follow up audits, as appropriate. It is generally conducted by the SQA group, and this group is independent of the groups being audited.

The audit planning is concerned with selecting projects/areas to be audited, determining who needs to be involved and dealing with the logistics. The audit meeting is a formal meeting with the audit participants to discuss their specific responsibilities in the project, the processes followed, and so on.

The audit report details the findings from the audit and includes audit actions that need to be resolved. Once the audit report has been published the auditor will track the assigned audit actions to completion, and once all actions have been addressed the audit may then be closed.