Home

The Prize 2009 - Review Team Report

This review report is focused on the assessment and the qualities of the 2009 winning performance audit report. The comments from the review team on the two other nominated reports are sent separately to each SAI.

The Prize - Introduction

To promote the development of performance audit AFROSAI-E has, in cooperation with the Swedish National Audit Office (SNAO) decided to launch the SNAO Prize for the best performance audit report of the year 2008 and the coming years. The purpose of the prize is to promote sharing of experiences and benchmarking best practice in performance audit but also to highlight the efforts done by performance auditors within the region.

Each SAI within AFROSAI-E has been invited to nominate one performance audit report for assessment.

At the AFROSAI-E Secretariat a formal assessment of the nominated reports has been done making sure they fulfil the requirements made out – such as acceptable production time and publication during 2009. No assessment of the quality is done at the Secretariat.

The Review Team

The quality of the performance audit reports nominated has been assessed by a review team organised by the SNAO consisting of members from SNAO as well as from other SAIs.

The AFROSAI-E Secretariat is not represented in the team. All the members of the review team are from a SAI involved in bilateral co-operation with SAIs members of AFROSAI-E.

We, the members of the team, are:

  • Harry Kramer, the Netherlands Court of Audit, Sector Manager for Performance Audit and EU audits
  • Jens-Are Enoksen, OAG Norway, Long Term Adviser in Performance Audit at the OAG in Zambia
  • Jeremy Lonsdale, NAO England, Director-General for Performance Audit
  • Göran Hyltander (Chairperson), SNAO, Director

The assessment has been carried out using the Lima declaration, the INTOSAI Implementation Guidelines for performance audit, INTOSAI Ethical Code of Ethics and Auditing Standards, the AFROSAI-E Manual for Performance Audit as well as our experience from performance audit.

Nominated reports

Three nominated audit reports have passed the formal assessment done at the AFROSAI-E Secretariat, namely:

  • “The processing of Terminal Benefits of Retirees at Central and Local Government of Tanzania”,  from the Controller and Auditor General of Tanzania
  • “Service delivery to orphans and vulnerable children in the country”, from the Auditor General of Namibia
  • “The integration of retrenched civil servants in the private sector by reconversion program under the Ministry of Public Service and Labour”, from the Auditor General of State Finance of Rwanda

Quality Criteria

In principal, the following criteria have been used when assessing the performance audit reports. As the assessment has been based only on the performance audit report, there have been cases where the review team has not been able to use all criteria to its full extent.

Quality of presentation

The report should be well written, in accordance with the INTOSAI and AFROSAI-E reporting standards and principles. Appropriate tables, charts, etc., should be used to help the reader understand the findings, conclusions and recommendations.

Timeliness

The audit report should be prepared and issued in a timely fashion in order to be of greatest use to readers and users, particularly Government and the audited entity which have to take the necessary corrective actions. It is therefore important that the report is published within the time schedule set up in the work plan. Unless the report is restricted by law or regulation, or contains privileged or confident information reports must be made public.

Completeness

The audit report should contain all information and arguments needed to fulfil the audit objective and provide answers to the audit questions. The relationship between audit objectives, findings and conclusions needs to be verifiable, complete and clearly stated.

Accuracy

Evidence presented in the audit report should be true and comprehensive and all findings correctly and logically portrayed. The need for accuracy is based on the need to assure the readers that what is reported is credible and reliable.

Objective and logical

The presentation of the report should be balanced in content and tone. For the credibility of the report, all evidence should be presented in an unbiased manner. Auditors should be aware of the risk of exaggeration and overemphasis of deficient performance. The audit report should only present arguments that are logically valid in a convincing manor.

Legitimacy

The report should not stray outside the SAI’s legal remit. There should be no questioning of politically decided objectives.

Balance of the various sections

The report should concentrate on the important aspects of the auditees’ management and performance and avoid details on minor issues.

The quality criteria have been transformed to a limited number of questions. Of course, neither the quality criteria nor the questions used by the team cover every important aspect of quality but it has given us good guidance. The questions are based on the condition that the review team has access to the audit reports only. It is also a condition that the review should be focused on the job done by the audit team.

After internal discussions the review team has come to an agreement on the following. Each reviewer does his assessment separately. The assessments are compiled, compared and discussed. The nominated reports are evaluated. We select a winner and we present our opinion on each report in short.

Assessment

The second year of the Prize, three nominated reports were accepted. In that way this year the competition for the Prize is limited.

We have made separate assessment of the three nominated performance audit reports. The reviewers ranked the three reviewed audit reports in the same way. The judgement is unanimous.

The reports have all with a high ambition focused efficiency problems in the public service and they have come up with substantial findings and recommendations. It has been quite a hard job to select a winner.

The winner 2009 is the nominated performance audit report from the Auditor General of Namibia.

  • Service delivery to orphans and vulnerable children in the country” from the Auditor General of Namibia
  • The team that carried out the audit comprised:  Auditors Ms J. Kahona and Mr A. Hasheela and Chief Auditor Ms G. Nelwamondo.
  • The report was reviewed and edited by Deputy Director Ms M. Hummel, Director Mr D.J. Kotze and Deputy Auditor- General W.P. Barth.

Comments on the report from Namibia

The report on “Service delivery to orphans and vulnerable children in the country” is well structured. The report is a pleasure to read. You easily gain a comprehensive view of the content. The report is focused on a specific limited issue. It has a good summary and clear explanation of the basic problem.

The rationale for the audit is well explained and reasonable. The basic data are set out simply at the front along with contextual information. The audit methods used are methodologically reasonable. The findings are substantiated against relevant audit criteria and the reader gets a good impression of the audit work carried out. The audit object is the Directorate of Child Welfare, but the responsibility of the superior body, the Ministry of Gender Equality and Child Welfare, is also clearly stated both in the conclusions and the recommendations. It should have been of interest to have the comments from the auditees included in the report.

The audit scope and the depth of the analysis provide useful lessons for future audits. There will be similar organisations and problems for which the major finding of this audit have relevance – the issue of vacant posts; the functioning of the general budget system, the system for recruiting staff and management shortcomings.

Other issues which could have been embraced in a wider scoped piece of work are the misuse of the control system, the consequences of the legislation for the care of orphans and vulnerable children and the implementation of the legislation; the possible lack of consistency between legislation and the system for budget allocation. Within the chosen scope, the auditors might have considered deepening the analysis of the reasons why vacant positions are not filled. A deeper analysis might have given both Parliament and the Ministry a better understanding of what to do to solve the problem with the vacant posts. Notwithstanding these comments on width and depth, it is a good report.

It is a well focused report with a limited issue which gives Parliament a clear indication that the system for service delivery to orphans and vulnerable children is not functioning as intended. Parliament may see the report as a good reason to ask for further analysis and action from the Government.

Overall comments on the three reports nominated

Generally the standard is higher than last year. The audit issues are, as far as the jury can assess, important issues in the respective countries. The ambition in method and scope is higher. Presentation is generally of good standard. The justifications for the audits and the description of policies and systems are clearly made, as are the presentation of the audit results. All the audit teams have taken a serious effort, worth respect from Parliaments and auditees, to make their audits a performance audit of good quality.

For future performance audits some issues are of importance to consider. The scope of the audit: the width of the design and the depth in the analysis of the reasons for the malfunctioning. The challenge for an audit office is to create new knowledge in some reports as well as confirming indications and existing common knowledge in the society in some reports. The use of a pre-study to go further in the identification of relevant problems and to get a good design and adequate methods is valuable. In conclusions and recommendations the focus should be on the causes and the recommendations should have more of an action-perspective and not only a normative perspective.