Report 404 Review of Auditor-General’s Reports

Introduction

Background

Climate change is recognised as a major issue for all nations. New and stronger evidence that humans are having an influence on the global climate through greenhouse gas emissions was presented by the Intergovernmental Panel on Climate Change in 2001.¹

In 1997, and subsequently in 1999, the Australian Government introduced two major spending packages with a total value of almost $1 billion. These packages, Safeguarding the Future (1997) and Measures for a Better Environment (1999), were designed to address the challenges posed by the issue of climate change and to meet Australia’s domestic and international climate change commitments. The Australian Government has agreed to ‘develop and invest in domestic programs to meet the target of limiting greenhouse gas emissions to 108 percent of 1990 emissions over the period of 2008–2012’ – the Kyoto Protocol target.

Since its inception in 1998, the Australian Greenhouse Office (AGO) has been responsible for the implementation of these two major packages and subsequent government greenhouse policies. The agency’s mission is to lead Australia’s greenhouse action to achieve effective and sustainable results. AGO seeks, amongst other things, to facilitate projects that maximise cost effective greenhouse gas abatement and reduce growth in greenhouse gas emissions.

The ANAO audit was conducted between May and August 2003. Since then, the administrative arrangements for the AGO have changed. Following the 2004 Federal Election, the AGO was moved to become part of the Department of Environment and Heritage (DEH). As such it ceased to operate as an Executive Agency and is now a division within DEH.

The audit

The objective of the audit was to examine and report on the administrative efficiency and effectiveness of significant programs administered through AGO. The audit examined seven programs across both the 1997 and 1999 packages, which accounted for 87 percent of total program cost estimates.

Since the ANAO’s audit report was tabled, the government has announced a new suite of climate change responses as part of its energy white paper, Securing Australia’s Energy Future (released June 2004), and also in the 2004-05 Federal Budget. While some existing AGO programs remained untouched, others were refined and some new programs were announced, such as the Solar Cities Programme. This new package is known as the government’s Climate Change Strategy. Total funding under the Climate Change Strategy (including existing funding from the 1997 and 1999 programs) now totals $1.8 billion.²

The programs reviewed in the ANAO’s audit are outlined in the table below.

Table 3.1 AGO programs examined in ANAO Audit Report no. 34

Source ANAO Audit Report No. 34 2003-04, p. 28.

Audit findings

The ANAO found that the administration of greenhouse programs had been characterised by substantial challenges.

The ANAO reported that administrative processes could have been better focused at the planning stage on comprehensive risk assessment, as well as in designing programs with more measurable objectives and targets. The absence of these factors made it difficult to measure results against program objectives and exposed some programs to risks that could have been better identified and treated in the early stages of the programs.

The ANAO found that catch-up administrative improvements had been put in place to overcome initial shortcomings in planning. Project appraisal and selection was generally rigorous and based on merit. AGO had put in place sound and well drafted agreements to manage residual risk at the program level. Monitoring and evaluation were also given sufficient priority. Linking payments to milestones assisted in the efficient management of funding allocations.

Nevertheless, ANAO concluded that substantial risks remained—particularly in terms of the timely achievement of program objectives. Areas for further improvement included refining performance measurement, including the use of intermediate measures and/or assessments to gauge progress towards longer term objectives. The ANAO also found that a more consistent approach to project appraisal and selection would also assist in improving the transparency of decision-making. Attention also needed to be given to the timeframes of negotiations over funding agreements. Finally, the ANAO reported that improvements to performance reporting were necessary to enable Parliament to come to a more informed view on the progress and effectiveness of AGO in implementing programs of national significance.

ANAO recommendations

The ANAO made five recommendations:

The AGO agreed with each of the recommendations and reported to the Committee on its progress in meeting each recommendation.

The Committee’s review

The Committee held a public hearing on 14 February 2005 to review the progress made against the audit’s recommendations. Witnesses from the ANAO, AGO and Commonwealth Scientific & Industrial Research Organisation (CSIRO) attended the hearing.

The Committee took evidence on the following major issues:

• risk assessment;
• measuring outcomes;
• selection and appraisal of projects;
• management and monitoring of agreements; and
• performance reporting.

Risk assessment

The ANAO’s publication Administration of Grants Better Practice Guide notes that effective planning is the cornerstone of an economic, efficient and effective program. The ANAO’s review of the 1997 and 1999 greenhouse funding packages found ‘substantial shortcomings in the initial risk assessments’.³ While this lack of risk assessment did not have significant ‘downstream’ implications for most of the programs examined by the ANAO, it had important consequences for two programs – the Photovoltaic Rebate Program (PVRP) and the Alternative Fuels Conservation Program (AFCP).

Photovoltaic Rebate Program

Photovoltaic systems convert sunlight into electricity. Under the Photovoltaic Rebate Program (PVRP), domestic householders and small community organisations could apply for a rebate of $8,250 against the costs of installation of a photovoltaic system.⁴

The ANAO found that the announcement of the PVRP created an immediate expectation in the market for the pending subsidy. However, the AGO was not ready to distribute subsidies, because appropriations and delivery agreements were not in place. As a result, there was an initial slump in sales of photovoltaic units prior to the introduction of the program.⁵

When the program became operational in January 2000, the pent up demand then exceeded supply. The AGO ran out of funds for the PVRP program, and was forced to reallocate resources from another greenhouse program to cover liabilities. This was undertaken with Ministerial approval. In order to slow down demand, the rebate amount was also reduced to $4 per watt, and a cap of $4,000 per domestic household was put on the rebate.⁶

The ANAO reported that the AGO had responded to the program challenges within a reasonable timeframe. However, ANAO stated:

…these findings highlight the importance of undertaking an early risk assessment. In this case, a risk assessment may have assisted in identifying the level for demand for the program and the need for strengthened controls on expenditure.⁷

The Committee notes that while the PVRP program was due to be phased out, the 2005-06 budget provided for an extra two years of PVRP, at a cost of $5.7 million per year. The extra funding is intended to provide a ‘bridge’ between the PVRP program and the new Solar Cities program announced in the Energy White Paper, which is due to commence in June 2006.⁸ The bridging program will help to allay solar energy industry fears of a consumer slump leading up to the introduction of the Solar Cities program, as was experienced prior to the introduction of the PVRP in 1999/2000.

Alternative Fuel Conversion Program

The ANAO also found that poor risk assessment affected the operation of the Alternative Fuel Conversion Program (AFCP). Under its original design, the AFCP aimed to reduce greenhouse gases and improve urban air quality by facilitating heavier commercial road vehicles and public transport buses to operate compressed natural gas (CNG) and liquefied petroleum gas (LPG). The program was initially funded for $75 million from 2000 – 2008, and was intended to provide a rebate of up to 50 per cent of the cost price of converting an existing commercial vehicle to CNG or LPG, or upgrading to a new CNG or LPG engine.

However, the ANAO found that although the AGO had consulted with industry in designing the scheme, the assessment proved to be optimistic and unrealistic, given the low uptake of available grants.

A key factor in the low uptake (particularly for CNG conversions or upgrades) was an absence of refuelling infrastructure to support CNG, and low consumer and industry confidence in CNG and LPG for heavy vehicles.

The AGO undertook a review of the program in November 2001, 15 months after the program commenced. This review was praised by the ANAO. However the ANAO did conclude that had AGO undertaken a more thorough risk assessment in the early stages of the program, some of the later issues could have been avoided:

In particular, the need to have a robust consultation and analysis to test the accuracy of sectoral industry advice is a key lesson learned from this program.⁹

The AGO stated that its review of the AFCP program had resulted in better targeting of the available funds. AGO reported that the redesigned AFCP program:

…focuses on pilots with significant fleet operators – for example, PO, Coates, Murray Goulburn and a couple more. We have agreements with those companies to actually work with them to develop a business case and to pilot the technology to facilitate import of engines, and the revamped program is actually working very well.

So we actually consider AFCP to be case of best practice where we monitored, we identified slow uptake, we undertook a review, we put the findings of the review and proposed redesign to our minister and that was then taken to the budget and the program was redesigned. We are now effectively progressing the development of those technologies.¹⁰

AGO response

In response to the ANAO’s recommendation regarding risk assessment, the AGO developed a Risk Management Policy. According to the AGO’s 2003-04 Annual Report, the Risk Management Policy recognises that risk management is a key component of effective corporate governance. In response to Committee questioning on risk assessment, the AGO advised that it has conducted comprehensive program risk assessment for all new climate change measures announced in the 2004-05 Federal Budget and the Australian Government’s energy white paper, Securing Australia’s Energy Future.¹¹

Measuring outcomes

As noted by the ANAO, measurable and precise objectives provide a solid foundation for effective performance management and accountability.¹² The ANAO found that the key objectives for most AGO programs were broad and not easily measurable. However, there were several good examples, such as the Greenhouse Gas Abatement Program (G Gap).

The G Gap program aims to reduce Australia’s net greenhouse gas emissions by supporting activities that are likely to result in substantial emissions reductions or activities to offset greenhouse emissions, particularly in the period 2008-2012. The G Gap program provides assistance to private sector projects which will lead to greenhouse savings or offsets.¹³ At the time of the ANAO’s audit, two rounds of the G Gap program had been completed, with a third and final round now nearing finalisation (at April 2005).¹⁴

The ANAO praised the G Gap program for including operational targets which underpinned broad objectives. ANAO commented that this was good practice and particularly useful in providing an indication of progress towards objectives.¹⁵

Other programs with operational targets included Greenhouse Challenge and the AFPC. However, for other programs, ANAO found that it was difficult to gauge to what extent programs could be achieved, particularly where there were long lead times involved. For example, for the PVRP there was no measurement of:

• to what extent the PVRP would assist in the development of the Australian photovoltaic industry; or
• by how much the PVRP would increase the use of renewable energy in Australia and reduce greenhouse gas emissions.

The AGO agreed with the ANAO’s recommendation to incorporate clearly defined and measurable intermediate outcomes and operational targets, where possible. However, in the ANAO’s report and also before the Committee, the AGO noted the difficulties in doing this:

For a large number of these projects – for example, with the development of renewable energy technologies – it is not possible to measure any greenhouse outcome or any other interim measure except where a milestone, be it the delivery of a boiler or the construction of a component of plant, has actually been met. It is not until the commissioning or the end of a project when different technologies are linked together that you can actually start to measure the greenhouse outcomes.¹⁶

However, AGO reported that the ANAO’s recommendation had been taken into account in measures announced in the 2003-05 budget, and in measures in the Energy White Paper.¹⁷

Selection and appraisal of projects

Appraisal and selection processes for grants must be transparent and lead to the selection of projects which represent value for money against program objectives and outcomes. The ANAO praised the AGO’s appraisal system for the programs covered by the audit. ANAO found that the AGO brought in specialist advice where needed.¹⁸

The ANAO noted one problem in project appraisals, where a grant of nearly $1 million was awarded for a project that was near completion. Although the funding was not critical to the implementation of the project, AGO advised the Minister that the grant should still go ahead, as there was nothing in the program guidelines to preclude funding for projects which are already fully committed.¹⁹ The AGO re-worded its guidelines for subsequent funding rounds.

The ANAO stated that the appraisal process for all future projects should contain a criteria related to a project’s need for financial assistance to proceed. If there is no need for financial assistance, the ANAO argued,

…funds paid in such circumstances provide no added value and represent an opportunity cost to the Australian Government.²⁰

The Committee agrees with the ANAO’s argument and calls for the AGO and other agencies to ensure that they support only those projects which genuinely require government assistance to proceed.

The ANAO found a problem in documentation of decisions in selecting projects for the Greenhouse Gas Abatement Program (GGAP). While the AGO grouped projects into risk categories when presenting them to the Minister for decision, unlike other programs, the AGO provided no recommendations for Ministerial decision makers. This was specifically because of a Ministerial request to not provide recommendations on GGAP projects.²¹

The ANAO found that Ministerial decisions for G Gap funding allocations could have been better documented. Greater detail in the reasons for decisions would have assisted in explaining why some projects received funding given the level of risks raised in the appraisal brief. The ANAO recommended that the AGO apply an order of merit rating scheme for projects, and recommendations on selection that highlight projects that are most likely to achieve project objectives.

The Committee asked the AGO whether it knew the reasons why the Ministerial Council had requested that selection briefs for GGAP programs not provide recommendations on projects or a suggested order of merit. The AGO replied that the minutes of the Ministerial Council meeting at which decision-making procedures were set down (16 February 2000) do not record an explanation for the Council’s decision.²²

The AGO advised the Committee that the ANAO’s recommendation, regarding advice to Ministers on proposed project funding, was undertaken for the third and final round of the GGAP program.²³

The AGO also advised that, at the time of the Committee’s hearing, the guidelines for administration of the new greenhouse abatement programs arising out of the Energy White Paper were still being signed off by Ministers. The Committee would like to ensure that the guidelines for these programs reflect the ANAO’s recommendations on assessment and selection procedures for project funding.

Management and monitoring of agreements

The ANAO found that, in general, the agreements between the AGO and program participants were well designed and appropriate for the specific design of each program. However the ANAO noted a number of specific examples where agreements could be strengthened. These included:

• the need to recognise foreign exchange risks in agreements to enable increase or decrease of grants according to currency variations;²⁴
• lengthy negotiations over funding agreements. The circumstances in which the project won funding approval (for example, market conditions) may change over a long period. The ANAO considered that one option was to set a deadline for negotiations to be completed with particular applicants, after which time funds would be reallocated to future funding rounds or alternative projects;²⁵ and
• strengthening the link between milestone payments and program outputs. In a $1.75 million AFPC project involving a bus company purchasing new Compressed Natural Gas (CNG) buses to replace a diesel fleet, payments were made against milestones which did not ultimately provide a greenhouse benefit, although they did result in improvements to air quality. The ANAO considered that in future, the AGO should withhold a portion of the grant funds, to be released at the end of the project when testing proves that greenhouse gas abatement has occurred as a result of the project.²⁶

The Committee believes the ANAO’s points are important to ensure the effective administration of project funds and that taxpayers are receiving maximum value for money. Accordingly, the Committee makes the following recommendation:

The ANAO analysed the performance monitoring of each AGO program included in its audit. The ANAO found that ‘performance monitoring to date has been thorough and given the necessary priority. It is too early to tell if programs such as GGAP, RECP, REEF will achieve their objectives in the longer term.’²⁷

Performance reporting

The ANAO found that there was room for improvement in the AGO’s annual reporting. The AGO’s annual reports were largely focused on inputs and activities, rather than reporting against targets for programs and reporting of trends and changes over time. For example, the AFCP had a target of 800 buses and 4,000 commercial trucks to be converted to CNG or LPG gas in each of the four years of the program. The AGO’s 2002-03 annual report stated that 568 buses had converted or been purchased, and that this represented a 150 per cent increase since the program’s inception. However, the AGO did not report against the original target of 800, which ANAO felt ‘could give the reader a misleading impression about the achievements of the program’. The actual performance on the number of trucks converted (or purchased) is not reported.²⁸

The ANAO also found that the AGO’s annual reports had a lack of year-by-year comparisons on performance. The ANAO felt these comparisons were particularly important given the long lead times involved in achieving results of between three and 13 years.

The ANAO noted that the suite of programs managed by the AGO have a target of abating 67 Mt of CO2 each year. The ANAO would like the AGO’s annual report to include a progress report towards the target from all the programs funded by AGO.

The AGO agreed with the ANAO’s recommendation to enhance public reporting, and told the Committee that its 2003-04 annual report included the following improvements:

• a profile of the Government’s climate change measures, with expected abatement;
• a summary table showing results against performance measures for each output;
• case studies to highlight the practical application of government program funds (including an analysis of a program which did not meet expectations); and
• more contact and analysis of the climate change issues, government policy/program response, and challenges facing the organisation.²⁹

At the hearing, the AGO also told the Committee that it had improved its performance reporting via new performance standards in the 2004-05 Portfolio Budget Statements (PBS), which will be reported against in the 2005-06 Department of Environment and Heritage annual report. For example, the AGO’s OutPut Group 2 – Emissions Management, includes the Renewable Remote Power Generation Programme, Photovoltaic Rebate Programme, Renewable Energy Commercialisation Programme, Greenhouse Gas Abatement Programme and others which were not reviewed by the ANAO’s audit. These programs will now be subject to performance reporting against criteria such as those outlined in the table below.

Table 3.3 Australian Greenhouse Office Portfolio Budget Statement 2004-05 – Emissions Management

Source Australian Greenhouse Office, Portfolio Budget Statement 2004-05, pp. 188 – 189.

The Committee looks forward to more detailed performance reporting from the AGO in the 2005-06 Annual Report from DEH.

Australian Greenhouse Office, internet site: http://www.greenhouse.gov.au/ago/index.html, accessed May 2005. Back

Australian Greenhouse Office, internet site: http://www.greenhouse.gov.au/ago/index.html, accessed May 2005. Back

Minister for Environment and Heritage, Media Release 8 February 2000, available at: http://www.deh.gov.au/minister/env/2000/mr8feb00.html, accessed May 2005. Back

ANAO Audit Report no. 34, 2003-04, p. 33. Back

ANAO Audit Report no. 34, 2003-04, p. 34. Back

ANAO Audit Report 34, 2003-04, p. 35. Back

Australian Greenhouse Office (AGO), Transcript of Evidence, 14 February 2005, p. 6. Back

Australian Greenhouse Office, Annual Report 2003-04, pp. 66-67; and Australian Greenhouse Office, Submission no. 5.1, p. 1. Back

ANAO Audit Report no. 34, 2003-04, p. 36. Back

Australian Greenhouse Office, Annual Report 2003-04, p. 24. Back

ANAO Audit Report no. 34, 2003-04, p. 37. Back

AGO, Transcript of Evidence, 14 February 2005, p. 4. Back

AGO, Submission no. 5.1, p. 1. Back

ANAO Audit Report no. 34, 2003-04, p. 50. Back

ANAO Audit Report no. 34, 2003-04, p. 50. Back

ANAO Audit Report no. 34, 2003-04, p. 50. Back

ANAO Audit Report no. 34, 2003-04, p. 55. Back

AGO, Submission no. 5.1, p. 3. Back

AGO, Submission no. 5.1, p. 2. Back

ANAO Audit Report no. 34, 2003-04, p. 78. Back

ANAO Audit Report no. 34, 2003-04, p. 81. Back