Report 404 Review of Auditor-General’s Reports

Introduction

The Standard Defence Supply System (SDSS) version 4 is a computer program that spans the three Services in its coverage of logistics management. SDSS is intended to be a key information system for financial management of Defence assets, and equally importantly, to facilitate Defence’s materiel management capability. Some key characteristics of the SDSS system are that it includes:

• 1.7 million items catalogued;
• $2.0 billion in Inventory;
• $7.5 billion in Assets;
• over 14,000 registered users (8,000 active users, approximately 1,600 concurrent daily users);
• 144 Districts (separate geographic business units);
• 1,200 warehouses;
• 600,000 transactions per day, 5,000 picking slips per day, 6,800 batch runs per day, 76,000 demands per week; and
• 250,000 purchase/transfer/workshop orders per year.¹

In keeping with Defence policy, the ANAO has assessed that the system qualifies as a strategic system. The initial SDSS program (then titled the Supply System Redevelopment Project) was rolled out in 1992, as the first joint logistics management process for the ADF. In 1992, the Joint Committee of Public Accounts reported on the Supply System Redevelopment Project, and found that there was poor administration and management of the project, the result of which was that sub-elements of the project continuously fell behind schedule.

In July 2000, the SDSS version 4 Upgrade Project (the Project) was initiated with an approved budget of $15.87 million with the main aim of delivering a Standard Supply Chain System across Defence by June 2002. The Project incorporated a new version of the operating software and improvements to the management of the Defence supply chain and its infrastructure. The aim was to provide a system for the management of the Defence spares inventory, valued at $1.9 billion.

As of November 2003, the Project had incurred costs of $49.9 million excluding $5.1 million in contract residuals contributed by e-Procurement and SDSS version 3 legacy training projects. Defence advised the ANAO that the formal Project closure was dependent on the delivery of the financial reporting functionality expected of the SDSS version 4 system.

Audit objectives

The objective of the audit was to undertake a performance audit of the project management environment governing delivery of Defence business information system projects, with specific reference to the SDSS Upgrade Project. The audit addressed the scope of the system being delivered, with specific regard to its ability to meet end user capability requirements. The audit was presented to Parliament in August 2004.

Audit conclusions

The ANAO found that the Project has not delivered value for money to Defence. The Project exhibited extensive scope reduction and the final schedule (at June 2004) was more than two years over the planned schedule. SDSS version 4 was intended to provide Defence with improved finance functions, tighter controls over data integrity and transaction processing, and improved reconciliation and reporting. The ANAO concluded that the Project failed to materially deliver many of the outcomes for which it was funded.

At the time of the ANAO’s report (August 2004), the project was still incomplete. The escalating cost of the project (excluding $5.1 million in contract deliverables from legacy training and e-Procurement projects) required a further allocation of $34 million towards what had originally been approved as a $15.87 million project. By November 2003, the Project had already exceeded its initial approved budget by more than 200 per cent. This excluded further funds earmarked for the SDSS Get Well Program. Defence has advised the ANAO that the anticipated delivery date for the Get Well Program remediation activity is December 2005.

The ANAO found that the delivered system did not satisfy many of the end user expectations. Significantly, the system was ineffective in its ability to manage Defence stock holdings to the extent originally envisaged, and restricts Defence’s ability to fully account for them. The system did not adequately alert appropriate Defence logistic management staff that strategically important stock holdings had fallen below levels able to support Defence operational requirements. Reports of this nature were not automatically routed to materiel managers responsible for replacing used stores. The ANAO believed that without appropriate workarounds, these shortcomings compromised Defence's ability to assure operational Force Element Groups that the stores necessary to implement their stated operational requirements could be delivered as required to support operational readiness.²

A Department of Defence minute to the Defence Minister dated March 2004 stated that:

the current operational status of SDSS indicates that it is below minimal levels of functionality…the SDSS version 4 upgrade, supposedly completed in July 2003, is non-performing and for some reason has actually taken progress backwards.³

ANAO recommendations

Table 8.1 ANAO recommendations, Audit Report no. 5, 2004-05

Establishment of the project

The project commenced in June 2000 as an Equipment Acquisition Strategy to upgrade the existing operating system upon which the Defence logistics management system was based while, concurrently, upgrading the business rules to roll out a Single Supply Chain Management System, and introducing changes to the financial records of the system to enable it to comply with accrual accounting standards. The Equipment Acquisition Strategy was approved with an estimated project cost of $27 million, and work began on the system upgrade in November 2000.

Defence manages acquisition projects under two main categories: Major Capital Equipment, which, at the time the SDSS upgrade was undertaken, was centrally located and managed by the Defence Acquisition Organisation in Canberra; and Minor Capital Equipment projects, which were controlled by any of the then 14 Defence Groups, which included the Support Command Group.

The ANAO found that the SDSS upgrade satisfied the conditions for classification as a strategic procurement activity (meaning it would deliver an outcome critical to Defence’s ability to meet its core objectives), and thus treatment as a Major Capital Equipment Procurement activity. The risks of program failure were high, and the costs associated with delay were also high. The procurement activity was very complicated, extending across more than 50 individual contracts of varying nature and complexity.

Approvals

Despite the complexity outlined above, the SDSS upgrade was raised as a Minor Capital Equipment acquisition project from operating funds. This decision was taken irrespective of the Equipment Acquisition Strategy, which estimated the cost associated with implementing the stated upgrade outcomes as being $27 million which would, at the time, have required the upgrade project to be approved by Cabinet, and managed as a Major Capital Equipment procurement activity. The following table details the approval required by the Defence procurement manual (CEPMAN-1), and the actual level of SDSS approvals:

Table 8.2 Defence procurement manual project approval guidelines

Source Audit Report no. 5, 2004-05; p. 40.

The ANAO report stated that Defence governance procedures have recently been strengthened to ensure that all strategic capability procurement exceeding designated limits will be referred for Ministerial consideration.

Governance

The SDSS upgrade project was established without a formally appointed Project Board, even though this was required by the Defence Equipment Acquisition Strategy. The Board was eventually established by July 2001. Defence reported to the ANAO that the original Board membership did not include a wide representation of user interests, or representation of Boards governing interdependent projects. However, Defence reported that it had made changes to the project governance board for the Get Well project (discussed further below).

Project scoping and management

PricewaterhouseCoopers Consulting (PwCC) undertook a scoping study in 1999, outlining proposals for upgrading the SDSS system. The Equipment Acquisition Strategy (June 2000) was based on the PwCC report.

Defence has since identified inadequate scoping as one of the main reasons for the cost blowout in the SDSS project. The Chief Executive Officer of DMO told the Committee:

It would appear that the project was not adequately scoped. It would appear that there was a march off into the grey never-never…without knowing quite what was out there. There was also an element of ‘wish list’, where the project was underscoped. In other words, people said ‘I wish I had this; I wish I had that’. Part of the cost blow-out was because some of those wishes were granted.⁴

In the Equipment Acquisition Strategy (based on the PwCC scoping study), Defence identified that it did not have the staff required to manage the project. Support Command Australia therefore approved the outsourcing of the project management to a Project Management Organisation (PMO).

Following a competitive tendering process involving three firms, PwCC was engaged as the PMO for the Project, and Dimension Data was engaged as the training development and delivery contractor. In November 2002 the PMO role was novated to IBM Business Consulting Services (IBM BCS) to coincide with the IBM acquisition of PwCC.

Under the PMO contract won by PwCC, project management costs increased from a projected cost of $5.2 million (July 2000) to $26.3 million by November 2003. This represented 47 per cent of the final project costs.

The Committee was concerned to learn that PwCC had scoped the SDSS project and then won the tender for project management. Defence later acknowledged that the initial scoping study was inadequate, which largely led to the blow-out in the management costs. Defence acknowledged the problems inherent in such an arrangement:

Yes, there is a potential for conflict in that sort of arrangement. That is why we will not be doing that sort of arrangement in the future.⁵

Defence advised the Committee that future projects such as the JP2077 replacement logistics system (discussed further in this chapter), will be project-managed internally by DMO staff rather than by a contracted company.

Contract management

The ANAO found that the contractual construct chosen for the project was deficient. The decision to retain a contracted PMO, on hourly rates, for a high-risk software development and roll out program during Phase 2 of the Project, proved to be inappropriate, and did not shift adequate risk to the PMO . A large proportion of the costs associated with the delays experienced by the Project were consumed by the PMO.

The PMO had no direct contractual authority over any of the internal, Defence suppliers to the Project, and limited contractual control over commercial suppliers, yet was expected to accept responsibility for the management of deliverables.

Defence acknowledged that there were insufficient numbers of project management staff to perform all the tasks required to ensure that internal suppliers delivered the required information to the PMO, and that the staff available did not have the skill sets necessary to complete Project tasks on time.

ANAO found that had the project been run as a Major Capital Equipment procurement activity, the documents required to manage the internal defence suppliers would have been produced as a matter of course during the approvals process.

According to ANAO, ‘the inability for the contracted PMO to effectively direct Defence identities, and the lack of required service by Defence suppliers, contributed to a large degree to Project delays.’⁶

Time and cost increases

The escalation of costs associated with the project required additional funding. Defence took the decision to fund the Project from disparate sources, initially from Support Command operational funding, and then from the emerging Project JP 2077 (designed to streamline the logistics support to the ADF through improvements to logistics information management systems).

When it became obvious that the available Project funds were not sufficient to complete the Project, further funding of $15 million was approved by the Minister for Defence with the Minister for Finance’s concurrence in October 2002. When rollout threatened system operability during the Defence deployment to Iraq in early 2003, Defence delayed rollout, on two separate occasions, which increased the Project cost by a further $8 million. The Minister for Defence approved the further $8 million cost increase.

The ANAO found that Defence reduced the scope of the project to ensure that Ministerial approval was secured in time to meet its existing contracts. If Defence had sought funding in excess of $20 million, Cabinet approval would have been required. By requesting $15 million, Ministerial approval (by the Minister for Defence and the Minister for Finance), was sufficient.

The ANAO found that the reduction in scope served to diminish the system’s capability at delivery. The ANAO also found that ‘management decisions to redirect allocated resources to cover increases in management expenses eventually contributed to poor network performance, loss of functionality, and loss of system acceptance by end users.’⁷

Committee comment

Throughout this inquiry the Committee has been extremely concerned to learn of the poor planning, implementation and management of this project. Many of the problems stem from the fact that it was initially raised as a Minor Capital Acquisition Project, rather than as a Major Capital Acquisition Project, as was fitting given the amount of money initially budgeted, and the importance of the system to Defence’s logistics management.

Approvals were not initially sought at Cabinet, Ministerial, or even Departmental head level, despite the project being estimated at $27 million. The project was scoped by an external provider (PriceWaterhouseCoopers Consulting), which eventually won the contract to undertake the SDSS upgrade. It has since been acknowledged that a major problem with the entire project was that it was poorly scoped in the first place. The contract allowed major cost blowouts to the Project Management Organisation, because of hourly rates for a high-risk software and rollout program.

When cost began to become a major problem, Defence ‘down-scoped’ parts of the project to ensure that the extra funding required came in under $20 million, and therefore did not require Cabinet approval. This downgrading of some parts of the project directly contributed to end-user problems such as slow network speeds. These problems subsequently needed to be fixed in the Get Well project.

The problems uncovered by the ANAO and again investigated by this Committee are simply unacceptable. Defence has assured the Committee that for future projects, planning and project management will be better managed. The Committee believes that in particular, proper approvals at Ministerial or Cabinet level must be obtained. Failures in this respect, in the Committee’s view, warrant that someone be held accountable and appropriate disciplinary action be undertaken.

The Committee would also be extremely concerned if project management functions were outsourced to an external provider, particularly on flexible contracts with provisions for hourly rates and other costly measures.

Delivery management and ongoing support

The ANAO found that the cultural changes required to bring about adoption of the new SDSS system were not accomplished by either the Defence Project Office or the Senior User Group. As a result, Defence told Senate Estimates hearings this year that many ADF personnel are not using the upgraded system.

…[SDSS] is working where it works – that is, where the management of the warehouse, the workforce, is prepared to use the system properly it can do the job, but it is the training and discipline issue that has to back it up.⁸

…during the upgrade process the system performance degraded, the willingness of people to use the system directly dropped off and there was also a problem with very low staff morale because of the DIDS (Defence Integrated Distribution System) transition, so there was reasonably high absenteeism during that period.⁹

The ANAO found that Defence had no single training authority responsible for managing the scope of the training required to educate SDSS operators. There were different delivery methods for each ADF service, and no formal certification of SDSS operator competency. The training was not easily planned or implemented for all end users. ¹⁰

The contract to Dimension Data for training had a value ‘not to exceed $7 million’. The final costs associated with discharging the contract escalated to $13.35 million. The Committee asked Defence why the costs had escalated by over $6 million.

Defence responded that delays in roll-out of the SDSS upgrade (due to operational activities in Iraq ) had led to a need for re-training for some officers. The ANAO report found that Defence’s reliance on external training contractors meant that when the roll-out delays occurred, Defence was exposed to large contractor ‘maintenance costs’.¹¹ In other words, Defence was paying training contractors when the work could not be carried out, due to roll-out delays.

The ANAO recommended that Defence establish a centralised Defence Training Authority to take responsibility for the management and delivery of all required SDSS training.¹²

At a public hearing, the Committee questioned Defence about its SDSS training. Defence advised that the Director General of Materiel Information Systems is now responsible for the delivery of training to SDSS users. A nominated contact within each Group is responsible to the Manager Joint Training for identifying annual training liability and requirements. New developments include an online training tool. SDSS training is also part of the wider Defence remediation plans under implementation in response to the ANAO’s adverse finding on the 2003-04 financial statements (see chapter 9 for further information).¹³

Get Well Program

In 2004 Defence put in place a remediation program for the SDSS project entitled SDSS ‘Get Well’. In June 2005 Defence advised the Committee that the Get Well program included the following enhancements to the SDSS v. 4 project:

• Systems performance
• significant mainframe performance improvement (up to 10 times quicker);
• improvement at specific sites – network upgrades which were dropped from the original SDSS upgrade;
• Financial capability
• new functionality to provide improved financial information;
• major enhancements to three key reports;
• five new exception reports;
• seven new management reports;
• Data quality
• coordination of resolution of Supply Customer Account ownership and loading of details onto SDSS;
• data cleanup in several key areas – 100,000 records archived;
• data ownership charters;
• Business Process Compliance
• ANAO provided 113 review reports covering 16 operational segments;
• 114 recommendation areas, resulting in a total of 132 individual recommendations;
• responsible officers assigned across Defence;
• summary document linking recommendations to ANAO findings;
• Software defects
• redesign of Problem Reporting and Support Centre Procedures;
• management Dashboard reporting;
• new supply and equipment reports;
• Change and communication
• development of intranet site and regular newsletters;
• specific end-user communications; and
• visits to key sites.

One of the first elements of the Get Well program was to make network improvements to allow the program to run quicker in the field. These network upgrades had been dropped from the original SDSS upgrade in order to limit the budget blowout.

Defence advised the ANAO that the Get Well program was established with a Program Governance Board. However, the ANAO audit noted that the Governance Board did not include any representatives of end users, a problem that had occurred in the SDSS v. 4 upgrade project.¹⁴ The ANAO recommended that Defence manage the Get Well Program within the framework of the Defence Information Environment, including wide end user involvement (with Joint Logistics Command representation) at the Governance Board level.

The Committee questioned Defence about whether it had changed the Get Well governance board to include end user representation. Defence replied that the project board, and the user group which sits below the board, now represented the groups that are day-to-day users of the system. Each Defence Base also now has a local site administrator, to whom users can relay their concerns in the first instance.¹⁵

Defence advised that the total expenditure on SMS/KPMG contractors for the Get Well program was $1,179,538. This expenditure covered the Program Management Office, including change and communications.¹⁶

Defence spends $20 million per year sustaining SDSS, including 68 full-time personnel who undertake training, run help desks, rewrite codes, increase functionality, and put new tools on the system.¹⁷

Tenix Toll Defence

In 2003 Tenix Toll Defence won a major 10-year contract for delivery of Defence’s national warehousing and distribution services. The contract is worth up to $920 million over 10 years.¹⁸ The contract was estimated to save the Government up to $40 million, and allow for re-assignment of around 500 Defence personnel onto other activities.

In June 2005 the Committee undertook an inspection at the Defence National Storage and Distribution Centre (DNSDC) at Moorebank, Sydney. Tenix Toll Defence took over operations at the DNSDC (under Defence direction) in 2004. The Committee was keen to see the SDSS system in operation, and talk to personnel on the ground who were using the system.

The Committee was also interested to hear from Tenix Toll about the changeover in providing the warehouse and distribution services at DNSDC, and asked a number of questions on notice. Tenix Toll Defence advised that it employs 413 permanent staff and 157 casual staff in warehouse management in support of Defence activities (at June 2005). Within Defence’s Joint Logistics Command, around 900 people have direct or indirect involvement in a number of contracts, including the Tenix Toll contract.¹⁹

The Committee asked whether the original tender documents and contract for Defence warehouse management accurately reflected the asset and warehouse system that Tenix Toll took over. Tenix Toll responded that it did, with the exception of the introduction of handheld data processors, which will be provided by the Government by February 2006 (see paragraph 8.52 for further detail). Tenix Toll provided additional, and unforseen, support to Defence’s efforts to improve its stocktaking methods as part of the remediation plans. Additional costs incurred by Tenix Toll in supporting these activities are claimable under the contract.²⁰

Handheld scanners

At DNSDC Moorebank, the Committee observed the SDSS program in operation in one of the 12 DNSDC warehouses. The storeman was using a handheld data processor, known as a Radio Frequency Portable Data Entry Terminal, to scan a barcode on items. The handheld processor communicated with the SDSS system to verify the item in front of the storeman, and told him how many items [bolts, in this case] should be in the container. He was then to enter into the system any bolts that he removed from the box.

The system on demonstration to the Committee at Moorebank was operating at a very slow speed. In one instance, the barcode was not able to be read because it had been entered incorrectly.

At the hearing later that day, Defence told the Committee that it intends to roll out this technology to all warehouses by the end of 2005. Defence acknowledged that the system witnessed by the Committee was slow in operation, however argued that it was a pilot system being tested before a full rollout to other warehouses.²¹

The Committee was pleased to be able to inspect the DNSDC facility at Moorebank, Sydney. The tour brought home to the Committee the complexities of the logistics management undertaken by Defence in partnership with Tenix Toll Defence. Several Committee members had the opportunity to talk to storeman on the ground and witness the use of the SDSS system. The Committee is concerned that the on-the-ground experience of warehouse personnel may not be taken into consideration by Defence when planning SDSS upgrades and rollouts of new technologies such as the handheld scanner.

Previous Committee review

The Joint Committee of Public Accounts reviewed the predecessor to the SDSS, the Supply Systems Redevelopment Project, in 1992 (Report 317: A champagne appetite but only a beer income). The Committee found that the achievements of the project were not encouraging, and that the project had continuously fallen behind schedule due to ‘poor administration and management of the Project at a global level’.²² One of the Committee’s recommendations was:

That the operation and management of the Supply Systems Redevelopment Branch be reviewed to ensure that appropriate project controls and procedures are now in place, especially in the area of quality assurance.²³

Other Defence projects

The Committee notes that the Audit Office has recently tabled its Audit Report No. 8, 2005-06, Management of the Personnel Management Key Solution (PMKeyS) Implementation Project. This new report highlights many of the same problems experienced with the SDSS upgrade project – the project did not receive the proper approvals, it ran over time and over budget, and the system is still not working effectively.

The Committee has yet to formally review Audit Report no. 8, 2005-06, but is most concerned to note that the problems highlighted in the ANAO’s report on the SDSS upgrade are not limited to that project. These two reports indicate systemic problems in project management at Defence, particularly for IT systems. The Committee intends to further examine these problems in a wider-ranging inquiry, commencing in 2006.

ANAO Audit Report no. 5, 2004-05: Management of the Standard Defence Supply System Upgrade (Department of Defence); Commonwealth of Australia; August 2004; p. 23. Back

ANAO Audit Report no. 5, 2004-05, p. 78. Back

Defence Materiel Organisation (DMO), Transcript of Evidence, 28 April 2005, p. 55. Back

DMO, Transcript of Evidence, 28 April 2005, p. 45. Back

ANAO Audit Report no. 5, 2004-05, Management of the Standard Defence Supply System upgrade – Department of Defence, Commonwealth of Australia, August 2004, p. 68. Back

ANAO Audit Report no. 5, 2004-05, p. 17. Back

Department of Defence (Defence), Senate Foreign Affairs, Defence and Trade Legislation Committee, Additional Estimates, Transcript of Evidence , 18 February 2005, p. 9. Back

ANAO Audit Report no. 5, 2004-05, p. 79. Back

ANAO Audit Report no. 5, 2004-05, p. 80. Back

ANAO Audit Report no. 5, 2004-05, p. 26. Back

Defence, submission no. 11, p. 3. Back

ANAO Audit Report no. 5, 2004-05, p. 90. Back

Defence, Transcript of Evidence, 28 April 2005, p. 44. Back

Defence, submission no. 11.1, p.1, and Transcript of Evidence, 27 June 2005, p. 43. Back

Senator the Hon. Robert Hill, Defence Integrated Distribution System Contract Signed Today, Press Release 18 December 2003, at: http://www.minister.defence.gov.au/Hilltpl.cfm?CurrentId=3399, accessed September 2005. Back

Defence, Submission 11.2, p. 1. Back

Defence, Submission no. 11.2, p. 2. Back

Defence, Transcript of Evidence, 27 June 2005, p. 20. Back

Joint Committee of Public Accounts, Report 317: A champagne appetite but only a beer income – Defence’s Supply Systems Redevelopment Project; Parliament of the Commonwealth of Australia; June 1992, p. xvi. Back