Report 407

Introduction

Background

Edge was a joint information technology project between the Australian Government Department of Family and Community Services (FaCS) and Centrelink to develop an expert system for the Family Assistance Office (FAO). Edge was a processing application, for the administration of claims and payments for people applying for entitlement to family-related payments. SoftLaw, a private sector company, was the successful tenderer for the project, supplying software and expertise for the application'fs development.¹

Development of Edge (then called the Life Events Expert System, LEES) commenced in March 2000. In June 2002, pilots of the system were assessed as successful and progressive roll out of the system started in July 2002.

However, in August 2003, FaCS and Centrelink jointly commissioned a review of the project to assess whether there was a viable business case for further development of the Edge system. In November 2003, the report of that review recommended discontinuing development of the project. FaCS and Centrelink accepted the recommendation.

The four reasons given in the 2003 review were:

• Edge in its planned form was no longer properly aligned with the business needs of the Families program
• the operation of Edge in parallel with ISIS was unsustainable
• changes to the Families program meant Edge could have only limited effect on a key driver?improvement in accuracy; and
• the level of anticipated benefits were unlikely to be realised, leading to a negative return on investment.²

The ANAO found that although the Edge project was terminated in November 2003, documents indicate that it would have been completed according to contract by December 2003. However, there would have been no guarantee that Edge would have been able to replace the equivalent part of ISIS at that time, or into the foreseeable future.

In addition to the reasons stated in the 2003 review report, the ANAO listed a number of other issues which adversely impacted on the project, including:

• the lack of an MOU between FaCS and Centrelink; including issues such as funding, savings, relative responsibilities, and what constituted work on Edge;
• Centrelink became reluctant to continue funding a project that had no clear finalisation. Savings were to have been shared between the two agencies, providing little incentive for Centrelink to conclude the project and then have to give up savings to FaCS, unless they could clearly obtain those savings. Assessment of the actual savings had been deferred until the Edge system was implemented;
• Centrelink had difficulties in integrating the expert system into its IT environment, due both to the constraints of that environment, and the complexities of the solution developed for integrating Edge with the mainframe;
• frequent legislative and other changes³ after the commencement of the project necessitated modifications to Edge. Ultimately, frequent adjustments of the Edge rulebase were required, which would have required redesign. Based on these issues, the 2003 Business Case Review team concluded that the system no longer met the requirements of the business;
• comparison with ISIS in 2001 indicated significant improvements were potentially achievable in the accuracy, consistency, and completeness of assessment decisions, by using Edge. The annual reconciliation process⁴ reduced these potential benefits of Edge, and a further comparison with ISIS in 2003 indicated little, if any, difference between the two systems; and
• early predictions of the functionality of Edge for processing customer claims were optimistic. Edge gained a poor reputation with Customer Service Officers (CSOs) who were mandated to attempt to process claims with Edge before using ISIS, which they felt was more reliable.⁵

The development, from an initial trial in 1997 to the end of 2003, had taken some six years and come at significant cost?for example contractual payments to SoftLaw of around $30 million, and involving up to 150 staff from three organisations; FaCS, Centrelink, and SoftLaw. Information provided by FaCS and Centrelink, confirmed to the extent possible by the ANAO, estimated the total expenditure on the Edge project to be around $64.4 million, for the almost four years from contract signing to termination.

FaCS and Centrelink

During the period of the Edge project, the Family and Community Services Portfolio was responsible for providing advice on a broad range of social policy issues affecting Australian society and the living standards of Australian families, communities and individuals. FaCS was the principal policy formulating and advising body in the portfolio. Centrelink was the service delivery agency in the portfolio, delivering a range of Commonwealth services, such as pensions, benefits and allowances to the Australian community.

Since the termination of Edge, in October 2004, as part of machinery of government changes affecting several departments and agencies, FaCS'f responsibilities have changed.⁶ In addition, Centrelink now resides within the Human Services Portfolio.

The fieldwork for this audit was completed before the agencies were restructured. The relationship between FaCS and Centrelink was a significant issue during the project, and is discussed in some detail in the ANAO report. In particular the agencies were unable to resolve disagreements in funding Edge. While the two agencies will still have a business relationship under the new administrative arrangements, the nature of the ongoing relationship is not yet clear. The ANAO has, therefore, made no specific recommendations on that relationship. However, the ANAO suggested that both agencies consider their processes for resolving disputes with other agencies.

The environment

Payments made by Centrelink are subject to increasingly complex, and frequently changing, rules. These changes follow from revised government policies and from new and revised legislation. At the start of the Edge project it was estimated there were 8000 such rules for Family Assistance payments. It was, therefore, appropriate that Centrelink, and its partner agencies, explore and research more sophisticated solutions to delivering the required services, with a view to reducing the administrative costs of delivery, while increasing the accuracy of advice and payments to recipients of the services. "Expert systems"⁷ promised both cost reductions, and greater accuracy of advice and payments.

Therefore, the ANAO considered that the question for the Edge project was not whether it should have been attempted, but whether management of the project, and decisions made during the project, were in accordance with better practice.

There was evidence of tension between some personnel of both FaCS and Centrelink during the project. The ANAO observed that it would have been surprising if there were not, as the two agencies had somewhat differing needs from the project. Generally, work on the project progressed despite the tensions. The tensions were greater at more senior levels, where funding could not be agreed, and effective high-level governance of the project was not evident.

The audit

The objectives of the audit were to determine whether FaCS and Centrelink had:

• a valid Business Case for the Edge project, as revised from time to time, including estimated costs, actual costs, and expected benefits;
• effective governance of the project, including reviews at critical points in the project and subsequent decisions to continue or, in the final analysis, to discontinue;
• an appropriate contract with SoftLaw, which was adequately managed;
• delivered appropriate advice on progress, project viability, and acceptable solutions to technical issues to Executive of FaCS and Centrelink during the project; and
• valid reasons for discontinuing the project.

The ANAO began this audit in March 2004, four months after the Edge project was terminated, following the Auditor-General’s agreement to a suggestion by the Joint Committee of Public Accounts and Audit that the project was a suitable subject for audit.

The audit report was tabled on 14 April 2005.

Overall audit conclusion

In short, the ANAO described the Edge project as over time, over budget, and terminated before completion. Direct financial savings from the project were not realised and the project was unsuccessful when assessed against its aims. There were deficiencies in the project, particularly in the governance of the project, from which lessons for the future can be learnt. While FaCS and Centrelink advised the ANAO that they did gain some benefits from the project, nevertheless it was appropriate for the agencies to terminate the project.

ANAO recommendations

The ANAO made the following recommendations:

Table 4.1 ANAO recommendations, Audit report no. 40, 2004-05

The Committee’s review

The Committee held a public hearing to examine this audit report on Friday 19 August 2005. Witnesses representing Centrelink, the Department of Family and Community Services, the Department of Human Services and Softlaw Corporation appeared at the hearing, as well as representatives from the Australian National Audit Office.

The Committee took evidence on the following issues:

• the joint agency nature of the project;
• connectivity of Edge with existing systems;
• aspects of the project contracting, including:

  Scoping; and

  Project management/responsibility: and

• the effect of changes to legislation.

JCPAA and Edge

The JCPAA had previously heard of the Edge project in hearings on the ANAO’s audits into the assessment of new claims for the age pension.⁸

The then CEO of Centrelink, Ms Suzanne Vardon, made a number of references to the new software system:

We have a partnership with Softlaw which is helping us develop a smart internet ready computer program called Edge which incorporates the thousands of rules within the family tax benefit system. Edge means that our staff, and eventually our customers, will only need to answer the personalised questions on the screen and they will get an accurate assessment of their entitlement.⁹

…the solution to some of our issues lies in enhancements to our computer system. [Such as] Edge, which is an expert decision support system, whereby we have taken the 8,000 rules in families and we have put them in the computer rather than in the heads of our staff … The standard of decision that comes out of those is very high, … we are working hard and using our technology to improve correctness every day.¹⁰

… our staff will be forced to go through the steps. There can be no shortcuts. When it comes to those errors that relate to process, we believe there will be a great improvement. When it comes to those errors that might be in determination, we know already that there is an improvement as well. So we hold a lot of hope for the new decision support systems.¹¹

… the failure to enter data is a human condition; so, to the best that we can, we want to get the data entry done at the front, when a person comes in, with a very experienced officer. [The Edge system] … spits out what we call an ‘offer’ in writing … that you sign it as a correct document. … The paper record out of the machine is the form … and people have a chance to inspect the information that has been put in and confirm it. That is a very significant improvement in data accuracy.¹²

The Committee would like to comment at this point to say that it is disappointing that a system which appeared to hold such promise was developed but never fully implemented.

At the same time, Centrelink still comes under criticism for errors in its data.¹³ The Committee is concerned that in the wake of the development of the Edge system, Centrelink should maintain the momentum to improve the systems in place overall to reduce the rate of errors in its data.

Joint agency project

The Committee explored the governance of the Edge project as a joint venture between Centrelink and FaCS.

The ANAO found no major problems with the governance that existed in each of the agencies individually, concluding that “the FaCS Executive Board was appropriately advised of the progress of the project¹⁴” and “the Centrelink governance committees were appropriately informed of the progress of the Edge project”¹⁵.

The ANAO did note that despite being the largest IT development undertaken by FaCS since its creation in 1997, the IT Committee was not involved on an ongoing basis with the Edge project.

The ANAO summarised the relationship between both agencies in Chapter 4 of the audit report:

Following the formation of Centrelink (under the Commonwealth Service Delivery Agency Act 1997) on 1 July 1997, FaCS (then the Department of Social Security) and Centrelink developed a Business Partnership Agreement (BPA) as a framework for managing the relationship between the two agencies. The BPA was revised in 1999, 2000, 2001, and in 2004 as part of the FaCS-Centrelink Alliance 2004 relationship project.

A Joint Project Agreement (JPA) for the Edge project was included in the 1999 BPA, and mentioned in later versions of the BPA. The 2001–04 BPA, which concluded on 30 June 2004, stated ‘details of funding allocated for and the financial principles governing the Edge (expert system) project are specified in the memorandum of understanding between FaCS and Centrelink’.

The two organisations developed a new version of the JPA during 1999 and 2000. … the agencies were unable to provide a copy signed by both parties. However, the ANAO was informed by both agencies that they operated according to the agreement.

Work continued on a further JPA. In November 2001, an email between FaCS and Centrelink stated that the document was still undergoing revisions in mid 2001, and that FaCS was refusing to sign until the MOU was agreed. The ANAO obtained further versions of both the draft MOU and the JPA dated as late as 2003. However, the ANAO was unable to locate any further agreed and signed version of the JPA.¹⁶

The audit report stated that the memorandum of understanding (MOU) was never agreed between the two agencies and was still outstanding at the termination of the project, and that consequently, a number of issues defining the relationship between FaCS and Centrelink were never agreed.

Centrelink stated that “the fact that the MOU was not signed at that time—as far as any of us can see, in view of the progress—was seen more as an administrative sort of oversight than any particular failure”.¹⁷

At the time, FaCS accounted for 90 per cent of Centrelink’s business, and there was a huge amount of interplay between the two organisations. Centrelink argued that for the Edge project they were trying to act as one organisation, albeit with different accountability and reporting lines.

FaCS expenditure on the project was made up of around $12.3 million principally associated with supporting the FaCS project team, and approximately $4 million which was paid to Centrelink for shared expenses.¹⁸ The FaCS representative was unable to explain why any money was paid to Centrelink without a signed MOU regarding the contract.

The absence of a signed MOU meant that the financial responsibilities of the agencies were never clearly articulated. When additional funding was needed for the project, ANAO reported, there were disagreements between FaCS and Centrelink as to the contribution of each agency and the Centrelink Edge team had to continually approach the BIC for additional funding:

FaCS wished to fund its contribution from expected savings. Centrelink needed cash from FaCS to complete the development. There was also disagreement between the agencies as to whether some developments associated with Edge should be considered part of the Edge project¹⁹.

The joint FaCS–Centrelink Steering Committee for the project was charged with managing the relationship between FaCS and Centrelink, including achieving agreement on the MOU. This was the steering committee that the ANAO reported “met regularly, at intervals of one to two months, in 1998 and 1999, once in 2000, and four times in 2001, but never as a full committee after November 2001, although the Edge project continued until November 2003²⁰ ”. The ANAO was advised that senior executives of the two agencies preferred to work ‘one-on-one’ to obtain agreement on various issues and such meetings were frequently not documented.²¹

The ANAO noted that the Edge project was of sufficient importance to be included in the Business Partnership Agreement between the two agencies, and therefore considered it “would have been appropriate for the Boards of both agencies to have been informed of the lack of progress on the MOU, and for both Boards to have stated their views on the situation.²² ”

The Committee does not agree that any joint agency project, let alone one of such a large scale, can be conducted without the agencies involved first developing and signing a memorandum of understanding.

Connectivity

The connectivity/mapping between Edge and M204 (the Database Management System Centrelink used on its central computer) was identified by the ANAO as an issue that “was not treated appropriately in the risk management plan²³”right from the outset.

When questioned as to why there wasn’t further examination of connectivity within the system prior to embarking upon the program, the Centrelink response included:

to some extent it would have been difficult for the people who were looking at what they were looking at back then to see the extent of the problems they were going to encounter. You can do it in hindsight. … They were looking at an entirely new piece of software and they were going to try to integrate that into a system that already had 10 years development behind it. … If you read the papers of the time … it was identified that connectivity would be a risk. You will find in the report that the risk level seemed to rise as the audit went on. But did they see that it was a risk that was eventually going to have such a profound effect on the outcome of the project? No, I do not think they did see it. ²⁴

The ANAO expressed the view that the difficulties Centrelink had in successfully integrating the expert system into its current IT environment was one of the reasons the Edge project was unsuccessful.

This was due both to the constraints of that environment, and the complexities of the solution developed for integrating Edge with the mainframe²⁵ . The solution was being developed concurrently with Edge. Connectivity with the mainframe was a major source of delays to the project. Edge had more functionality (that is: ability to process customer claims) than it was able to process through to Centrelink’s mainframe. The communications and data matching needs of interfacing Edge with the mainframe had been identified as a high risk in the original Edge Business Case. However, the mitigation strategies were inappropriate and proved to be ineffective.²⁶

Information provided to the ANAO by Centrelink outlined two aspects to the connectivity problems:

the mapping from detailed EDGE data items to more granular Income Security Integrated System (ISIS)²⁷ data items²⁸; and the need to capture extra items needed for ISIS processing or management information purposes which were not explicitly mentioned in the legislation; and

at the technical level, EDGE needed to sponsor the creation of a large number of transactions to handle all of the Families program data and hence ‘scale up’ the usage of this expert technology from its developing use other applications.

As Softlaw described:

the connectivity proved very difficult ... particularly in some of the complex areas like maintenance payments et cetera, where the mainframe had evolved over a long period of time, it was quite difficult to marry up the rule base that was in Edge with the rules that were being applied in model 204. … It was always going to take time to make sure that the two systems operated correctly together.²⁹

The parties had agreed that the expertise around the mainframe to manage the connectivity issues lay with Centrelink rather than the software contractor. Therefore the responsibility for the mainframe connectivity ultimately sat with the project steering committee.

As the ANAO reported, this was the joint FaCS–Centrelink Steering Committee which “did not accord with the project management rules of the two agencies [or] the project governance structure as stated in the contract with SoftLaw.³⁰”

The Committee notes that despite the risks associated with connectivity being identified very early on, and the responsibility for mitigating these risks residing with Centrelink, the issue was still identified by the ANAO as contributing to the unsuccessful project.

The Committee is concerned that the project contract did not contain provisions for Centrelink to withhold payments based on the inability to mitigate identified risks.

Accordingly, the Committee makes the following recommendation:

Contracting

Scoping

According to Centrelink, “the Edge contract was originally signed by the Deputy Chief Executive Officer who held the necessary financial and contract signing delegations assigned by the Chairman of the Centrelink Board”.³¹

When questioned as to how Centrelink spent $30 million on software without a stronger initial milestone of a scoping study, and how the agency then spent a total resource of $60.4 million on the project, Centrelink’s response was:

At the time it was always recognised that this was going to be a big and complex project. You do not engage in this sort of thing unless it is going to be a multimillion dollar project. I think the ANAO report does look at this. It says that at the start everything looked okay. The ANAO is not critical, really, of the start of this project.³²

The ANAO was advised by FaCS and Centrelink that since the Edge procurement process, each agency has implemented readily-available, improved frameworks to assist staff with procurement activities.³³

This was confirmed to the Committee by Centrelink, however the lack of Ministerial oversight remains:

In the present day, the procurement process for a complex procurement is subject to multiple control points in accordance with Centrelink Chief Executive Instructions, procurement policy and financial delegations. The process can include obtaining external legal and independent probity reviews. Ministerial approval is not sought or required for the Centrelink delegate to sign a contract.³⁴

The Committee remains concerned that without appropriate oversight, there is the possibility of an agency becoming caught up in the enthusiasm of a project and being blind to some potentially major problems. An outside appraisal of the governance and technical aspects of the situation could provide a more objective viewpoint and enable early intervention to mitigate issues identified.

Accordingly, the Committee makes the following recommendation:

Project management/responsibility

Ultimate responsibility for contract and project management of the Edge project lay with Centrelink as representative of the Commonwealth in the contract³⁵. Delegations within Centrelink were outlined to the Committee:

The authority to sign off a progress payment would have been delegated from the chair of the board of Centrelink through the chief executive officer to appropriate officers within Centrelink, of which the project manager would have been one³⁶.

Centrelink’s National Manager responsible for the Edge project was authorised to sign off on deliverables. This position was held by a very senior officer in Centrelink whose delegation “would have effectively come from the chair of the Centrelink board through the CEO to officers, including this project manager³⁷”. The person who held this position through most of the contract dealings between Centrelink and Softlaw is no longer with Centrelink’s IT division.

The Centrelink CEO said of the Edge project, “I do not think it was a failure of the project team. Nor … a failure of the project team manager … I think it is a failure of governance³⁸”.

The Committee is concerned by the apparent trend in Public Service project management in which no one is willing to take responsibility for mistakes occurring within those projects. It is the Committee’s opinion that greater care needs to be taken in all aspects of project management, regardless of how promising the outcome of the project looks.

According to Centrelink, “the then Chief Executive Officer of Centrelink and the then Secretary of the Department of Family and Community Services accepted the recommendations of the [Edge Business Case] review and so took responsibility for the cessation of the Edge project³⁹”.

It is recognised that both agencies have accepted there were problems with the management of the project, however the Committee finds it unacceptable that no one is wiling to claim responsibility for the difficulties that led to the decision to terminate the project, particularly when there are substantial amounts of money involved.

The Committee is concerned that governance lessons learnt from the Edge project be incorporated in future projects, particularly the IT Refresh⁴⁰ project described as “the biggest project that is happening in Centrelink at the moment”.⁴¹

Accordingly, the Committee makes the following recommendation:

The Committee considers that this issue is important enough to warrant further investigation of the progress of improvements in contracting procedures in Centrelink.

Accordingly, the Committee makes the following recommendation:

Legislative changes

The Committee notes that the Edge project was extremely complex and required the consideration of legislative, policy, and business rules for the Family Assistance Office which, at the start of the project, were estimated to number over 8000.⁴²

Appendix 1 to the ANAO report lists the Family Assistance Legislation changes which impacted on the Edge project. The legislative and other changes were frequent and required changes to Edge. For example, the introduction of the More Choice for Families (MCFF) initiative in 2002 required changes to ISIS, and Edge necessarily had to be changed to match it. This “resulted in a requirement for the ability to continuously adjust the Edge rulebase. This was not envisaged in the original design and would require some redesign”⁴³.

But as Softlaw explained, “one of the whole premises of our product was to try to absorb the complexity of having a lot of rules and having constant changes in the rules. The main difficulty we had was not in building the rule base. That was done quite quickly and it was maintained quite successfully over the project⁴⁴ ”. Softlaw explained that the problem was then taking that rule base and marrying it up with the mainframe, which had been developed over 15 years of adding to those rules.

Amongst the key findings from the 2003 review was a statement that “the operation of Edge in parallel with ISIS … places an unsustainable workload of dual development and maintenance on the available information and technology (I&T) resources⁴⁵”. This situation was exacerbated by the degree of legislative change that had to be accommodated.

The issue of struggling to keep pace with changing legislation is relevant at a larger scale than just those areas covered by Centrelink.

ANAO Audit Report No 40 2004–05 The Edge Project. p.13. Back

ANAO Audit Report No 40 2004–05 The Edge Project. p.22 Back

ANAO Audit Report No 40 2004–05 The Edge Project. p.18 Back

ANAO Audit Report No 40 2004–05 The Edge Project. p.87 Back

ANAO Audit Report No 40 2004–05 The Edge Project. p.102 Back