|
|||
|
|||
|
|
|
|
|
1. |
To provide adequate assurance that the NHMRC grant funds are being managed appropriately by Administering Institutions, the ANAO recommends that the NHMRC: · complete the development and implementation of a risk-based assessment for approval of Administering Institutions, and systematically maintain complete records of those approvals; and · implement arrangements to improve monitoring of the Administering Institutions’ compliance with the requirements of the Deed of Agreement, including conducting audit activity where a high risk is indicated or persistent non-compliance evident.
NHMRC response: Agreed |
|
2. |
To improve the transparency and probity of its peer review process, the ANAO recommends that the NHMRC: · monitor the incidence and reasons underpinning the allocation of Grant Review Panel (GRP) members’ application to their own GRP for assessment; and · enhance the documentation of key actions and recommendations of the GRPs, in order to provide a defensible record of the selection proceedings and strengthen feedback to applicants.
NHMRC response: Agreed |
|
3. |
In order to improve the identification and management of conflict of interest, the ANAO recommends that the NHMRC: · amend its conflict of interest guidelines to strengthen guidance on acceptable and unacceptable conflicts of interest; and · develop a risk-based strategy for more systematic monitoring and review of conflict of interest compliance, including a register of private interests.
NHMRC response: Agreed |
|
4. |
To improve accountability of grant funds, the ANAO recommends that the NHMRC implement risk-based arrangements including enhanced systems to: · manage overdue annual financial reports and final acquittal statements; · recover debt due to overpayments and unspent funds; and · achieve timely receipt, review and analysis of grants’ progress and final reports.
NHMRC response: Agreed |
|
5. |
To strengthen the NHMRC’s management of grants, the ANAO recommends that the NHMRC include as part of the new Research Grant Management System (RGMS): · appropriate compliance controls to identify breaches of legislative, key policy and eligibility requirements for all grant applications; · a suitable interface between RGMS and the NHMRC’s financial system to allow accurate information exchange and regular reconciliation of the systems; · a regular program of data verification and cleansing to prevent corruption of future NHMRC data; and · a structured training program and complete documentation for all key processes.
NHMRC response: Agreed |
2.20 The Committee held a public hearing on Wednesday 12 May 2010, with the following witnesses:
- Australian National Audit Office (ANAO); and
- National Health and Medical Research Council (NHMRC).
2.21 The Committee took evidence on the following issues:
- research areas;
- commercialisation;
- conflict of interest;
- assessment and selection of grants;
- fairness;
- documentation and procedures; and
- peer review process;
- Administering Institutions;
- hospitals and smaller institutions; and
- post-award grant management.
2.22 The Committee understands that the National Health and Medical Research Council Act 1992 requires the NHMRC to determine its targets for research grants in accordance with major national health issues identified in consultation with the Minister for Health and Ageing. According to the NHMRC Strategic Plan the research areas remain flexible to accommodate the changing needs of the Australian community over the period of the Plan:
NHMRC will help Australia deal successfully with health issues as they arise. These include emerging issues for the health system or individuals, or new health and medical research developments. ... NHMRC, therefore, needs to be flexible to meet unforeseen challenges that may arise during the period covered by this Strategic Plan.[11]
2.23 The Committee sought clarification regarding how the NHMRC narrows its research priority areas for grant funding and whether or not the agency places any restrictions on research areas. The NHMRC explained to the Committee that there are no restrictions provided an applicant satisfies the eligibility requirements.[12] The CEO added that NHMRC has developed a series of funding vehicles to ensure a balance of funding between a variety of types of research including laboratory research, clinical research and public health research.[13]
2.24 The Committee asked specifically whether or not complementary health and alternative health were included in the targeted research areas. The NHMRC confirmed that both areas are included in the current Strategic Plan.[14] The ANAO report noted that in 2009, NHMRC received 35 applications for research in these areas and that 12 (34.3 per cent) were funded at a cost of $4.5 million.[15]
2.25 The Committee expressed concern that research aimed at discrediting complementary medicine may be funded and asked for examples of the types of successful projects. The NHMRC maintained that a grant application that showed a ‘clearly prejudiced expectation’ would not be considered scientifically sound and would not therefore be funded.[16] Professor Anderson described the focus of a number of successful projects in this research area:
There have been quite a lot looking at Chinese traditional medicine, about the effectiveness of that. There have been some looking at Indigenous Aboriginal traditional medicines. There have been grants looking at the chemistry of extracting of herbs to increase purity and that sort of stuff.[17]
2.26 The Committee was particularly interested in the commercialisation of research funded through the NHMRC grant program and asked the agency if it had data on the issue. The NHMRC informed the Committee that no rigorous study had been undertaken into the overall monetary benefits of the program but that limited research has been done by Access Economics and the NHMRC itself. The CEO told the Committee that these studies indicated considerable benefits are flowing to the Australian economy from the investment in research:
(The Access Economics study found) that the benefits to the Australian economy of the cochlear ear implant and CSL, including Gardasil, are about equivalent to the entire government’s investment in health and medical research over that period of time. We also did a study a couple of years ago – and we are repeating it – where we looked at 1,208 grants and asked the grantees what the benefits were. Quite apart from the rapid growth in patents and intellectual property protection they have done, they also reported on their leverage of funds into Australia. I think for every government dollar these people levered about 30c one way or another on top of that.[18]
2.27 The ANAO acknowledged that conflicts of interest were inevitable in the peer review process used by NHMRC to assess grant applications, given the ‘small pool of researchers available to assess applications in specialist areas’ in Australia.[19] However, the ANAO found that the NHMRC guidelines needed to be strengthened to provide ‘greater clarity on the types of conflict of interest and situations in which these are relevant’ and that conflict of interest declarations needed to be monitored for compliance.[20]
2.28 The Committee asked the ANAO how many potential reviewers declared a conflict of interest. The ANAO replied that, over a two year period, 1,200 researchers had declared a conflict of interest.[21]
2.29 The Committee asked the NHMRC to clarify how the conflict of interest process works. The NHMRC explained to the Committee that every reviewer is asked to declare on every grant application if they have a conflict of interest.[22] If the reviewer declares a conflict of interest they will not see that particular grant application and will be excluded from all consideration of that application:
The first step is that you do not even get to see the grant in the first place on the panel. Then, when the panel meets, you are outside the room. When the panel is ranking the grants, you are outside the room and the ranking is blind to you.[23]
2.30 Nevertheless, the NHMRC admitted that a grant can go to someone who is on a selection panel. Asked by the Committee to explain how this occurs, the NHMRC reiterated that the reviewer would not be in the room when the application was being assessed but would be present when the cluster of applications was being considered:
... they are not there when they are being ranked and not there when they are being reported.[24]
2.31 The NHMRC added that the research community expects an open and transparent system and that the panels are independently monitored to ensure this:
We have our own staff as well as observers. So we also recruit lay people to look at these panels and report directly to us on how the panel has been operating. This is a very important perception that the research community keep us very much up to the mark on. They want a level playing ground, so they are very diligent with us ...[25]
2.32 The Committee raised the issue of the fairness of the assessment of grant applications and asked the NHMRC if a reviewer’s bias could affect the success of a grant application particularly with regard to fashionable or popular research topics. The NHMRC assured the Committee that the assessment process mitigated the risk of personal bias or prejudice influencing a decision:
These panels have 10 to 12 people, and the chair’s role is to make sure that they are all put to the test in what they are saying. Remember that not only do the panel have another 10 people looking at what they are doing but there are one or two written external reviews on every grant, so there is an independent review by experts. It is not impossible but hard to bring personal prejudice to the table because in this scheme we are funding one in five applications and they are scoring as ‘outstanding’ and ‘excellent’ – the very top of the very good. For every one we fund, our panel has ranked three more as worth funding but unable to be funded. The pressure is so high that something that is just a passionate view is very unlikely to get up.[26]
2.33 The ANAO found that the documentation for grant approval often lacked a clear trail and that it was difficult to establish the reason why scores had been altered or budgets reduced.[27] The ANAO told the Committee that records were incomplete and that online databases and hard copy records could not be reconciled.[28] While the ANAO emphasised that it was not questioning the legitimacy of grant approval decisions, it identified the issue as a serious impediment to transparency and accountability.[29]
2.34 The Committee asked the NHMRC what steps had been taken to improve documentation and procedures. The NHMRC told the Committee that it is developing and implementing a new Research Grant Management System (RGMS) that will enable online application, appraisal and management of grants.[30] The system is expected to streamline and standardise the process, eliminating many of the discrepancies identified by the ANAO audit.[31]
2.35 Other improvements identified by the NHMRC include the capacity to undertake the peer review process in-house which will allow greater control of documentation and the appointment of an independent chair for every grant review panel.[32]
2.36 Given the inherent nature of the peer review process, the Committee questioned whether or not the ANAO’s recommendations for greater accountability and transparency were too demanding. The CEO admitted he had initially had doubts about the practicality of the recommendations but has come to the conclusion that the process will be improved by meeting the requirements:
I think in the discussions during the review I probably did have some concerns that there was perhaps a misunderstanding about what would work and what does not. Peer review is eventually an opinion by somebody who is worthy to give that opinion, but at the end of the day I think that the recommendations are compatible with nevertheless having high-quality peer review without influencing that. Indeed ... it will improve that. For example, this year for our panels we are going to not only document it more but we are going to be making them nail their colours to the wall on each of our selection criteria, not just the overall one.[33]
2.37 The ANAO report was critical of the both the NHMRC’s certification process for Administering Institutions and the ongoing monitoring of the Institutions for compliance.[34] The ANAO noted that NHMRC’s inability to ‘provide assurance that Administering Institutions are sound administrative entities with suitable capability to administer grants’ presents a significant risk to Commonwealth funds.[35] The NHMRC had advised the ANAO that it was revising its certification policy and developing and implementing a new compliance framework for Administering Institutions.[36]
2.38 The Committee asked the NHMRC what steps had been taken to implement the new framework. The NHMRC informed the Committee that the agency has set up a separate, independent quality and regulation branch to consolidate the staff and processes involved in the certification procedure.[37] This branch also handles complaints.[38]
2.39 Additionally, the NHMRC has revised the Deed of Agreement, the legal contract between the agency and Administering Institutions setting out terms and conditions for the administration of grants.[39] As at May 2010 the new Agreement is out for consultation with the community and, after relevant feedback is incorporated, will be forwarded to the NHMRC research advisory committee and Council.[40]
2.40 Further, the NHMRC told the Committee that the overall policy framework for Administering Institutions has been updated and as at May 2010 is out for comment.[41] The CEO explained that there have been some delays outside the agency’s control in finalising the new policy documentation but it expects completion by the middle of 2010:
The first is that the Department of Innovation, Industry, Science and Research had just changed their rules about the support of the indirect costs of research, and that has impacted on the relationship between universities and medical research institutes. That has affected the way we need to approach our policy. As you are probably aware too, the government’s considerations around health reform, the role of research in that, and the comments about supporting the current costs of research and training also mean that we need to make sure that our policy does not stand in the way of the cooperation that we hope for. So there have been a couple of background things that have led us to be a little slower than we want.[42]
2.41 The Committee raised concerns regarding the impediments faced by hospitals and smaller institutions attempting to access NHMRC grant funding. Members of the Committee had received anecdotal evidence that smaller institutions faced significant on-costs in applying for and administering grants.
2.42 The NHMRC admitted that it is easier for larger institutions to absorb on-costs and suggested that ‘smaller institutions need to think about whether their overhead costs compared to their research activities are not disproportionate’.[43] However, the NHMRC is aware that their size allows smaller institutions to be more flexible and innovative in their research proposals.[44]
2.43 The NHMRC advised the Committee that it encourages collaborative research approaches to ensure that the capacity of smaller institutions and hospitals can be tapped.[45] The agency explained that Australia has an excellent reputation for collaborative research and that the majority of its grants are awarded to research teams rather than individuals:
That is the way of medical research these days. You often need, say, a cutting-edge geneticist, a behavioural scientist, an epidemiologist and then a clinical oncologist on a grant. I do think there is something in the Australian characteristic, if you like, that makes that collaboration fairly easy. We hear this from Australians – who think this is a good thing – who come back to Australia after years in, say, the United States, where the culture around this is very different.[46]
2.44 The Committee asked for clarification regarding the difficulties hospitals face in accessing research grants. While emphasising the importance of clinical research, the NHMRC explained that currently hospital research does not attract a distinct stream of funding and that research projects may not be a high priority for hospital administrators:
It does not mean that some hospitals have not been fantastic about that, but, if you are a hospital administrator – if you have ambulance bypass problems or patients stacked up in emergency – you can understand the priority. So I think it is important for the future that the support for clinical research is there beside the patients in the hospital and that the health services research is there in the health system and is supported in that sort of way.[47]
2.45 The ANAO found a number of deficiencies in the NHMRC’s administration of post-award grant management.[48] The ANAO noted that this has historically been a problematic area for the NHMRC but acknowledged that the agency is taking steps to rectify the issues.[49]
2.46 The Committee asked the NHMRC what changes it had made to achieve the improvements and what success it had had to date. The NHMRC told the Committee it had set up a separate section to handle post-award management and had instituted rigorous processes and procedures to address the issues raised by the ANAO:
We set up a specific section, developed some very comprehensive standard operating procedures and really focused on the acquittal process.[50]
2.47 The Committee was pleased to hear that outstanding acquittals had been reduced from 1275 in 2008 to approximately 100 as at May 2010.[51]
2.48 The Committee acknowledges the benefits to Australian health flowing from research supported by the NHMRC grants system and notes the upheaval caused by the NHMRC’s recent separation from the Department of Health and its move to a statutory agency. The Committee recognises that NHMRC is still consolidating its new status but is pleased to see evidence of a tightening in administrative practices, processes and procedure.
2.49 While acknowledging the difficulties inherent in the peer review process, the Committee urges the NHMRC to implement the ANAO recommendations aimed at strengthening accountability and transparency throughout the peer review process. In particular, the Committee is concerned that conflict of interest issues may encourage a perception that the assessment and selection of grant applications is not open and fair.
2.50 The Committee commends the NHMRC for the ongoing improvement in post-award grant management but urges the agency to implement the ANAO recommendation to implement risk-based arrangements to ensure better management of Commonwealth monies.
