Login | Register
Assessing the Payback from AHFMR-funded research

- SECTION 2:

METHODS


Most of the work was undertaken in a two-month period during mid-June to mid-August, 1998, and completed in October 1998. Professor Martin Buxton (of the Health Economics Research Group, Brunel University) who had led the UK studies on which this project was based, and Wendy Schneider from the AHFMR undertook the work. Within the limited time available, it was agreed to use a selective case study approach in which the case studies would be developed only as far as was necessary to begin to test the applicability of the model. It was not feasible, and was never the intention, to make the case studies exhaustive accounts of the research in question. It was the approach that was being tested, neither the research nor the researchers, chosen as test cases.

The first step was to attempt to get a reasonable understanding of the context of the work of the AHFMR and of the health system in Alberta, with a concern to establish whether there seemed to be any particular characteristics which might cause problems in applying the existing Buxton/Hanney approach. This was achieved through review of earlier studies undertaken in Alberta pertaining to this broad question, and background information on the AHFMR, the Alberta Health Care system, through informal discussions with the AHFMR's staff and with a number of researchers involved in health research. Additional insights came through discussion with others involved in the health service and analysis of the AHFMR's funding processes relating to HSR. In addition Wendy Schneider brought first-hand knowledge and experience of the local system.

To test whether the Buxton/Hanney approach could readily be applied, two case studies were chosen initially as good examples of applied HSR funded by the AHFMR. As such, they had already been fairly well documented. Indeed, the work by one researcher had been written up as a case study in the report by Kerry Toll (35). It was largely possible to undertake both case studies to the extent necessary to test the model using materials on file: research applications, reviewers comments, annual reports and copies of main publications and secondary accounts of the research to the extent necessary to test the model. Interviews with the researchers were done to verify the conclusions drawn from the file materials and to identify other categories of payback. In addition to these case studies of funded HSR, two case studies (3a and 3b) were undertaken on Health Technology Assessment (HTA) projects, part of the work of the AHFMR's HTA program. These, too, were undertaken principally from on-file documentation, and with the first hand knowledge of, and access to, the staff researchers concerned.

The second and more innovative part of the work involved establishing whether the Buxton/Hanney approach, or a variant of it, could usefully be applied to biomedical and clinical research. For this it was agreed, in this first phase to use a number of case studies to test the model and focus thinking about the problems posed by more basic science. On the advice of the AHFMR's staff, four distinguished researchers, from the Universities of Alberta and Calgary, were chosen each of whom clearly was acknowledged as a leader in their respective fields, and whose research is of the highest standard. The first two were Clinical Investigators: Dr. "A", and Dr. "B". The second two were basic biomedical scientists: Dr. "C" and Dr. "D". All four had received extensive AHFMR funding over a number of years. In each case the mode of working was, again, initially to undertake a desk review of materials on file and then to interview the researchers themselves and to explore the way that they perceived and conceptualized the 'payback' on their own research and to consider how this payback might be assessed. For these four interviews, a common structure of broad questions was used, although general questions were individualized somewhat as a result of the initial desk review of their research. The general questions are set out in Figure 3. Due to time constraints, it was not possible to follow-up issues raised in these interviews with others, such as those who may have used the research findings, and hence, once again, the case studies should not be seen as definitive analyzes of the payback from these streams of research. They simply provide an initial indication of how the model might be used in such contexts.

In parallel with the case studies, a continuing review of relevant literature on the evaluation of the benefits of research, particularly basic science research was undertaken. While at this stage this does not represent a systematic review of that diffuse, and difficult to access, existing knowledge base, our work does draw on some of this research, by way of ideas or contrasting approaches.


Figure 3: Interview questions for researchers

  • What was the original impetus for this program of research?

  • What drives your research. Scientific curiosity? The need to fill certain gaps in knowledge? The inadequacy of current treatments?

  • What are your main research objectives and how will they be important scientifically and clinically? What do you see as the main expected benefits of your planned research?

  • How internationally competitive is this area and how collaborative?

  • What kind of interaction is involved in determining your research plans? With funders? With University colleagues? With peers internationally in specific research communities?

  • How do you see the results of your immediate research being used and by whom?

  • What are the really important ways in which results and ideas get shared between the main players in the research community?

  • Have other centres picked up and further developed ideas stemming from your research?

  • Have others adopted specific techniques or tools you have developed?

  • Have the plans for your program been affected by results from elsewhere?

  • How will these results feed through into the development of improved clinical therapy/treatment?

  • What are the key steps in this process, or links in this chain?

  • Realistically what sorts of time scales are likely to be involved?

  • How many years are you from having an established proven clinical therapy that might be adopted by clinical centres elsewhere?

  • What do you see as the best indicator of the importance or value of research such as this?

  • Is publication in the major peer-review journals the key channel or an indicator of quality of the work?

  • What for you personally are the key indicators of success?

  • What features of your situation at this University are particular advantages/disadvantages to pushing this research forward successfully?
MJB and WLS 14/7/98


Next >>ContentsSummary<< Previous

-------------------------------------