Comparisons of RCT and preference-based interventions – facilitated by Associate Professor Justin Keogh.

Methodological Question: What research questions might be best addressed by RCTs or preference designs whereby at least some of the participants get a choice on what intervention they are allocated to?

Reference: Janevic, M. R., Janz, N. K., Dodge, J. A., Lin, X., Pan, W., Sinco, B. R., & Clark, N. M. (2003). The role of choice in health education intervention trials: a review and case study. Social Science and Medicine, 56(7), 1581-1594.

Abstract: Although the randomized, controlled trial (RCT) is considered the gold standard in research for determining the efficacy of health education interventions, such trials may be vulnerable to “preference effects”; that is, differential outcomes depending on whether an individual is randomized to his or her preferred treatment. In this study, we review theoretical and empirical literature regarding designs that account for such effects in medical research, and consider the appropriateness of these designs to health education research. To illustrate the application of a preference design to health education research, we present analyses using process data from a mixed RCT/preference trial comparing two formats (Group or Self-Directed) of the “Women take PRIDE” heart disease management program. Results indicate that being able to choose one’s program format did not significantly affect the decision to participate in the study. However, women who chose the Group format were over 4 times as likely to attend at least one class and were twice as likely to attend a greater number of classes than those who were randomized to the Group format. Several predictors of format preference were also identified, with important implications for targeting disease-management education to this population.

There is a general belief that lifestyle factors like physical activity and a good diet are important for health and in reducing the symptoms and/or rates of many chronic conditions.  However, many people are still insufficiently active and/or consume an inadequate diet.  In addition, many participants in trials involving physical activity or dietary interventions do not complete the intervention.  The question then becomes, is the RCT always the best design for an intervention study and if not, how can the preference design be used in some research and/or in practice contexts to improve these outcomes?  For example, a GP might try encouraging a patient to be more physically active and wishes to determine what form of activity the patient may most like to do, so to maximize the potential that the patient will adhere to this activity in the medium- and long-term.

 

Group Discussion

  • Initial discussions centred on the relative advantages and disadvantages of the RCT and preference design.  These focused on the better internal validity of the RCT and the better external validity of the preference design.  This discussion then moved into comparing the various types of preference designs and what loss of internal validity by using a preference design would be acceptable.
  • Based on these relative strengths and weaknesses, it was felt that these two approaches serve quite different needs.  The RCT assesses the potential benefit and risk of an intervention in tightly controlled circumstances; with the aim being to demonstrate an effect.  When sufficient research has been conducted to clearly demonstrate an effect, the preference design is then better suited to determine the real-world uptake of an intervention and the determinants of this uptake.  Some discussion again centred on how much research was enough and how closely the samples used in these studies related to the patients with whom you work in practice.
  • Overall, it was felt that the preference design is perhaps under-utilised in many research fields and when used appropriately can better demonstrate the real-world effects of interventions, as may be of interest to many clinicians.