Development of an online tool for linking behavior change techniques and mechanisms of action based on triangulation of findings from literature synthesis and expert consensus

Abstract Researchers, practitioners, and policymakers develop interventions to change behavior based on their understanding of how behavior change techniques (BCTs) impact the determinants of behavior. A transparent, systematic, and accessible method of linking BCTs with the processes through which they change behavior (i.e., their mechanisms of action [MoAs]) would advance the understanding of intervention effects and improve theory and intervention development. The purpose of this study is to triangulate evidence for hypothesized BCT–MoA links obtained in two previous studies and present the results in an interactive, online tool. Two previous studies generated evidence on links between 56 BCTs and 26 MoAs based on their frequency in literature synthesis and on expert consensus. Concordance between the findings of the two studies was examined using multilevel modeling. Uncertainties and differences between the two studies were reconciled by 16 behavior change experts using consensus development methods. The resulting evidence was used to generate an online tool. The two studies showed concordance for 25 of the 26 MoAs and agreement for 37 links and for 460 “nonlinks.” A further 55 links were resolved by consensus (total of 92 [37 + 55] hypothesized BCT–MoA links). Full data on 1,456 possible links was incorporated into the online interactive Theory and Technique Tool (https://theoryandtechniquetool.humanbehaviourchange.org/). This triangulation of two distinct sources of evidence provides guidance on how BCTs may affect the mechanisms that change behavior and is available as a resource for behavior change intervention designers, researchers and theorists, supporting intervention design, research synthesis, and collaborative research.

. BCT changing behaviour through mechanism(s) of action. Note that there may be more than one mechanism of action for any BCT, and a BCT-mechanism of action link may be direct or indirect.

Information Provided
In Appendix D, you will find confidential data relating to inconsistencies and uncertainties between the literature synthesis and expert consensus findings. This information will be presented to you on screen during the consensus exercise, and relates to the following types of hypothesised BCT-mechanism of action links:

Question
Links from the four categories above will be presented in a randomised order, along with data from the literature synthesis and expert consensus. You will be asked to answer the following question:   BCT-mechanism of action links, as hypothesised by authors, were extracted from 277 behaviour change intervention papers.  In order for a link to be extracted, the author(s) must have explicitly hypothesised that a particular BCT would have its effect on behaviour through a particular mechanism of action.  BCTs were extracted using the coding guidelines for BCT Taxonomy v1.

Response
Mechanisms of action were extracted using the intervention authors' definitions, and then categorised into one of the 26 mechanisms of action listed in Appendix B.  The 'frequency' of a given BCT-mechanism of action link refers to the number of papers that reported that link (out of a total 277).  In order to determine whether a particular link occurred more frequently than would be expected compared to chance, a series of binomial tests were conducted on the data. The expected frequency used for comparison in the binomial test was calculated for each individual link, and was the product of the probability a given BCT was coded in a paper × the probability a given mechanism was coded in a paper.  The p value in the tables below represents an index of the likelihood that an observed frequency of a BCT-mechanism of action link occurred more often than would be expected if that link was a random pairing of a BCT and a mechanism of action (i.e. lower p values denote higher likelihood).  We are considering there to be 'evidence' for a specific link where p < .05.

Expert Consensus Study Method:
 105 behaviour change experts were asked to consider links between BCTs and mechanisms of action in an online expert consensus exercise. Experts were divided into 5 groups, with 21 experts per group, and each group rated a sub-sample of BCTs. Thus, for the links below, n = 21 unless otherwise specified.  Experts were asked the question 'When [BCT X] works, does it work through changing [Mechanism of Action Y]?'  The response options were 1. 'Definitely Yes' (i.e. When this BCT is effective in changing behaviour, it does so by changing this mechanism of action); 2. 'Definitely No' (i.e. When this BCT is effective in changing behaviour, it does not do so by changing this mechanism of action); 3. 'Possibly' (i.e. When this BCT is effective in changing behaviour, it is possible that it does so by changing this mechanism of action) and; 4. 'Don't Know / Not Possible to Say' (i.e. It is not possible to say (e.g. not enough information to decide) and/or you are very uncertain about whether or not the BCT changes behaviour by changing this mechanism of action).  We are considering there to be 'evidence' for a link where ≥80% of experts answered 'Definitely Yes'. All tables below are ordered according to the percentage of experts who answered 'Definitely Yes'.

Theories and Techniques of Behaviour Change Project -Study 3 Susan Michie, Marie Johnston, Alex Rothman, Mike Kelly, Marijn de Bruin
Email bct.theory@ucl.ac.uk

Appendix D: Data from Literature Synthesis and Expert Consensus
This appendix contains the links for which there are inconsistencies or uncertainties across literature synthesis and expert consensus (i.e. the links you will be asked to consider in the consensus exercise). Note that all data is confidential and should not be reproduced without permission.