Assessing scientific research: don't forget to also look at the (in)dependence of the researchers

Currently, we assess scientific research primarily by looking at elements of the research design, such as: was it randomized, was it blinded, and so on. Occasionally, we also look at the influence of the funding agency. Were the researchers able to do their work independently? Thomas Hoogeboom (IQ Health) and colleagues went a step further and examined whether we should not also look at the independence of researchers from the research idea; a phenomenon called "Researcher Allegiance.

Research Allegiance is the researchers' belief in the superiority of the treatment or theory they are investigating. Belief, of course, is a difficult concept to quantify. Hoogeboom, in collaboration with researchers at the University of Indianapolis and George Washington University, combined insights from psychotherapy and medicine in this area to develop a methodology. They then applied this methodology to the literature on the effectiveness of Dry Needling. They found that every study showed at least some form of Researcher Allegiance, while no conflicts of interest were reported. From other studies, studies with higher Researcher Allegiance also appear to show greater effects. It is easy to see Researcher Allegiance as something negative. But that is not necessarily the case. The noted epidemiologist John Ioannidis previously stated about this phenomenon, "It is very informative, if even the best clinical experts cannot achieve clinically meaningful effects. In that scenario, the researchers can safely quit further testing, as the intervention will do worse in the hands of others."

Hoogeboom and colleagues' advice is that researchers should be reflexive as much as possible. In other words, researchers should examine their own feelings, reactions and motives regarding what they study and report on it in scientific publications. This is already happening in qualitative research, but not yet in quantitative research. According to Hoogeboom, this is a missed opportunity to better understand what works, what doesn't and why.
 

Read the publication here .