Introduction
Systematic reviews are a core component of our ability to practice evidence-based care. There is no doubt that these reviews have been a tremendous development and changed orthodontic clinical practice. Unfortunately, I am beginning to tire of them. This post is about my current thoughts on the current state of systematic reviews.
I was prompted to write this post after reading an excellent editorial in the AJO-DDO. A UK/Ireland-based team of well-known researchers wrote the article.
Systematic reviews in orthodontics: A fresh look to promote renewal and reduce redundancy
Declan Millett et al. Am J Orthod Dentofacial Orthop 2022;162:1-2
The rise of systematic reviews
I will start this discussion by going back to the mid-1990s. Bill Shaw asked me to come to a meeting about a potential research opportunity. I turned up and found that the meeting was with Iain Chalmers, one of the founders of the Cochrane Collaboration. He wanted to discuss the development of a Cochrane Oral Health Group. We were utterly blown away by the concept of systematic reviews as a research method.
At that time, I had just started the UK early Class II treatment trial. As a result, I was too busy to take anything else on, and I decided to concentrate on setting up this study. Consequently, I simply watched the great work Bill did setting up the Cochrane Oral Health Group.
Nevertheless, I got involved with some editorial work and contributed to several systematic reviews. It was a great time for orthodontic research. Many people started reviews that condensed our evidence base and changed practice. Systematic reviews were great, and we were going to change the World!
Now many years later. I wonder if researchers are doing too many systematic reviews. Unfortunately, we are now all too familiar with this standard conclusion of many reviews.
“The overall quality of the evidence was low. There is a need for high-quality RCTs in this area”.
All good things must pass.
The cynic in me wonders if the systematic review has simply become a method to produce a paper and build a CV. This would be similar to the multitude of bonding studies that were done in the mid-1980s. There is nothing wrong with these papers. In fact, my first paper was a bonding study. But there comes a time when we need to take stock and consider whether a type of research adds to clinical knowledge. I wonder if we are at this point with orthodontic systematic reviews.
I feel this way because, unfortunately, many reviews include low-quality papers and retrospective studies full of bias. The authors seem to include these papers because they could not find any trials. This simply dilutes the quality of the review, no matter which convenient “tool for bias” is used. As a result, the reviews are bound to conclude that the evidence was not strong. Thus, adding to research waste.
This problem has now reached the point where I don’t read many systematic reviews. We certainly have reduced the number of reviews discussed in this blog.
Solutions from the editorial?
The editorial authors discussed this situation much more politely than I did. Importantly, they highlighted the research time wasted by many reviews. I was, therefore, pleased to read that they put forward some solutions to this problem.
The central solution that they proposed was straightforward. The authors suggested that instead of stating that “more evidence is required.” The researchers should develop a protocol for a trial that could address the deficiencies that they identified. Importantly, they should publish this protocol in one of the journals that publish protocols. This would lead to the development of much-needed research.
Final comments
In my general grumpy state, I would go further and suggest that a journal should not publish a systematic review without outlining future trials. Neither should they publish reviews that include mostly retrospective studies. This, of course, is up to the journal editors, who are the gatekeepers to the publication of papers.
As the paper editors suggested, this will lead to the “nirvana” of reviews being carried out, studies designed, and collaborators found to carry out the trials. Importantly, this would move orthodontic research forwards to address some clinical problems we face. Furthermore, it would allow us to build our evidence base. So let’s adopt the suggested principles and look forward to improving our orthodontic research.