|Year : 2014 | Volume
| Issue : 7 | Page : 761-767
The methodological quality of systematic reviews comparing intravitreal bevacizumab and alternates for neovascular age related macular degeneration: A systematic review of reviews
Pradeep Paul George, Joseph Antonio DeCastro Molina, Bee Hoon Heng
Health Services and Outcomes Research, National Healthcare Group HQ, Singapore
|Date of Submission||04-Oct-2013|
|Date of Acceptance||20-Feb-2014|
|Date of Web Publication||13-Aug-2014|
Pradeep Paul George
National Healthcare Group, 6 Commonwealth Lane, #04-01/02 GMTI Building, 149547
Source of Support: None, Conflict of Interest: None
Objective: To systematically collate and evaluate the evidence from recent SRs of bevacizumab for neo-vascular age related macular degeneration. Materials and Methods: Literature searches were carried out in Medline, Embase, Cochrane databases for all systematic reviews (SRs) on the effectiveness of bevacizumab for neo-vascular age related macular degeneration, published between 2000 and 2013. Titles and abstracts were assessed against the inclusion/exclusion criteria using Joanna Briggs Institute (JBI) study eligibility form. Data was extracted using the JBI data extraction form. The quality of the SRs was assessed using JBI critical appraisal checklist for SRs. Decisions on study eligibility and quality were made by two reviewers; any disagreements were resolved by discussion. Results: Nine relevant reviews were identified from 30 citations, of which 5 reviews fulfilled the review's inclusion criteria. All 5 reviews showed bevacizumab to be effective for neovascular AMD in the short-term when used alone or in combination with PDT or Pegaptanib. The average quality score of the reviews was 7; 95% confidence interval 6.2 to 7.8 (maximum possible quality score is 10). The selection and publication bias were not addressed in all included reviews. Three-fifth of the reviews had a quality score of 7 or lower, these reviews had some methodological limitations, search strategies were only identified in 2 (40%) reviews, independent study selection and quality assessment of included studies (4 (80%)) were infrequently performed. Conclusion: Overall, the reviews on the effectiveness of intravitreal/systemic bevacizumab for neovascular age-related macular generation (AMD) received good JBI quality scores (mean score = 7.0 points), with a few exceptions. The study also highlights the suboptimal reporting of SRs on this topic. Reviews with poor methodology may limit the validity of the reported results; hence efforts should be made to improve the design, reporting and publication of SRs across all journals.
Keywords: Bevacizumab, intravitreal, neovascular age related macular degeneration, systemic, systematic review, systematic review of reviews
|How to cite this article:|
George PP, DeCastro Molina JA, Heng BH. The methodological quality of systematic reviews comparing intravitreal bevacizumab and alternates for neovascular age related macular degeneration: A systematic review of reviews. Indian J Ophthalmol 2014;62:761-7
|How to cite this URL:|
George PP, DeCastro Molina JA, Heng BH. The methodological quality of systematic reviews comparing intravitreal bevacizumab and alternates for neovascular age related macular degeneration: A systematic review of reviews. Indian J Ophthalmol [serial online] 2014 [cited 2020 Mar 31];62:761-7. Available from: http://www.ijo.in/text.asp?2014/62/7/761/138615
Age-related macular degeneration (AMD) is a progressive chronic disease of the central retina and the leading cause of blindness in elderly people worldwide; including Singapore. ,,,,,,,,,,,,, With increasing life expectancy, the worldwide prevalence of AMD is set to increase. Neovascular AMD is characterized by abnormal growth of new blood vessels under the macula, it accounts for only 20% of the cases with AMD, but responsible for 90% of the cases of legal blindness. 
Over the past decade, there have been a variety of medical therapies introduced with variable success to treat the neovascular form of AMD. Recently, the anti-vascular endothelial growth factors (VEGF) pegaptanib and ranibizumab became available for the treatment of exudative AMD. ,,,, In contrast to pegaptanib, intravitreal injections with ranibizumab led to a significant vision improvement. In fact, ranibizumab was the first drug to improve vision in exudative AMD, compared to interventions used earlier which only delayed progression. , Major disadvantages of this intervention are the costs and the need to give intravitreal injections repeatedly. Bevacizumab which is closely related to ranibizumab is derived from the same parent murine monoclonal anti-body as ranibizumab and is an off-label drug used for neovascular AMD. Interestingly both these drugs are from the same biotech company, which is not willing to approve bevacizumab for exudative AMD, as doing so would jeopardize its market for ranibizumab, which is the approved drug for the condition.
Bevacizumab was originally approved for the treatment of colon cancer but not for the treatment of AMD. A major advantage of bevacizumab is its price, which is about 1-5% the price of ranibizumab. Mainly for this reason, it is now used worldwide and on a large-scale off-label for the treatment of exudative AMD.
Medical and public health decisions are informed by systematic reviews, which make the quality of reviews an important scientific concern. Evidence from observational studies and few randomized clinical trials (RCT's) suggest that off-label bevacizumab is effective for the treatment of exudative AMD. However, evidence from systematic reviews on the effectiveness of bevacizumab for AMD is equivocal, with some systematic reviews in favor of intravitreal bevacizumab while some are not. This sets the stage for critical appraisal of quality of systematic reviews on intravitreal bevacizumab for neovascular age related macular degeneration.
| Materials and Methods|| |
We adopted a 'review of reviews' approach as proposed by the Health Development Agency (HDA):  Given the increasing number of systematic reviews of interventions in ophthalmology literature,  the goal of this systematic review of reviews is to collate the evidence and appraise the quality of evidence from published reviews on intravitreal bevacizumab for neovascular AMD.
Inclusion and exclusion criteria
We searched the literature for published systematic reviews (and meta-analyses) of observational and experimental studies whose primary purpose was to ascertain the effectiveness of intravitreal bevacizumab for neovascular AMD. Reviews which looked at patients with neo-vascular AMD regardless of the stage/severity of disease or with subfoveal choroidal neovascularization (CNV) secondary to neovascular AMD were included. The intervention of interest in the review was intravitreal bevacizumab compared against photodynamic therapy (PDT) with Verterporfin, Pegaptanib or Ranibizumab. Reviews which were not labeled as systematic but included an explicit statement of search methods, inclusion, exclusion, data synthesis were included as well. The primary outcome of interest for the review was improvements in visual acuity (VA) measured by Snellen or ETDRS charts and decrease in central retinal thickness (CRT). All types of primary studies (observational and experimental) were excluded; similarly reviews which depended upon previous systematic reviews for their primary data were excluded.
| Search Strategy|| |
We conducted searches in three databases Medline (via PubMed), EMBASE, Cochrane Library and Centre for Reviews and Dissemination (CRD) databases for systematic reviews published between 2000 and 2013. The following are free text keywords used "Macula*" or "AMD" or "ARMD" or "intra (-) vitreous" or "intra-vitreal" in any field, and for the intervention the search terms are "bevacizumab" or "avastin" in any field. The following mesh terms were also searched in combination with the above free text keywords, (("Intravitreal Injections"[Mesh] OR "Injections, Intravenous"[Mesh]) AND "bevacizumab "[Substance Name]) AND "Wet Macular Degeneration"[Mesh] AND ("humans"[MeSH Terms] AND (Meta-Analysis[ptyp] OR Review[ptyp])). The search was restricted to English language publications and systematic reviews. Search for unpublished systematic reviews included searches conducted in ProQuest Dissertations and Thesis, Conference Proceedings and Mednar.
Review selection and critical appraisal
Titles and abstracts of identified reviews were examined independently by 2 reviewers to assess each review for inclusion using Joanna Briggs Institute (JBI) study eligibility form ( Appendix I). [Additional file 1] Data from eligible reviews was extracted using the JBI data extraction form (Appendix II). [Additional file 2] The quality of the systematic reviews was assessed using JBI critical appraisal checklist (Appendix III) [Additional file 3] for systematic reviews which looks at 10 critical aspects of the systematic review. Scores were given for adherence to each of those aspects, A minimum score of 1 and a maximum score of 10 would signify a well conducted SR. Decisions regarding study eligibility and quality were made by two reviewers, any disagreements were resolved by discussion.
Data extraction and synthesis
Information on primary studies was extracted from the reviews; in the case where reviews reported discrepant study findings, the primary studies were consulted. Information on the reviewers' assessment of the evidence, design and findings of relevant primary studies were extracted from each review. The level of evidence in support of (or discounting) the effect of an intervention was classified as: 'sufficient';' tentative'; 'insufficient'; or 'no' evidence from reviews. These were derived using a framework [Table 1] based on the quality of the reviews, the reviewers' conclusions and the designs/findings of the primary studies included in the reviews.  In the absence of controlled trials, longitudinal cohort and case-control designs (involving incident cases) were considered to be more 'robust', whereas ecological, serial cross-sectional and cross-sectional designs were considered to be 'weaker'.
|Table 1: Types of evidence statement and the level of evidence that was required to support each statement |
Click here to view
| Results|| |
Nine relevant reviews were identified from over 30 citations, of which 5 reviews fulfilled the review's inclusion criteria. The following were the reasons for excluding 4 reviews: Studies primarily looked at the safety of bevacizumab without focusing on its effectiveness (2), the major criteria for quality appraisal were not satisfactorily fulfilled (2), (search sources inadequate, criteria for critical appraisal inappropriate, critical appraisal not done independently, data extraction not independently done, methods used for combining studies were inappropriate). [Table 2] shows the included and excluded studies with their corresponding JBI quality score.
|Table 2: JBI critical appraisal quality score and summary of conclusions of the retrieved reviews |
Click here to view
Quality of studies included in the reviews
The 5 included reviews were published between 2009 and 2012; they summarized 126 publications (after excluding duplicate publications) on intravitreal bevacizumab for neovascular age related macular degeneration. 19% of these publications were prospective studies and 10% of these were randomized controlled trials (RCTs) and the rest had designs with no control group which includes case series and before-after studies.
The review question was clearly stated in all included reviews either in the title, abstract or in text. The average quality score of the reviews was 7 (95% confidence interval 6.2 to 7.8); maximum possible quality score is 10 [Table 3]. Publication bias was not addressed in all included reviews. Three-fifth of the reviews had a quality score of 7 or lower, these reviews had some methodological limitations. The search strategy was appropriate only in 40% of the reviews. The sources of studies and inclusion criteria were appropriate for 60% of the reviews. Criteria for appraisal were appropriate and independent appraisal was conducted in most of the reviews. Methods used to minimize error in data extraction were not specified in most of the reviews [Table 4]. Three out of the five reviews searched 2 or more databases. Data were tabulated as narrative summaries due to the heterogeneity of the design and the findings [Table 5].
|Table 3: Quality score of JBI critical appraisal checklist by 2 independent reviewers |
Click here to view
|Table 4: Validity assessments with JBI critical appraisal check list after adjudication |
Click here to view
|Table 5: Selected reviews published on intravitreal bevacizumab and age-related macular degeneration by outcome and year published |
Click here to view
All 5 reviews showed bevacizumab to be effective for neovascular AMD in the short-term when used alone or in combination with PDT or Pegaptanib. [Table 2] shows the quality of reviews. The SR by JSAG Schouten  which was of good quality [Table 2] showed improvements in visual acuity and decrease in central retinal thickness in patients with exudate AMD after bevacizumab [Table 3]. RCT's showed that intravitreal bevacizumab was more effective than PDT with a mean change in visual acuity (VA) of +12.8 ETDRS letters (range +11 to +14) and weighted mean change for CRT of 129μm (range 100 to 202). The before-after studies of bevacizumab showed a mean change in VA of +8.6 letters (range +2 to +26) and change in CRT was 90 um (range 46 to 190). Review by Ziemssen et al.,  found off-label use of bevacizumab superior to PDT, in the short term, however the evidence was of moderate quality (2b level of evidence) primarily from before-after studies, case series and RCT's of bevacizumab for neovascular AMD. Andriolo et al., in their meta-analysis showed that BCVA was higher among those treated with bevacizumab than those in the PDT group (risk ratio, RR, 0.49; 95% confidence interval (CI), 0.31 to 0.78; P = 0.01). Jyothi et al., summarized the findings from 5 RCT's and 50 observations studies that met "Strengthening the Reporting of Observational Studies in Epidemiology" (STROBE) criteria.  The review showed Intravitreal bevacizumab led to mean gain in vision at 3 months (+7.76 ± 5.4 ETDRS letters (range +2 to +14.4); this effect was maintained at 6 months in studies with longer follow-up. Review by Pitlick et al.,  summarized findings from 4 RCT's, the review showed that bevacizumab in comparison to verteporfin PDT or pegaptanib to be superior in improving VA and reversing vision loss. Bevacizumab when compared to ranibizumab was found to have equivalent efficacy and significantly less expensive.
| Discussion|| |
Systematic reviews in ophthalmology are increasing at a rapid pace, but little information is available on the quality of these reviews. This systematic review of reviews appraises the quality of evidence from SRs on effectiveness of bevacizumab for neovascular AMD. Overall, the published reviews on this topic were of good quality, the reviews used 7 of 10 best practices for an objective and systematic methodology as enumerated by the JBI quality appraisal checklist with an a priori study design.
Bias arising from publication has an important influence on the results of systematic reviews and is more difficult to detect and adjust for than other forms of bias. Thus, when searching for studies to include in a systematic review, it is critical that the literature search strategy is explicitly described, comprehensive, and inclusive of both unpublished and published research.  The description of the search strategy provides some assurance that the authors have conducted a comprehensive, detailed, and an exhaustive search for literature relevant to the study question. The search strategy and sources of studies were found to be appropriate for 40% and 60% of the reviews respectively. Another key feature of the systematic review is to assess bias, to assess the funding sources and financial conflicts of authors as they can influence the outcomes the studies. Industry funded studies are more likely to have results that favor their product, even when controlling for other methodological biases. In our review, only one study had disclosed the conflict of interest,  this study also had much methodological weakness and hence was not included.
A well-conducted systematic review provides readers with an accurate summary and defensible synthesis of the available evidence. The inclusion of a methods section is essential for the transparency of results and provides the reader with the means to reproduce the review if need be. In the absence of these basic methodological features, conclusions shown in reviews may be little more than personal subjective opinions informed by the scientific evidence but not based on strong methodological grounds.  In this review of the reviews we found that none of the reviews were rated as a 10 out of 10 for quality, moreover 1 review had significant flaws, the methods sections of this review was insufficient and therefore difficult to evaluate. As far as the quality of the systematic review is concerned there is scope for improvement. We recommend that ophthalmology journals should adopt uniform reporting standards for systematic reviews; this would pave way for high-quality systematic reviews in ophthalmology literature.
The review points to the overwhelming evidence available in support of bevacizumab for AMD, intravitreal bevacizumab was associated with a mean gain in visual acuity of 7.76-12.8 ETDRS letters and mean decrease in CRT of 129μm after intravitreal bevacizumab. We do not have other systematic reviews of bevacizumab for neovascular age related macular degeneration with which to compare our conclusions, but they are consistent with the conclusions of 7 of the 9 most recent systematic reviews.
Our study is a review of systematic reviews. There are some inherent weaknesses in this approach. In general due to resource constraints, we relied on the information in the included reviews. The quality of the reviews may vary; the reviews may have done a poor job in specifying their inclusion and exclusion criteria, the searches may not be comprehensive, the review authors may not have assessed or extracted data from the primary studies adequately, nor analyzed and synthesized the ﬁndings across the studies properly. But even using high quality reviews, we necessarily lose information and details that we can only ﬁnd if we go back to the primary studies. Our literature search only covered the three databases and hand searching was not done due to resource constraints. Not withstanding the previously mentioned concerns, to our knowledge, this is the first study which appraises the quality of published systematic reviews on bevacizumab for neovascular AMD.
| Conclusion|| |
Reviews have found bevacizumab to be effective for neovascular AMD when used either alone or in combination with alternates such as PDT or Pegatanib. This study also highlights the quality of systematic reviews examining the effectiveness of bevacizumab for neovascular age related macular degeneration, highlighting the methodological shortcomings and the need for uniform reporting of the preferred items for systematic reviews in ophthalmology literature.
| References|| |
Klein R, Klein B, Knudtson MD, Wong TY, Cotch MF, Liu K, et al
. Prevalence of age-related macular degeneration in 4 racial/ethnic groups in the multi-ethnic study of atherosclerosis. Ophthalmology 2006;113:373.
Mitchell P, Smith W, Attebo K, Wang JJ. Prevalence of age-related maculopathy in Australia. The Blue Mountains Eye Study. Ophthalmology 1995;102:1450.
Varma R, Fraser-Bell S, Tan S, Klein R, Azen SP. Prevalence of age-related macular degeneration in Latinos: The Los Angeles Latino eye study. Ophthalmology 2004;111:1288-97.
Wong T, Loon S, Saw S. The epidemiology of age related eye diseases in Asia. Br J Ophthalmol 2006;90:506-11.
Ferris FL 3 rd
, Fine SL, Hyman L. Age-related macular degeneration and blindness due to neovascular maculopathy. Arch Ophthalmol 1984;102:1640.
Brown DM, Kaiser PK, Michels M, Soubrane G, Heier JS, Kim RY, et al
. Ranibizumab versus verteporfin for neovascular age-related macular degeneration. N Engl J Med 2006;355:1432-44.
Rosenfeld PJ, Brown DM, Heier JS, Boyer DS, Kaiser PK, Chung CY, et al
. Ranibizumab for neovascular age-related macular degeneration. N Engl J Med 2006;355:1419-31.
Kovach JL, Schwartz SG, Flynn HW Jr, Scott IU. Anti-VEGF Treatment Strategies for Wet AMD. J Ophthalmol 2012;2012:786870.
Kelly MP, Swann C, Morgan A, Killoran A, Naidoo B, Barne, et al
. Methodological problems in constructing the evidence base in public health. London: Health Development Agency, 2002. Available from: http://www.hda.nhs.uk/evidence
. [Last accessed on 2013 Sep 10].
Chen H, Jhanji V. Survey of systematic reviews and meta-analyses published in ophthalmology. Br J Ophthalmol 2012;96:896-9.
Ellis S, Barnett-Page E, Morgan A, Taylor L, Walters R, Goodrich J. HIV prevention: A review of reviews assessing the effectiveness of interventions to reduce the risk of sexual transmission: Evidence briefing summary. Health Development Agency; 2003.
Schouten JS, La Heij EC, Webers CA, Lundqvist IJ, Hendrikse F. A systematic review on the effect of bevacizumab in exudative age-related macular degeneration. Graefes Arch Clin Exp Ophthalmol 2009;247:1-11.
Ziemssen F, Grisanti S, Bartz-Schmidt KU, Spitzer MS. Off-label use of bevacizumab for the treatment of age-related macular degeneration. Drugs Aging 2009;26:295-320.
Andriolo RB, Puga ME, Belfort Junior R, Atallah AN. Bevacizumab for ocular neovascular diseases: A systematic review. Sao Paulo Med J 2009;127:84-91.
Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al
. Strengthening the reporting of observational studies in epidemiology (STROBE): Explanation and elaboration. PLoS Med 2007;4:e297.
Pitlick JM, Vecera KF, Barnes KN, Reski JW, Forinash AB. Bevacizumab for the treatment of neovascular age-related macular degeneration. Ann Pharmacother 2012;46:290-6.
Weed DL, Althuis MD, Mink PJ. Quality of reviews on sugar-sweetened beverages and health outcomes: A systematic review. Am J Clin Nutr 2011;94:1340-7.
Palmateer N, Kimber J, Hickman M, Hutchinson S, Rhodes T, Goldberg D. Evidence for the effectiveness of sterile injecting equipment provision in preventing hepatitis C and human immunodeficiency virus transmission among injecting drug users: A review of reviews. Addiction 2010;105:844-59.
Jyothi S, Chowdhury H, Elagouz M, Sivaprasad S. Intravitreal bevacizumab (Avastin) for age-related macular degeneration: A critical analysis of literature. Eye (Lond) 2010;24:816-24.
Schmucker C, Ehlken C, Agostini HT, Antes G, Ruecker G, Lelgemann M, et al
. A safety review and meta-analyses of bevacizumab and ranibizumab: Off-label versus goldstandard. PloS One 2012;7:e42701.
Schmucker C, Loke YK, Ehlken C, Agostini HT, Hansen LL, Antes G, et al
. Intravitreal bevacizumab (Avastin) versus ranibizumab (Lucentis) for the treatment of age-related macular degeneration: A safety review. Br J Ophthalmol 2011;95:308-17.
Lynch SS, Cheng CM. Bevacizumab for neovascular ocular diseases. Ann Pharmacother 2007;41:614-25.
Smit DP, Meyer D. Intravitreal bevacizumab: An analysis of the evidence. Clin Ophthalmol 2007;1:273-84.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]