ASH Clinical News October 2017 | Page 64

The Rank Tank
Each ranking entity has its own ratings protocol , so the simple question , “ How are the rankings developed ?” has a complex answer .
U . S . News & World Report offers multiple ranking lists : “ Best Hospitals Specialty ,” “ Best Hospitals Procedure and Conditions Ratings ,” “ Best Hospitals Honor Roll ,” to name a few , along with rankings for regional and children ’ s hospitals . The methodology for the 2017-2018 “ Best Hospitals Specialty ” rankings is laid out in a publicly available , 137- page PDF . 5 Cancer centers ’ scores account for :
a general surgeon at Advanced Surgical Care of Northern Illinois in Barrington and vice president of medical management at Advocate Good Shepherd Hospital in Downers Grove , Illinois .
“ I ’ m coming at this from the perspective of a physician and a hospital administrator ,” he added . “ As [ the latter ], we care a great deal about these scores , but many of these companies won ’ t share their raw data unless you
enter a consulting engagement with them , and that can be quite costly .”
The mystery surrounding the scores also limits their usefulness for patients . “ Rankings are not necessarily informative to patients because the ranking entities take so many factors and aggregate them to create a ‘ star ’ rating ,” noted Charles Dinerstein , MD , a senior medical fellow at the American Council on Science and Health , where he has written
extensively on the pitfalls of ranking systems . “ What ’ s interesting and valuable to patients may very well be lost in the aggregation .”
According to a 2008 Health Affairs study comparing five nationwide ranking services and one state service , the potential for disagreement among ranking services “ appears likely to confuse , rather than inform , consumers .” 8 Michael Rothberg , MD , MPH , vice
• structure ( reflecting resources that a center makes available to patients , like the number of nurses who care for patients , 30 %)
• process / expert opinion ( measuring a hospital ’ s reputation , based on the average number of nominations from physician surveys , 27.5 %)
• outcomes ( including 30-day mortality , 37.5 %)
• patient safety ( quantifying instances when patients may be avoidably harmed or put at risk , and taken from the Health Services Cost Review Commission all-payer database , 5 %)
Consumer Reports explains its ranking system in a comparatively slim , 38-page document ; rankings reflect scores on patient outcomes , patient experiences , and hospital practices domains . 6 Leapfrog uses a proprietary survey to calculate its data , 7 but most other ranking entities mine historical Medicare data , to which they apply their ranking methods . “ You need a PhD to read the methodology documents ; I have one , and I still don ’ t understand it ,” joked Susan Moffatt-Bruce , MD , PhD , MBA , a cardiothoracic surgeon and chief quality and patient safety officer at The Ohio State University Wexner Medical Center in Columbus .
Herein lies the root of the criticism against hospital-ranking systems : They don ’ t use a universal , standardized calculation method and they publicize composite or aggregated scores , not raw data .
Even when ranking systems are transparent about their formulas , the process seems “ to take place in a black box , and when it ’ s a composite score , it ’ s difficult for a hospital to [ know where to ] take action to improve that score ,” said Barry Rosen , MD ,
62 ASH Clinical News
© 2017 Shire US Inc ., Lexington , MA 02421 . All rights reserved . 1-800-828-2088 SHIRE and the Shire Logo are registered trademarks of Shire Pharmaceutical Holdings Ireland Limited or its affiliates . ADVATE , ADYNOVATE , FEIBA , OBIZUR , RIXUBIS , and VONVENDI are registered trademarks of Baxalta Incorporated , a wholly owned , indirect subsidiary of Shire plc . S29103 05 / 17