"Transparent" Hospital Mortality Data

The Center for Medicare & Medicaid Services (CMS) has updated its US Department of Health and Human Services site to include a way for consumers to compare (see: http://www.hospitalcompare.hhs.gov/hospital-compare) three hospitals in their geographic area.  Although this new information is a terrific step in the right direction (i.e., transparency), what is presented is still rather rudimentary in terms of: 

• ease of locating -- unless you click on "Resource Locater," you won't find "Hospital Compare" -- there is nothing on the site to draw you to this new feature
• breadth of specific disease comparisons --there are only six choices of diseases to compare in terms of how well the three chosen hospitals have performed, and even here we are being generous with six, since three of the choices are 'heart failure,' 'heart attack' and 'chest pains;'  if you group these three, there are actually only four possible disease treatment comparisons
• specificity around reporting results -- if you click the "View Tables" button on the Hospital Compare page, and you click on 'Pneumonia' from the drop down menu of disease treatments to compare, you see a table with a row citing 4877 hospitals in the US that have been studied for pneumonia readmissions; in looking at the information, you will see that 367 hospitals are immediately eliminated because of insufficient data; that leaves 4,510 hospitals, 421 of which are either 'better' or 'worse' than the national rate, meaning 93% of those studied with sufficient data were 'no different' than the national rate
• completeness -- not all the fields are fully populated -- in the table below: "Out of 198 in" ... in where?  my state? my region? -- On the page 'Calculation of 3-Day Risk-Standardized Mortality Rates and Rates of Readmission,' the programmer's notes somehow got uploaded -- 'All-cause mortality is defined as death from any cause within 30 days after the add link here index admission- Opens in a new window date, regardless of whether the patient dies while still in the hospital or after discharge.' 
• data validity -- the data was collected over a recent 3-year period (2007-2010) from Medicare patients; patients who have Medicare Advantage Plans (like an HMO or PPO) and people who don't have Medicare were not included; plus data was derived solely from billing records, not patient charts, so there is no allowance for multiple diagnoses.

Table from Compare Hospitals (Pneumonia Readmissions):
Out of 4877 in the United States → 201 hospitals in the United States were Better than U.S. National Rate 4089 hospitals in the United States were No different than U.S. National Rate 220 hospitals in the United States were Worse than U.S. National Rate 367 hospitals in the United States did not have enough cases to reliably tell how well they are performing Out of 198 in → 11 hospitals were Better than U.S. National Rate 172 hospitals were No different than U.S. National Rate 10 hospitals were Worse than U.S. National Rate 5 hospitals in did not have enough cases to reliably tell how well they are performing

Interestingly, we did manage to find more underlying data on the "Mortality Graphs" page (click on "Graphs" instead of "Tables,"), which shows for three hospitals selected (Brookhaven Memorial Hospital, Peconic Bay Medical Center and Southampton Hospital) 'Rates of Readmissions for Pneumonia Patients' of 22.4, 19.1 and 18.7, respectively.  So I know from this that when it comes to readmissions for pneumonia patients, from the 3-closest hospital comparative standpoint, Brookhaven is the worst, Peconic Bay is a little better and Southampton is better even still.  I also know from this graph that Brookhaven is worse than the national average, while Peconic Bay and Southampton are considered within the bracket for the national average (18.4%).  

Data showing readmission rates for pneumonia patients is helpful, particularly when compared.  But it's unclear what the exact relationship is between pneumonia readmission rates and actual pneumonia mortality rates.  Are those readmitted for pneumonia more likely to die?  And if they do, was it because they were readmitted to a hospital?  Are those with pneumonia alone, versus those with pneumonia as part of a multiple diagnosis, less likely to be readmitted?  If so, are those admitted skewing "pneumonia data."  

We only explored the pneumonia comparison section so our review of the new site isn't complete, but it appears that there is granularity lacking in the reportage because there is little granularity or distinctions made in the precise information being gathered.  The questions above apparently aren't being asked.  It is my hope that one day they will, so real meaningful information can ultimately be gathered and valid comparisons made.  Until then, we would urge the CMS to hold off on self-promotion and the media like USA TODAY to hold off in general.  Either that, or learn how to use caveats liberally.