04 December 2007

Kudos to CBS!

CBS updated their methodology webpage concerning their report on veteran suicides. I found the updated page via a post on their blog, Primary Sources.

I have to admit that I'm practically giddy that CBS responded this way:

Since CBS News first aired our veteran suicide report on November 13, we have received several questions from viewers wanting more information. In order to provide as much information as possible and to fully answer all questions, this is a summary of the methodology and results of the data we presented.
The updated page is much, much better than the original one!

I had criticized CBS for not making the data publicly available. Here's their explanation:
Privacy Issues
Obtaining suicide data from the states involved more than just a basic public records request.

Initially, several states refused to provide the data to CBS News for privacy concerns. Here’s why: the suicide numbers in some categories are small enough that individuals could be identified, violating state privacy policies. For example, one state could have two non-white females between the ages of 30-34 committing suicide in 2004 who had served in the armed forces. Because of the small number in that category, those individuals could be identified and the cause of their death would then be made public.

Therefore, in order to get the data, CBS News had to give assurance to the states that we would keep the raw data confidential. Some states insisted upon written agreements to this effect. The data, however, can be obtained upon request from the files of each individual state. [my emphasis]
That's a pretty good explanation and I understand now why CBS would not provide the state-by-state cross-tabulation data. [UPDATE: It does not, however, get them off the hook from providing any data - see table below.] As a bonus for anyone wanting to recreate the data directly from the states, CBS also provides their cost ($3000) to reimburse the states for processing fees.

CBS provides this additional note:
Important Note: Suicide death rates that are publically available (by going to the CDC’s Wisquar’s website) are for the general U.S. population. The general population includes both veterans and non-veterans together. The rates CBS News presented will look nothing like those for the general population because we compared “veterans” to “non-veterans.” Comparing veteran suicides to the general population is misleading and an inaccurate analysis because, the general population includes veterans. Furthermore, the CDC general population suicide rates are age-adjusted only. The CBS analysis is adjusted for both age and gender. [my emphasis]
I agree with CBS that comparing only the veteran rates with the CDC general population rates is incorrect, for the reason they state: the general population includes veterans.

I emphasized the last two sentences because they directly apply to my basic analysis of their 2004 results. Earlier in their methodology, CBS says this:
The veteran population, for example, is mostly made up of older males, so the data had to be statistically adjusted state by state in order to accurately compare with the non-veteran populations.

Important Note: All of the rates of suicide that CBS News presented are adjusted. The overall rates are adjusted for age and gender in both the veteran and non-veteran populations. The male and female rates are age adjusted. And, the age breakdowns are adjusted for gender.
The question for me is: Does the explanation that the CBS data is gender adjusted account for the high female veteran and non-veteran populations in the overall 2004 rate, and the very high 20-24 age group veteran population?

The answer is ... maybe? To be sure, I would have to come up with a way to reverse engineer adjusting for age and gender on a state-by-state basis without the crude (raw) suicide data or state cohort population numbers.

I have to admit that given the updated methodology, I'm much more willing to trust CBS on this.

My previous questions to CBS based on the overall rate and 20-24 age group were:
  • Why would female non-veterans be represented at twice the rate of male non-veterans? Why wouldn't CBS tell us this or at least explain it?

  • Why is the veteran female population also twice the male veteran population?

  • How does CBS explain a 20-24 age group veteran population 10 to 20 times the Census/VA 1.5% population figure, if we use the CDC suicide rate as a substitute for their missing data?

  • Why would the veteran population in this 20-24 age group be more than two to three times the overall veteran population of 8.5% in the general population?

  • Why is the 20-24 veteran population a higher percentage than even the 12.1-19.5% veteran population calculated when using CBS and CDC data?
CBS also tells us that the male and female rates are age adjusted, which is comparable to the CDC data. My previous questions on these results were:
  • Why are both the male non-veteran rate and male veteran rate reported by CBS higher than the CDC rate for males in the general population?

  • Why are both the female non-veteran rate and female veteran rate reported by CBS higher than the CDC rate for females in the general population?
I think these questions are still valid. Did CBS limit the age range to 20+ years? Would that account for the higher rates?

It might help if CBS could (please?) provide a table, stratified by 5-year age group, with male and female columns for veterans and non-veterans. Something like this:

A table for the 2004 data and a table for the 2005 data would be very useful to calculate age and gender adjusted rates based on Veterans Administration and Census Bureau data.

Anyway, bottom line is this is an awesome response by CBS! Kudos!

Questions for Keteyian
Astounding Arrogance at CBS
Seems right to me
Sensationalizing Suicide II
Sensationalizing Suicide

No comments: