While perusing the National Center for Education Statistics' website, I came across this study. Using data from the National Education Longitudinal Study : 1988, the researchers analyzed trends of voluntarism and community service among a cohort of young people. The researchers took the initial cohort from 1988 and interviewed them again in 1992 (when the young people were in 12th grade) then again in 1994 and 2000 in an attempt to gain some insight into patterns of service 2 years and 8 years after high school graduation.
The researchers present their findings as general conclusions, some of which are what I would expect:
* females were found to volunteer more in high school than males and;
* students from high socio-economic status families were more likely to volunteer than those from middle to low socio-economic status.
Some of the other conclusions, particularly those relating to how mandatory or incentive-based volunteering in high school relates to the incidence and persistence of volunteering and civic activity after high school are a little more difficult to unpack. Those conclusions are interesting, but I found myself drawn to a couple of seemingly innocuous footnotes.
The first footnote is an answer about what their cohort is doing 2 and 8 years after high school graduation. It reads:
In 1994, 56 percent of the 1992 senior cohort reported being enrolled in at least one academic course in a 2- or 4- year college, and 63 percent reported being employed in either full- or part-time jobs. Eighty-eight percent reported being involved in at least one of these activities.
So where's the other 12 percent? Are they missing? Incarcerated? In the army? Part of the answer can be found in the appendix to the study where the researchers discuss their statistical methods. Individuals that were identified as dropouts in the second follow up interview (1992) were dropped from the study entirely. The same paragraph notes that some individuals were dropped from the sample because of "emigration, death, and other reasons".
The authors are forthcoming about the limitations of their study, due in part to a small sample size. They note that their conclusions are still useful because they identify general trends that may be of interest to other researchers, particularly those researchers interested in the impact of mandatory voluntarism through schools on motivations to serve in later life.
It would seem to me, however, that those "missing members" may also hold some answers to questions about the limitations of current models of youth service programs.
3 comments:
That's a good point about the difficulty of these kind of statistical "cohort" evaluations over time-what if people just stop answering your questions? I think about all those times I was given surveys or teacher evaluations to fill out and may or may not have given them the attention whoever was looking at them assumed I was. I would be interested if there are good methods for avoiding this problem. It's almost like a form of self-selection-if you are interested or motivated, you might answer the questions in a more thoughtful or a more extreme manner than you otherwise would.
Good catch...evaluating statistics is half the battle, and evaluating the evaluation is an even harder task. When I have difficulty in understanding or even reading statistics in a journal article or text, does this take away credibility of the author? I thought the point of statistics is to add validity to work.
Keep on digging through the footnotes! It is important for the group to know that some of these studies we find may be garbage. At some point, I am sure that we will find two different articles that contradict each other. We need to be able to evaluate the validity of the statistical methods and choose the better study. Good work. That being said, statistics is all an estimation. Some information is better than no information, so even a study with questionable methods may give us a better idea than no study at all.
Post a Comment