| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | 2 | |||||
| 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 10 | 11 | 12 | 13 | 14 | 15 | 16 |
| 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| 24 | 25 | 26 | 27 | 28 | 29 | 30 |
| 31 |
Authors' Committee
Chair:
Andy Eggers (Gov)
Members:
Weihua An (Soc)Kevin Bartz (Stats)
Sebastian Bauhoff (HealthPol)
John Graves (HealthPol)
Justin Grimmer (Gov)
Jens Hainmueller (Gov)
Mike Kellermann (Gov)
Ellie Powell (Gov)
Gary King (Gov)
Weekly Research Workshop Sponsors
Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Kevin Quinn, Jamie Robins, Don Rubin, Chris Winship
Recent Comments
- Re: data (non)sharing
By: Gary King - Re: data (non)sharing
By: Janet Rosenbaum - Re: data (non)sharing
By: Benjamin O. Fordham
Recent Entries
- Gans and Leigh on the "Baby Bump"
- Exxon-tainted research?
- Awards for IQSS faculty
- The Tree-Friendly Academic, Part II: The Editing Process, and Getting Off the Monitor
- The Tree-Friendly Academic: Whither A Useful Free PDF Editor?
- Nicholas and James are Featured in the NYT again
- Matching Portfolios
- Harvard Program on Survey Research (on Youtube)
- Placebo effects and the probability of assignment to active treatment
- IN, NC Rehash; WV Prediction
Categories
Blogroll
Brad DeLongCognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science
Archives
- August 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- July 2006
- June 2006
- May 2006
- April 2006
- March 2006
- February 2006
- January 2006
- December 2005
- November 2005
- October 2005
- September 2005
Notification
« Causation and Manipulation VI: The cognitive science version | Main | Further readings on the Iraqi excess deaths study »
11 October 2006
data (non)sharing
Jeremy Freese, an RWJ Health Policy Scholar at IQSS this year, sent me this amazing abstract (below) from the front lines of the replication movement, in psychology. On the same topic, but different discipline, don't miss Jeremy's "Reproducibility Standards in Quantitative Social Science: Why Not Sociology?" (find the pdf at his homepage) forthcoming, Sociological Methods and Research, July 2006. (I've written some on this topic too).
"The Poor Availability of Psychological Research Data for Reanalysis" By Wicherts, Jelte M.; Borsboom, Denny; Kats, Judith; Molenaar, Dylan American Psychologist. 61(7), Oct 2006, 726-728.
Abstract
The origin of the present comment lies in a failed attempt to obtain, through e-mailed requests, data reported in 141 empirical articles recently published by the American Psychological Association (APA). Our original aim was to reanalyze these data sets to assess the robustness of the research findings to outliers. We never got that far. In June 2005, we contacted the corresponding author of every article that appeared in the last two 2004 issues of four major APA journals. Because their articles had been published in APA journals, we were certain that all of the authors had signed the APA Certification of Compliance With APA Ethical Principles, which includes the principle on sharing data for reanalysis. Unfortunately, 6 months later, after writing more than 400 e-mails--and sending some corresponding authors detailed descriptions of our study aims, approvals of our ethical committee, signed assurances not to share data with others, and even our full resumes-we ended up with a meager 38 positive reactions and the actual data sets from 64 studies (25.7% of the total number of 249 data sets). This means that 73% of the authors did not share their data.
Posted by Gary King at October 11, 2006 7:38 PM
Comments
In spite of policies requiring replication datasets at many journals, I wonder how well we are really doing in political science. Graduate students in a methods class in my department at Binghamton University were required to replicate the analysis from a journal article. They have had a surprisingly difficult time obtaining data. Even authors in journals that require the posting of replication datasets were sometimes reluctant to help. In cases where journals maintain their own website for replication data, many articles are missing or are represented only by broken links. We're probably better off than Psychology in Freese's account, but not as much as we'd like to think.
Posted by: Benjamin O. Fordham at October 16, 2006 10:21 PM
In some cases, people say that they are reluctant to share data because it wasn't meant for dissemination and they don't have the resources to provide another researcher with documentation or technical support. That's a situation that I'm dealing with right now (literally) with a federal agency, and my inclination is to say that I'm sure that whatever they can give me would be just fine for my purposes.
Does anyone here have experience in persuading people to turn over their data anyhow? What happened?
Posted by: Janet Rosenbaum at October 18, 2006 2:50 PM
Hi Janet, if its a federal agency, you can file a Freedom of Information Act request. They then pretty much must turn over the data, subject to various rules. Its actually not that difficult to file these requests so you might consider it.
Posted by: Gary King at October 20, 2006 8:56 AM