I believe public management research is better and more useful if we publish not only key findings but also share the underlying data and materials, as long as doing so is ethical and feasible. Such transparency helps others review what we did and build on it. In that spirit, Deanna Malatesta and I wrote Public Administration Review’s transparency guidelines.
Here is a collection of relevant materials, including scholarly articles and links to websites. I thank Jinsol Park for her assistance in setting up this collection.
1. General Guidance on Research Transparency and Openness
The links below include research transparency policies, guidelines, and articles with general guidance on research transparency in the social sciences.
American Economic Association (AEA): Data Availability Policy
The policy applies to the AEA’s journals, including the American Economic Review. As a policy, AEA journals publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication.
Data Access and Research Transparency (DA-RT)
DA-RT is an organization to improve the credibility, legitimacy, and value of social science research through increasing transparency and openness. This site provides research transparency policies and guidelines by political science journals and resources on data sharing and research transparency.
Qualitative Transparency Deliberations
The American Political Science Association’s (APSA) Organized Section for Qualitative and Multi-Method Research sponsored the Qualitative Transparency Deliberations (QTD). It provided an inclusive process for deliberations over the meaning, costs, benefits, and practicalities of research transparency, openness and explicitness for a broad range of qualitative empirical approaches in Political Science.
APSA Guide to Professional Ethics in Political Science
This guide provides an authoritative statement of ethical principles for political scientists. For data access and production transparency, refer to pages 8-10.
Lupia, A., & Elman, C. (2014). Openness in Political Science: Data Access and Research Transparency: Introduction. PS: Political Science & Politics, 47(1), 19–42. https://doi.org/10.1017/S1049096513001716
This article discusses new policies for data access and research transparency developed by the American Political Science Association (APSA) Council.
Qualitative Data Repository (QDR) – Publications
This site holds papers related to the Qualitative Data Repository (QDR), published by QDR personnel.
Garret Christensen. (2016). Manual of Best Practices in Transparent Social Science Research. Retrieved from http://www.bitss.org/education/manual-of-best-practices/
This manual covers the entire process of research, from hypothesis generation to publication, for empirical social science researchers who desire to make their research transparent to others.
2. Journal articles about research transparency and openness
The list below includes publications about data/research transparency in quantitative and qualitative research as well as across multiple disciplines including political science, economics, psychology, and business.
Moravcsik, A. (2014). Transparency: The Revolution in Qualitative Research. American Political Science Association. PS: Political Science & Politics 47 (1): 48-53.
https://doi.org/10.1017/S1049096513001789
This article discusses active citation, a tool for enhancing qualitative research transparency, and its benefit for improving openness in qualitative research in the social sciences.
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., … Van der Laan, M. (2014). Promoting Transparency in Social Science Research. Science, 343(6166), 30. https://doi.org/10.1126/science.1245317
The authors survey recent progress toward research transparency in the social sciences and make the case for standards and practices that help realign scholarly incentives with scholarly values.
Burlig, F. (2016). Improving Transparency in Observational Social Science Research: A Pre-Analysis Plan Approach. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2731920
The author proposes three scenarios in which study pre-registration can be credibly applied to non-experimental research to increase transparency practices in the social sciences.
Anderson, R. G., Greene, W. H., McCullough, B. D., & Vinod, H. D. (2008). The role of data/code archives in the future of economic research. Journal of Economic Methodology, 15(1), 99–119.
https://doi.org/10.1080/13501780801915574
This essay examines the role of data and program‐code archives in making economic research ‘replicable.’
Dewald, W. G., Thursby, J. G., & Anderson, R. G. (1986). Replication in Empirical Economics: The Journal of Money, Credit and Banking Project. The American Economic Review, 76, 587–603. https://www.jstor.org/stable/1806061
This paper provides information about the extent and causes of failure to replicate published results in economics. The findings suggest that inadvertent errors in published articles are commonplace.
Christensen, G., & Miguel, E. (2016). Transparency, Reproducibility, and the Credibility of Economics Research. https://doi.org/10.3386/w22989
This article provides the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in economics and discusses recent progress in this area. It also discusses effective ways to make economics research more credible in the future.
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., … Wicherts, J. M. (2013). Recommendations for Increasing Replicability in Psychology. European Journal of Personality, 27(2), 108–119. https://doi.org/10.1002/per.1919
This article offers research practices and guidelines to increase replicability in psychology.
Gorgolewski, K. J., & Poldrack, R. A. (2016). A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research. PLoS Biology, 14(7), e1002506. https://doi.org/10.1371/journal.pbio.1002506
This article provides a practical guide for researchers in neuroscience, psychology, and other related fields that will help them make their research more reproducible and transparent while minimizing the additional effort.
LeBel, E. P., & John, L. K. (2017). Toward Transparent Reporting of Psychological Science. Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions, 73-84.
Google Books preview
This study describes developments aimed at increasing transparency in psychological science and allied fields.
Antelman, K. (2004). Do Open-Access Articles Have a Greater Research Impact? College & Research Libraries, 65(5), 372–382. https://doi.org/10.5860/crl.65.5.372
The finding suggests that freely available articles do have a greater research impact in philosophy, political science, electrical and electronic engineering and mathematics.
Hubbard, R., & Vetter, D. E. (1996). An empirical comparison of published replication research in accounting, economics, finance, management, and marketing. Journal of Business Research, 35(2), 153–164. https://doi.org/10.1016/0148-2963(95)00084-4
The finding shows published replication and extension research is uncommon in the business disciplines. Strategies for cultivating a replication research tradition to facilitate knowledge development in the business disciplines are suggested.
Playford, C. J., Gayle, V., Connelly, R., & Gray, A. J. (2016). Administrative social science data: The challenge of reproducible research. Big Data & Society, 3(2), 205395171668414. https://doi.org/10.1177/2053951716684143
This article discusses the importance of sharing research code to enable research reproducible.
3. Other essays about research transparency and openness
The Atlantic (2018): “The Scientific Paper Is Obsolete: Here’s what’s next.”
A fascinating look into the potential of dynamic notebooks to become the standard for reporting research, instead of static PDFs. Enthusiastic commentary by Paul Romer
Why Jupyter is data scientists’ computational notebook of choice
Essay in Nature highlighting the use of JupyterLab for transparent, reproducible research
Büthe, T., Jacobs, A. M., Parkinson, S. E., & Wood, E. J. (2015). Qualitative & Multi-Method Research. Newsletter of the American Political Science Association Organized Section for Qualitative and Multi-Method Research, 13(1). Retrieved from https://scholar.harvard.edu/files/dtingley/files/qmmr2015-1.pdf
This newsletter presents a symposium on an issue of research transparency in qualitative and multi-method research.
Mccullough, B. D., & Mckitrick, R. (2009). Check the Numbers: The Case for Due Diligence in Policy Formation. Retrieved from http://www.pages.drexel.edu/~bdm25/DueDiligence.pdf
This study discusses the required disclosure practice for replication, focusing on economics studies.
nickpetrovsky.com 