Deanna Malatesta and I currently serve as Public Administration Review’s Transparency and Openness Promotion (TOP) editors. As part of the journal’s effort to draw on the experience of related fields and to add to the debate among the broader PAR community, we have started a collection of relevant materials, including scholarly articles and links to websites. We thank Jinsol Park for her assistance in setting up this initial collection.
1. General Guidance on Research Transparency and Openness
The links below include research transparency policies, guidelines, and articles with general guidance on research transparency in the social sciences.
The policy applies to the AEA’s journals, including the American Economic Review. As a policy, AEA journals publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication.
DA-RT is an organization to improve the credibility, legitimacy, and value of social science research through increasing transparency and openness. This site provides research transparency policies and guidelines by political science sournals and resources on data sharing and research transparency.
The American Political Science Association’s (APSA) Organized Section for Qualitative and Multi-Method Research sponsors the Qualitative Transparency Deliberations (QTD). It provides an inclusive process for deliberations over the meaning, costs, benefits, and practicalities of research transparency, openness and explicitness for a broad range of qualitative empirical approaches in Political Science.
This guide provides an authoritative statement of ethical principles for political scientists. For data access and production transparency, refer to pages 8-10.
Lupia, A., & Elman, C. (2014). Openness in Political Science: Data Access and Research Transparency: Introduction. PS: Political Science & Politics, 47(1), 19–42. https://doi.org/10.1017/S10490965130
This article discusses new policies for data access and research transparency developed by the American Political Science Association (APSA) Council.
This site holds papers related to the Qualitative Data Repository (QDR), published by QDR personnel.
Garret Christensen. (2016). Manual of Best Practices in Transparent Social Science Research. Retrieved from http://www.bitss.org/education/manual-of-best-practices/
This manual covers the entire process of research, from hypothesis generation to publication, for empirical social science researchers who desire to make their research transparent to others.
2. Journal articles about research transparency and openness
The list below includes publications about data/research transparency in quantitative and qualitative research as well as across multiple disciplines including political science, economics, psychology, and business.
Moravcsik, A. (2014). Transparency: The Revolution in Qualitative Research. American Political Science Association. https://doi.org/10.1017/S1049096513001789
This article discusses active citation, a tool for enhancing qualitative research transparency, and its benefit for improving openness in qualitative research in the social sciences.
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., … Van der Laan, M. (2014). Promoting Transparency in Social Science Research. Science, 343(6166), 30. https://doi.org/10.1126/science.1245317
The authors survey recent progress toward research transparency in the social sciences and make the case for standards and practices that help realign scholarly incentives with scholarly values.
Burlig, F. (2016). Improving Transparency in Observational Social Science Research: A Pre-Analysis Plan Approach. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2731920
The author proposes three scenarios in which study pre-registration can be credibly applied to non-experimental research to increase transparency practices in the social sciences.
Anderson, R. G., Greene, W. H., McCullough, B. D., & Vinod, H. D. (2008). The role of data/code archives in the future of economic research. Journal of Economic Methodology, 15(1), 99–119. https://doi.org/10.1080/13501780801915574
This essay examines the role of data and program‐code archives in making economic research ‘replicable.’
Dewald, W. G., Thursby, J. G., & Anderson, R. G. (1986). Replication in Empirical Economics: The Journal of Money, Credit and Banking Project. The American Economic Review, 76, 587–603. https://doi.org/10.2307/1806061
This paper provides information about the extent and causes of failure to replicate published results in economics. The findings suggest that inadvertent errors in published articles are a commonplace.
Christensen, G., & Miguel, E. (2016). Transparency, Reproducibility, and the Credibility of Economics Research. https://doi.org/10.3386/w22989
This article provides the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in economics and discusses recent progress in this area. It also discusses effective ways to make economics research more credible in the future.
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., … Wicherts, J. M. (2013). Recommendations for Increasing Replicability in Psychology. European Journal of Personality, 27(2), 108–119. https://doi.org/10.1002/per.1919
This article offers research practices and guidelines to increase replicability in psychology.
Gorgolewski, K. J., & Poldrack, R. A. (2016). A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research. PLoS Biology, 14(7), e1002506. https://doi.org/10.1371/journal.pbio.1002506
This article provides a practical guide for researchers in neuroscience, psychology, and other related fields that will help them make their research more reproducible and transparent while minimizing the additional effort.
LeBel, E. P., & John, L. K. (2017). Toward Transparent Reporting of Psychological Science. Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions, 73-84.
This study describes developments aimed at increasing transparency in psychological science and allied fields.
Antelman, K. (2004). Do Open-Access Articles Have a Greater Research Impact? College & Research Libraries, 65(5), 372–382. https://doi.org/10.5860/crl.65.5.372
The finding suggests that freely available articles do have a greater research impact in philosophy, political science, electrical and electronic engineering and mathematics.
Hubbard, R., & Vetter, D. E. (1996). An empirical comparison of published replication research in accounting, economics, finance, management, and marketing. Journal of Business Research, 35(2), 153–164. https://doi.org/10.1016/0148-2963(95)00084-4
The finding shows published replication and extension research is uncommon in the business disciplines. Strategies for cultivating a replication research tradition to facilitate knowledge development in the business disciplines are suggested.
Playford, C. J., Gayle, V., Connelly, R., & Gray, A. J. (2016). Administrative social science data: The challenge of reproducible research. Big Data & Society, 3(2), 205395171668414. https://doi.org/10.1177/2053951716684143
This article discusses the importance of sharing research code to enable research reproducible.
3. Other essays about research transparency and openness
Included below are a number of newsletters and reports regarding research transparency and its practice.
Büthe, T., Jacobs, A. M., Parkinson, S. E., & Wood, E. J. (2015). Qualitative & Multi-Method Research. Newsletter of the American Political Science Association Organized Section for Qualitative and Multi-Method Research, 13(1). Retrieved from https://scholar.harvard.edu/files/dtingley/files/qmmr2015-1.pdf
This newsletter presents a symposium on an issue of research transparency in qualitative and multi-method research.
Golder, Matt and Sona Golder (eds.) 2016. “Symposium: Data Access and Research Transparency (DA-RT).” CP: Newsletter of the Comparative Politics Organized Section of the American Political Science Association 26(1): 1-108.
This article provides a diverse set of positions with respect to DA-RT from contributors.
Mccullough, B. D., & Mckitrick, R. (2009). Check the Numbers: The Case for Due Diligence in Policy Formation. Retrieved from http://www.pages.drexel.edu/~bdm25/DueDiligence.pdf
This study discusses the required disclosure practice for replication, focusing on economics studies.