PAR TOP resources

Deanna Malatesta and I currently serve as Public Administration Review (PAR)’s Transparency and Openness Promotion (TOP) editors.  We wrote PAR’s transparency guidelines.

As part of the journal’s effort to draw on the experience of related fields and to add to the debate among the broader PAR community, here is a collection of relevant materials, including scholarly articles and links to websites.  We thank Jinsol Park for her assistance in setting up this collection.

 

1. General Guidance on Research Transparency and Openness

The links below include research transparency policies, guidelines, and articles with general guidance on research transparency in the social sciences.

 

American Economic Association (AEA): Data Availability Policy

The policy applies to the AEA’s journals, including the American Economic Review.  As a policy, AEA journals publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication.

 

Data Access and Research Transparency (DA-RT)

DA-RT is an organization to improve the credibility, legitimacy, and value of social science research through increasing transparency and openness. This site provides research transparency policies and guidelines by political science sournals and resources on data sharing and research transparency.

 

Qualitative Transparency Deliberations

The American Political Science Association’s (APSA) Organized Section for Qualitative and Multi-Method Research sponsored the Qualitative Transparency Deliberations (QTD). It provided an inclusive process for deliberations over the meaning, costs, benefits, and practicalities of research transparency, openness and explicitness for a broad range of qualitative empirical approaches in Political Science.

 

APSA Guide to Professional Ethics in Political Science

This guide provides an authoritative statement of ethical principles for political scientists. For data access and production transparency, refer to pages 8-10.

 

Lupia, A., & Elman, C. (2014). Openness in Political Science: Data Access and Research Transparency: Introduction. PS: Political Science & Politics, 47(1), 19–42. https://doi.org/10.1017/S1049096513001716

This article discusses new policies for data access and research transparency developed by the American Political Science Association (APSA) Council.

 

Qualitative Data Repository (QDR) – Publications

This site holds papers related to the Qualitative Data Repository (QDR), published by QDR personnel.

 

Garret Christensen. (2016). Manual of Best Practices in Transparent Social Science Research. Retrieved from http://www.bitss.org/education/manual-of-best-practices/

This manual covers the entire process of research, from hypothesis generation to publication, for empirical social science researchers who desire to make their research transparent to others.

 

2. Journal articles about research transparency and openness

The list below includes publications about data/research transparency in quantitative and qualitative research as well as across multiple disciplines including political science, economics, psychology, and business.

 

Moravcsik, A. (2014). Transparency: The Revolution in Qualitative Research. American Political Science Association. PS: Political Science & Politics 47 (1): 48-53.
https://doi.org/10.1017/S1049096513001789

This article discusses active citation, a tool for enhancing qualitative research transparency, and its benefit for improving openness in qualitative research in the social sciences.

 

Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., … Van der Laan, M. (2014). Promoting Transparency in Social Science Research. Science, 343(6166), 30. https://doi.org/10.1126/science.1245317

The authors survey recent progress toward research transparency in the social sciences and make the case for standards and practices that help realign scholarly incentives with scholarly values.

 

Burlig, F. (2016). Improving Transparency in Observational Social Science Research: A Pre-Analysis Plan Approach. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2731920

The author proposes three scenarios in which study pre-registration can be credibly applied to non-experimental research to increase transparency practices in the social sciences.

 

Anderson, R. G., Greene, W. H., McCullough, B. D., & Vinod, H. D. (2008). The role of data/code archives in the future of economic research. Journal of Economic Methodology, 15(1), 99–119.
https://doi.org/10.1080/13501780801915574

This essay examines the role of data and program‐code archives in making economic research ‘replicable.’

 

Dewald, W. G., Thursby, J. G., & Anderson, R. G. (1986). Replication in Empirical Economics: The Journal of Money, Credit and Banking Project. The American Economic Review, 76, 587–603. https://www.jstor.org/stable/1806061

This paper provides information about the extent and causes of failure to replicate published results in economics. The findings suggest that inadvertent errors in published articles are commonplace.

 

Christensen, G., & Miguel, E. (2016). Transparency, Reproducibility, and the Credibility of Economics Research. https://doi.org/10.3386/w22989

This article provides the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in economics and discusses recent progress in this area. It also discusses effective ways to make economics research more credible in the future.

 

Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., … Wicherts, J. M. (2013). Recommendations for Increasing Replicability in Psychology. European Journal of Personality, 27(2), 108–119. https://doi.org/10.1002/per.1919

This article offers research practices and guidelines to increase replicability in psychology.

 

Gorgolewski, K. J., & Poldrack, R. A. (2016). A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research. PLoS Biology, 14(7), e1002506. https://doi.org/10.1371/journal.pbio.1002506

This article provides a practical guide for researchers in neuroscience, psychology, and other related fields that will help them make their research more reproducible and transparent while minimizing the additional effort.

 

LeBel, E. P., & John, L. K. (2017). Toward Transparent Reporting of Psychological Science. Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions, 73-84.
Google Books preview

This study describes developments aimed at increasing transparency in psychological science and allied fields.

 

Antelman, K. (2004). Do Open-Access Articles Have a Greater Research Impact? College & Research Libraries, 65(5), 372–382. https://doi.org/10.5860/crl.65.5.372

The finding suggests that freely available articles do have a greater research impact in philosophy, political science, electrical and electronic engineering and mathematics.

 

Hubbard, R., & Vetter, D. E. (1996). An empirical comparison of published replication research in accounting, economics, finance, management, and marketing. Journal of Business Research, 35(2), 153–164. https://doi.org/10.1016/0148-2963(95)00084-4

The finding shows published replication and extension research is uncommon in the business disciplines. Strategies for cultivating a replication research tradition to facilitate knowledge development in the business disciplines are suggested.

 

Playford, C. J., Gayle, V., Connelly, R., & Gray, A. J. (2016). Administrative social science data: The challenge of reproducible research. Big Data & Society, 3(2), 205395171668414. https://doi.org/10.1177/2053951716684143

This article discusses the importance of sharing research code to enable research reproducible.

 

3. Other essays about research transparency and openness

The Atlantic (2018): “The Scientific Paper Is Obsolete: Here’s what’s next.”

A fascinating look into the potential of dynamic notebooks to become the standard for reporting research, instead of static PDFs.  Enthusiastic commentary by Paul Romer

 

Why Jupyter is data scientists’ computational notebook of choice

Essay in Nature highlighting the use of JupyterLab for transparent, reproducible research

 

Büthe, T., Jacobs, A. M., Parkinson, S. E., & Wood, E. J. (2015). Qualitative & Multi-Method Research. Newsletter of the American Political Science Association Organized Section for Qualitative and Multi-Method Research, 13(1). Retrieved from https://scholar.harvard.edu/files/dtingley/files/qmmr2015-1.pdf

This newsletter presents a symposium on an issue of research transparency in qualitative and multi-method research.

 

Mccullough, B. D., & Mckitrick, R. (2009). Check the Numbers: The Case for Due Diligence in Policy Formation. Retrieved from http://www.pages.drexel.edu/~bdm25/DueDiligence.pdf

This study discusses the required disclosure practice for replication, focusing on economics studies.