Research Reproducibility workshop: References

Unit 1 - Reproducibility for Everyone
Victoria Stodden, Friedrich Leisch, and Roger D. Peng, eds., Implementing Reproducible Research (Boca Raton: Chapman and Hall/CRC, 2018), https://osf.io/s9tya/.

National Academies, Reproducibility and Replicability in Science (Washington (DC): National Academies Press (US), 2019), http://www.ncbi.nlm.nih.gov/books/NBK547537/

Olivia Kowalczyk et al., “What Senior Academics Can Do to Support Reproducible and Open Research,” PsyArXiv, September 18, 2020, https://repub.eur.nl/pub/133263/

Roger D. Peng and Stephanie C. Hicks, “Reproducible Research: A Retrospective,” Annual Review of Public Health 42, no. 1 (April 1, 2021): 79–93, https://doi.org/10.1146/annurev-publhealth-012420-105110 

Broman, Karl, Mine Cetinkaya-Rundel, Amy Nussbaum, Christoper Paciorek, Roger Peng, Daniel Turek, and Hadley Wickham. January 18, 2017 2017. Recommendations to Funding Agencies for Supporting Reproducible Research. American Statistical Association. https://www.amstat.org/asa/files/pdfs/POL-ReproducibleResearchRecommendations.pdf.

Plesser, Hans E. 2018. "Reproducibility vs. Replicability: A Brief History of a Confused Terminology." Frontiers in neuroinformatics 11: 76-76. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5778115/.

Goodman, S. N., D. Fanelli, and J. P. Ioannidis. 2016. "What does research reproducibility mean?" Sci Transl Med 8 (341): 341ps12. https://doi.org/10.1126/scitranslmed.aaf5027.

Unit 2 - Impacts of Publishers and Funders

Ahmed Elgazzar, Abdelaziz Eltaweel, Shaimaa Abo Youssef et al. Efficacy and Safety of Ivermectin for Treatment and prophylaxis of COVID-19 Pandemic, 28 December 2020, PREPRINT (Version 3) available at Research Square https://doi.org/10.21203/rs.3.rs-100956/v3 

Berghmans S et al (2017) Open Data: the researcher perspective – survey and case studies. https://data.mendeley.com/datasets/bwrnfb4bvh/1

Bishop, Dorothy, “Improving Reproducibility: What Can Funders Do? Guest Post by Dorothy Bishop,” Retraction Watch (blog), November 24, 2015, https://retractionwatch.com/2015/11/24/improving-reproducibility-what-can-funders-do-guest-post-by-dorothy-bishop/.

BMJ, “Data Sharing - BMJ Author Hub,” accessed September 22, 2021, https://authors.bmj.com/policies/data-sharing/.

Borgman, C. L. (2012), The conundrum of sharing research data. J Am Soc Inf Sci Tec, 63: 1059–1078. doi:10.1002/asi.22634

Brown, Nick, “Some Problems in the Dataset of a Large Study of Ivermectin for the Treatment of Covid-19,” Nick Brown’s Blog (blog), July 2021. https://steamtraen.blogspot.com/2021/07/Some-problems-with-the-data-from-a-Covid-study.html.

Colavizza et al (2019) The citation advantage of linking publications to research data. https://arxiv.org/abs/1907.02565

Conover, Adam, “Professor Brian Nosek on Science’s Reproducibility Crisis and Opportunity,” Adam Ruins Everything (2017), https://maximumfun.org/episodes/adam-ruins-everything/adam-ruins-everything-episode-38-professor-brian-nosek-sciences-reproducibilit/.

data.mendeley.com/datasets/bwrnfb4bvh/1

Digital Science (2017) The state of open data report 2017. Digital Science, London

Dryad, “Joint Data Archiving Policy” (2020). https://datadryad.org/docs/JointDataArchivingPolicy.pdf

F1000, “Article Policies” https://f1000research.com/about/policies#dataavail

F1000, “Submit your Research,” accessed September 22, 2021, https://f1000research.com/for-authors/publish-your-research.

Fang FC, Steen RG, Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109:17028–17033. https://doi.org/10.1073/pnas.1212247109

Federer, Lisa. “Journal Data Sharing Policies Are Moving the Scientific Community towards Greater Openness but Clearly More Work Remains,” Impact of Social Sciences (blog), June 14, 2018, https://blogs.lse.ac.uk/impactofsocialsciences/2018/06/14/journal-data-sharing-policies-are-moving-the-scientific-community-towards-greater-openness-but-clearly-more-work-remains/.

Federer et al., “Data Sharing in PLOS ONE: An Analysis of Data Availability Statements,” PLOS ONE 13, no. 5 (May 2, 2018): e0194768, https://doi.org/10.1371/journal.pone.0194768.

Garza K, Fenner M (2018) Glad you asked: a snapshot of the current state of data citation. https://blog.datacite.org/citation-analysis-scholix-rda/

Giofrè D, Cumming G, Fresc L, Boedker I, Tressoldi P (2017) The influence of journal submission guidelines on authors’ reporting of statistics and use of open research practices. PLoS One 12:e0175583

Goodman S. N., Fanelli D., Ioannidis J. P. A. (2016). What does research reproducibility mean? Sci. Transl. Med. 8:341ps12. 10.1126/scitranslmed.aaf5027

Greenwald, A. (1975). Consequences of Prejudice Against the Null Hypothesis. Psychological Bulletin, 82, 1-20. https://www.semanticscholar.org/paper/Consequences-of-Prejudice-Against-the-Null-Greenwald/1b291af66d9b9f61271dd842558ea508440bb22d

Grieneisen ML, Zhang M (2012) A comprehensive survey of retracted articles from the scholarly literature. PLoS One 7:e44118. https://doi.org/10.1371/journal.pone.0044118

Hardwicke TE et al (2018) Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition. R Soc Open Sci 5:180448

Hrynaszkiewicz, Iain. “Publishers’ Responsibilities in Promoting Data Quality and Reproducibility,” in Good Research Practice in Non-Clinical Pharmacology and Biomedicine, ed. Anton Bespalov, Martin C. Michel, and Thomas Steckler (Cham: Springer International Publishing, 2020), 319–48, https://doi.org/10.1007/164_2019_290.

JISC, “Funding Agreed to Develop Platform That Will Change Research Culture,” Jisc, August 2021. Accessed September 22, 2021, https://www.jisc.ac.uk/news/funding-agreed-to-develop-platform-that-will-change-research-culture-06-aug-2021.

Kidwell MC et al (2016) Badges to acknowledge open practices: a simple, low-cost, effectivemethod for increasing transparency. PLoS Biol 14:e1002456. https://doi.org/10.1371/journal.pbio.1002456

Liron Ganel et al., “Mitochondrial Genome Copy Number Measured by DNA Sequencing in Human Blood Is Strongly Associated with Metabolic Traits via Cell-Type Composition Differences,” Human Genomics 15, no. 1 (June 7, 2021): 34, https://doi.org/10.1186/s40246-021-00335-2.

McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, et al. (2012) Adaptive Pacing, Cognitive Behaviour Therapy, Graded Exercise, and Specialist Medical Care for Chronic Fatigue Syndrome: A Cost-Effectiveness Analysis. PLOS ONE 7(8): e40808. https://doi.org/10.1371/journal.pone.0040808Author Shannon Palus, “PLOS ONE Issues Retraction Watch, Editor’s Note over Controversial Chronic Fatigue Syndrome Research,” Retraction Watch (blog), December 16, 2015, https://retractionwatch.com/2015/12/16/plos-one-issues-editors-note-over-disputed-chronic-fatigue-research/.

Nature, “Code & Software Submission checklist”. https://www.nature.com/documents/nr-software-policy.pdf (2021)

Nature, “Extending Transparency to Code,” Nature Neuroscience 20, no. 6 (June 2017): 761–761, https://doi.org/10.1038/nn.4579.

Nuijten MB et al (2017) Journal data sharing policies and statistical reporting inconsistencies in psychology. Collabra Psychol 3:31

Piwowar HA, Day RS, Fridsma DB (2007) Sharing Detailed Research Data Is Associated with Increased Citation Rate. PLoS ONE2(3): e308. https://doi.org/10.1371/journal.pone.0000308

Rathi V et al (2012) Clinical trial data sharing among trialists: a cross-sectional survey. BMJ 345:e7570

Reinhart, Alex. “Pseudoreplication: Choose Your Data Wisely.” Statistics Done Wrong, (2015). https://www.statisticsdonewrong.com/pseudoreplication.html.

Roche DG, Kruuk LEB, Lanfear R, Binning SA (2015) Public Data Archiving in Ecology and Evolution: How Well Are We Doing? PLoS Biol13(11): e1002295. https://doi.org/10.1371/journal.pbio.1002295

Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015). Public data archiving in ecology and evolution: How well are we doing? PLoS Biology, 13(11) doi:http://dx.doi.org/10.1371/journal.pbio.1002295 

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638

Rowhani-Farid A, Allen M, Barnett AG (2017) What incentives increase data sharing in health and medical research? A systematic review. Res Integr Peer Rev 2:4

Savage CJ, Vickers AJ (2009) Empirical study of data sharing by authors publishing in PLoS journals. PLoS One 4:e7078

Scientific Data, “Editorial & Publishing Policies | Scientific Data,” accessed September 22, 2021, https://www.nature.com/sdata/policies/editorial-and-publishing-policies.

Stanford University, Stanford, and California 94305, “Data Management Plans,” Stanford Libraries, accessed September 22, 2021, https://library.stanford.edu/research/data-management-services/data-management-plans.

Stuart D et al (2018) Whitepaper: practical challenges for researchers in data sharing. https://figshare.com/articles/Whitepaper_Practical_challenges_for_researchers_in_data_sharing/5975011

Tedds, Jonathan. Distinguished Lecture at DLab, UC Berkeley, 12 Sep 2013: "The Open Research Challenge: Peer Review and Publication of Research Data“ Licensed under CC BY.

Vanpaemel W, Vermorgen M, Deriemaecker L, Storms G (2015) Are we wasting a good crisis? The availability of psychological research data after the storm. Collabra 1

Vasilevsky NA, Minnier J, Haendel MA, Champieux RE (2017) Reproducible and reusable research: are journal data sharing policies meeting the mark? PeerJ 5:e3208

Vines TH et al (2013) Mandated data archiving greatly improves access to research data. FASEB J27:1304–1308

Vines, Timothy H, Arianne Y K. Albert, Rose L Andrew, Florence Débarre, Dan G Bock, Michelle T Franklin, Kimberly J Gilbert, et al. "The Availability of Research Data Declines Rapidly with Article Age." Current Biology 24, no. 1 (1/6/ 2014): 94-97. https://linkinghub.elsevier.com/retrieve/pii/S0960-9822(13)01400-0

Wicherts JM, Borsboom D, Kats J, Molenaar D (2006) The poor availability of psychological research data for reanalysis. Am Psychol 61:726–728

Wiley Open Science Researcher Survey (2016) https://figshare.com/articles/Wiley_Open_Science_Researcher_Survey_2016/4748332/2

Wudan Yu. “Coronavirus Tests Science’s Need for Speed Limits.” New York Times, April 14, 2020. https://www.nytimes.com/2020/04/14/science/coronavirus-disinformation.html