Open Science

Tools and Resources for Open Science 

General Resources for Open Science

Preregistration

Transparent Processes and Research Design

Project TIER – Teaching integrity in empirical researchEQUATOR – Enhancing quality in health research
Qualitative Transparency Deliberations – Musings on transparency in qualitative researchConsolidated Criteria for Reporting Qualitative Research (COREQ) - Reporting guidelines for interviews and focus groupsPRISMA - Reporting guidelines for systematic reviews and meta-analysesPRISMA 2020 - R package and Shiny app for creating PRISMA flow diagramsEDA diagrams - Experimental research design assistant

Data Management

FAIR sharing – Data and metadata standards, inter-related to databases and data policiesDMP tool - Create your data management plan hereMANTRA – Free course on data managementDataWiz – Automated assistant for data managementManaging Qualitative Data (interactive course)
Example guidelines for anonymizing data

Analytical Tools

The Reproducible Open Coding Kit (ROCK) – R package for qualitative researchersInterface for the Reproducible Open Coding Kit (iROCK) – GDPR-compliant, browser-based qualitative coding toolEpistemic Network Analysis – Network models (mainly for quantified narratives)nCoder – Develop, validate, and implement automated classifiersCATMA – Tool for qualitative annotation and various analysesTaguette – Text tagging and highlightingJamovi – Easy to use tool for statistical analysesOrange – Machine learning and visualizationKH Coder – Qualitative coding, content analysis, and text miningText Analyzer – Simple content analysis (frequency and lexical density)Charticulator – Data visualizationELAN – Annotation tool for audio and video recordingsCohen's Kappa calculators - first, secondRho calculator
DOCA - Database of Variables for Content Analysis
ASReview - Open source software for systematic reviews

Data Storage and Sharing

Open Science Framework (unlimited)Gitlab (10GB/repo; unlimited repos)Github (1GB/repo; unlimited repos)FigShare (5GB)UK Data ServiceEuropean Open Science Cloud (EOSC)The Qualitative Data RepositoryRe3data – Registry of research data repositoriesRegistry of Open Access Repositories – Exactly what it sounds likeZotero – Citation manager, shareable librariesJabRef - Cross-platform citation and reference management softwareFAIRsharing - Open databases and standardsCOVID19-specific repositories (quant)

Preprint Servers

Open Access Publications

Core – Open access papersInternet Archive – Non-profit open libraryCrossref – Metadata of publications

Licensing

Other awesome things to help good quality research, dissemination

Asana – For managing group workGather town – Communication Platform; free for up to 25 users
Miro - Collaborative multimedia whiteboards (for sync or async work)OBS Studio – Screen capture; Open broadcaster softwareKialoEdu – For asynchronous, structured debates and discussionsWorld Health Organization – Informed consent formsCOS Reproducible Research Training + IRB and Consent form examplesOpen (qualitative) data consent supplement for informed consent forms (draft)European Charter & Code for Researchers – General ethics considerationsInkScape – Free and open source vector graphics editor; tutorials
DreamPuf - Synchronous mind mapping with code
Figma - Free vector graphics editor
Otter.ai – Transcription toolKaggle – Machine learning and data science communityGuidelines on picking colors for visualizationsCRediT – Contributor roles taxonomyRegular expression tutorialRStudio cheatsheets

Download SQAFFOLD

Simple Qualitative Administration For File Organization, Licensing, and Development
SQAFFOLD is a directory system with tools and resources for qualitative and unified research projects to aid organization and making project materials public.

Some thoughts on Open Science in qualitative and quantitative ethnographic research


Many people think Open Science = Open Data, and precisely because of this reason, it is less feasible in qualitative research

I've heard colleagues say: Open Science (OS) principles don't apply to qualitative research. What exactly would we make public? Our data can be anonymized to an extent (e.g. proper nouns omitted or replaced with pseudonyms), but can it really, truly be anonymized so that the interviewee is unrecognizable, even to their own social environment? Sometimes we analyze life stories, personal experiences, sensitive topics...how is it possible to anonymize these narratives? And wouldn't we be omitting exactly the things that qualitative researchers are so keen on scrutinizing? Supposing we could anonymize texts to a full extent (beyond proper nouns), would that even be useful to other researchers?

These are incredibly important questions and things to discuss more widely among qualitative and quantitative ethnographic researchers. I'm not going to address open qualitative data here, it's far too complex. I want to talk a bit about everything else. OS is much more than "just" making your data publicly accessible. It is about being open concerning the entire research process, well, as open as possible. OS is a spectrum; it is not a dichotomy.

Qualitative researchers might ask: Well, what else IS there to share besides data? There's a lot you can make public that other researchers can benefit from! Without attempting to compile ALL aspects of qual research that can be made open, below are some ideas on what you CAN share with others; and although each researcher should consider the implications for their own project, these probably don't entail any legal, ethical, or moral complications.


So, what can be made public in a qualitative research project?

- Conceptualization (key terms and their descriptions/definitions, theoretical framework, etc.)
- Research design/Operationalization (research questions and type of measurement, sampling strategy, sample size planning, recruitment, etc.)
- Process of data collection (tools: interview structure, focus group discussion guide, etc., stopping criteria, type of data collected)
- Codes (code development, different versions of code structure and round/s of triangulation, final codebook, etc.)
- Segmentation and data transformation (if qual data was segmented what considerations were behind that, how was it achieved, etc.)
- Coding process (changes in codes while coding: merges, splits, additions, etc., software used, number of raters, inter-rater reliability measures, etc.)
- Analysis (analytical approach, analytical tools, analysis scripts, etc.)
- Credibility strategies (respondent validation, peer debriefing, etc.)
- If you are using Epistemic Network Analysis: model parameters: unit, conversation, stanza window, SVD/means rotation, edge weights, etc.

...And, of course, not merely stating the decisions you have made in your process, but also the justifications and reasoning behind these decisions. Making these aspects of qualitative research initiatives public not only facilitates the transparency, reproducibility, and interoperability of a project, but also helps future researchers make better decisions. Sharing our challenges and the solutions we came up with adds to a pool of knowledge that will advance science in general.