When conducting research and engaging with participants, communities, or organizations, it’s crucial to clearly outline your procedures and context. This transparency helps in understanding participant engagement and enhances replicability. Include details on inclusion/exclusion criteria, sample size, and recruitment methods (American Psychological Association, 2021; U.S. Department of Education, 2022; British Educational Research Association, 2018). Respect participants’ cultural sensitivities and ensure they can opt out at any stage. Ethical considerations like consent and mitigating biases should be central to your study design (British Educational Research Association, 2018).

In computing education research (CER), especially with K-12 students, researchers must obtain informed consent from caregivers and assent from minors, outlining participant rights and study details (Denson et al., 2010; Van Mechelen et al., 2020). This is crucial as K-12 students are a vulnerable group with limited autonomy. Consent must align with local laws and be acquired for any additional use of data beyond the study’s original purpose (Scottish Educational Research Association, 2005).

Be mindful of power dynamics, such as those between teachers and students. Address potential harms, like trauma from survey questions, and inform participants about any mandatory disclosures (Scottish Educational Research Association, 2005).

Effectively categorizing and disaggregating participants by their identities is key to understanding diverse learners and avoiding further disadvantages (Yeung & Mun, 2022). Assumptions about category groupings should be avoided since these are dynamic and vary across cultures and self-identities. For instance, persistent stereotypes like “women don’t enjoy computing” can hinder advocacy for change. Additionally, broad labels like underrepresented minorities often mask the diversity within these groups, leading to an over-generalized representation (Bhatti, 2021).

Disabilities. Research on students with disabilities in computing education remains sparse, with only 2-5% of articles since 2017 addressing disability status among participants (Upadhyaya et al., 2020). Given that 15% of U.S. students receive special education services (National Center for Education Statistics, 2023), there is a significant opportunity to explore how these students engage with computing (Blaser & Ladner, 2020). This need is underscored by the rising acknowledgment of autistic individuals in STEM fields (Grandin, 1999; Stuurman et al., 2019).

Race and Ethnicity. Overgeneralizations in race and ethnicity data can diminish its relevance. For example, the U.S. census category “Asian Americans” includes people from diverse regions like the Far East, India, and Pacific Islands, masking socio-cultural differences (Yeung & Mun, 2022). Similarly, India’s 705 officially recognized ethnic groups are often grouped into one category despite their diverse backgrounds.

Cultural Identity. Cultural identity encompasses language, social group, religion, economic status, gender, age, and sexual orientation (Office for National Statistics, 2003). Computing education research should reflect this diversity to inform and guide practitioners and policymakers (UK Department of Health, 2005; Altugan, 2015). Accurate categorization helps understand various learner groups and their needs.

Gender. Women hold only 25% of positions in the tech industry (​​National Center for Women & Information Technology, 2022). To address this disparity, understanding why it persists and identifying promising interventions is crucial. Yet, only 64% of research papers gather gender data, often limited to a binary perspective (McGill & Decker, 2017). There is a growing need to collect data that acknowledges non-binary gender identities.

Emerging Language Learners. Framing students who speak different languages from the classroom language can shape research interpretations. Terms like emerging bilingual/multilingual and English fluent students reflect an asset-based approach to their existing skills (Hughes et al., 2022).

Educator Identities. Teacher identities significantly impact minoritized students’ learning (Montecinos, 2004). However, research often focuses on the experiences of White preservice teachers (Sleeter, 2001). Accurate data on instructor identities and their prior experiences with computing can provide valuable context for students’ learning experiences. Despite this, reporting on instructor race is still rare, though it is increasing: no studies collected this data in 2014-2016 in computing education research, while 17% did in 2022 (McGill & Decker, 2017).

Strategies for Engaging with Participants

Select participants

Clearly define the procedures and context for participants in the research study.

Choose and enlist participants from all relevant groups, taking into account factors such as sample size, power analysis, and appropriate representation.

Specify inclusion/exclusion criteria, sample size rationale (when it fits into the study design), recruitment and retention procedures, assignment to conditions, and participant follow-up during and, where appropriate, after the study (American Psychological Association, 2021; British Educational Research Association, 2018; Schulz et al., 2010; U.S. Department of Education, 2022).

 Identify how the criteria for choosing participants will answer the research questions(s), particularly from an equity-enabling perspective.

Define diversity dimensions as appropriate for the research question(s) and context to maximize transferability of findings (American Psychological Association, 2021; Natural Sciences and Engineering Research Council of Canada, 2022).
Provide opportunities for participants to share their own identities (e.g., gender, race/ethnicity) as appropriate for the research (Oleson, et al., 2022).

Cultivate relationships to broaden understanding of and access to sites/participants, improving quality, equity, generalizability, and transferability (Amundsen et al., 2017).

When possible and where needed (e.g., deception studies), provide participant debriefing of the study. Similarly, provide opportunities for participants to give feedback, particularly on hardware and software tools used in the study.

Interact ethically

Ensure ethics for human subject research informs procedures, including consent and consideration for vulnerable populations (e.g., Indigenous peoples) (American Psychological Association, 2021). This may include obtaining caregiver consent for all minors, consent/assent from participants, and consent to use publicly accessible data for research purposes where appropriate.

Honor participant choice and withdrawal from the study.

Anticipate and mitigate potential harms to participants or related groups.

Inform study participants (including, where appropriate, caregivers for minors) of their rights, the nature of the study, and their role in it with awareness of cultural sensitivities (American Psychological Association, 2021).

Consider power dynamics

 Be aware of and, when possible, mitigate power dynamics between researchers and participants (e.g., teacher/student relationships) (Scottish Educational Research Association, 2005).

Share data results and analysis with participants to ensure that the research is appropriately representing them.

 

Additional Resources

References

Altugan, A. S. (2015). The relationship between cultural identity and learning. Procedia-Social and Behavioral Sciences, 186, 1159–1162. Elsevier.

American Psychological Association. (2021). Journal Article Reporting Standards (JARS). Retrieved from https://apastyle.apa.org/jars

Amundsen, D. L., Msoroka, M., & Findsen, B. (2017). ’It’s a case of access.’ The problematics of accessing research participants. Waikato Journal of Education, 22(4).

Asmal, L., Lamp, G., & Tan, E. J. (2022). Considerations for improving diversity, equity and inclusivity within research designs and teams. Psychiatry Research, 307, 114295.

Bhatti, H. A. (2021). Toward “inclusifying” the underrepresented minority in STEM education research. Journal of Microbiology & Biology Education, 22(3), e00202-21.

Blaser, B., & Ladner, R. E. (2020). Why is data on disability so hard to collect and understand? In 2020 Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) (Vol. 1, pp. 1–8). IEEE.

British Educational Research Association. 2018. Ethical Guidelines for Educational Research, fourth edition (2018) (4 ed.). British Educational Research Association. 1–48 pages.

Burke Reifman, J., White, M., & Kalish, L. (2022). Students as Researchers and Participants: A Model of Iterative Member-Checking for Inclusive, Equity-Centered Assessment Research. Intersection: A Journal at the Intersection of Assessment and Learning, 3(1), n1.

Cuskelly, M. (2005). Ethical inclusion of children with disabilities in research. In Ethical research with children (pp. 97–111).

Denson, C., Austin, C., Hailey, C., & Householder, D. (2015). Benefits of informal learning environments: A focused examination of STEM-based program environments. Journal of STEM Education, 16(1). Laboratory for Innovative Technology in Engineering Education (LITEE).

Grandin, T. (1999). Choosing the right job for people with autism or Asperger’s syndrome. Indiana Resources Center for Autism.

Hatch, J. A. (2002). Doing qualitative research in education settings. State university of New York press.

Higher Education Quality Council of Ontario, Cyr, D., Childs, R., & Elgie, S. (2013). Recruiting students for research in postsecondary education: A guide. Higher Education Quality Council of Ontario.

Hughes, B. S., Corrigan, M. W., Grove, D., Andersen, S. B., & Wong, J. T. (2022). Integrating arts with STEM and leading with STEAM to increase science learning with equity for emerging bilingual learners in the United States. International Journal of STEM Education, 9(1), 1–19. SpringerOpen.

Khatamian Far, P. (2018). Challenges of recruitment and retention of university students as research participants: Lessons learned from a pilot study. Journal of the Australian Library and Information Association, 67(3), 278–292.

McGill, M. M., & Decker, A. (2017). Computer Science Education Resource Center. https://CSEdResearch.org

McGill, M. M., Decker, A., & Abbott, Z. (2018). Improving Research and Experience Reports of Pre-College Computing Activities: A Gap Analysis. Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 964–969. New York, NY, USA: ACM.

Montecinos, C. (2004). Paradoxes in multicultural teacher education research: Students of color positioned as objects while ignored as subjects. International Journal of Qualitative Studies in Education, 17(2), 167–181. Taylor & Francis.

National Center for Education Statistics. (2023). Students with disabilities. https://nces.ed.gov/programs/coe/indicator/cgg/students-with-disabilities

National Center for Women & Information Technology. (2022). By The Numbers. https://ncwit.org/resource/bythenumbers/

Natural Sciences and Engineering Research Council of Canada. (2022). Guidelines on the assessment of contributions to research, training and mentoring. Retrieved from https://www.nserc-crsng.gc.ca/NSERC-CRSNG/Policies-Politiques/assessment_of_contributions-evaluation_des_contributions_eng.asp

Office for National Statistics. (2003). Ethnic group statistics: A guide for the collection and classification of ethnicity data (ISBN 1857745612). www.clickanduse.hmso.gov.uk

Oleson, A., Xie, B., Salac, J., Everson, J., Kivuva, F. M., & Ko, A. J. (2022, August). A Decade of Demographics in Computing Education Research: A Critical Review of Trends in Collection, Reporting, and Use. In Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1 (pp. 323-343).

Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42, 533–544. Springer.

Patton, M. Q. (2002). Qualitative research & evaluation methods. SAGE Publications. https://books.google.com/books?id=FjBw2oi8El4C

Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340. doi:10.1136/bmj.c332

Scottish Educational Research Association. 2005. Ethical Guidelines for Educational Research. Scottish Educational Research Association. 1–15 pages.

Shah, N., Christensen, J. A., Ortiz, N. A., Nguyen, A.-K., Byun, S., Stroupe, D., & Reinholz, D. L. (2020). Racial hierarchy and masculine space: Participatory in/equity in computational physics classrooms. Computer Science Education, 30(3), 254–278.

Sinclair, J., Hansen, S. G., Machalicek, W., Knowles, C., Hirano, K. A., Dolata, J. K., … Murray, C. (2018). A 16-year review of participant diversity in intervention research across a selection of 12 special education journals. Exceptional Children, 84(3), 312–329.

Sleeter, C. E. (2001). Preparing teachers for culturally diverse schools: Research and the overwhelming presence of whiteness. Journal of Teacher Education, 52(2), 94–106. Sage Publications.

Stuurman, S., Passier, H. J. M., Geven, F., & Barendsen, E. (2019). Autism: Implications for inclusive education with respect to software engineering. In Proceedings of the 8th Computer Science Education Research Conference (pp. 15–25).

UK Department of Health. (2005). Research governance framework for health and social care. Health and Social Care in the Community, 10(1), 1–54.

Upadhyaya, B., McGill, M. M., & Decker, A. (2020). A longitudinal analysis of K-12 computing education research in the United States: Implications and recommendations for change. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 605–611).

U.S. Department of Education. 2022. What Works Clearinghouse: Procedures and Standards Handbook, Version 5.0. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf

Yeung, G., & Mun, R. U. (2022). A Renewed Call for Disaggregation of Racial and Ethnic Data: Advancing Scientific Rigor and Equity in Gifted and Talented Education Research. Journal for the Education of the Gifted, 45(4), 319–351.

Van Mechelen, M., Baykal, G. E., Dindler, C., Eriksson, E., & Iversen, O. S. (2020). 18 years of ethics in child-computer interaction research: A systematic literature review. In Proceedings of the Interaction Design and Children Conference (pp. 161–183). Association for Computing Machinery. https://doi.org/10.1145/3392063.3394407