Building Recommendations for Conducting Equity-Focused, High Quality K-12 Computer Science Education Research

What’s a working group at ITiCSE? It’s a collaboration between five to ten researchers from around the world to produce a high-value report on a topic of interest in computing education.

Our objective for this working group is to develop recommendations for expanding coverage of high-quality, equitable K-12 computing education research. To engage in this work, our proposed plan includes six phases. These four phases will be discussed with the working group members at the first meeting so we can reflect on how to improve the research design. We will revise the design if ideas are brought forth that can improve its design while still being able to maintain a reasonable scope. However, the intent of the research and the focused deadlines will remain unchanged.

To register, please complete the working group application form.
Additional working groups can be found on the ITiCSE website.

Phase I: Define high-quality and equity-focused

Unfortunately, some still view addressing the needs of marginalized students and diversity, inclusion and equity as being at odds with high-quality. In this initial phase, we will reflect upon what is meant by the phrases high-quality education research and equity-focused education research, creating definitions for each. By defining these terms, we can ensure that our group is focused on the same objective and provide the community with tools for broader discussions about these terms.

Phase II: Review quality gaps and standards bodies

In this phase, we will summarize key features of high-quality research from education research standards bodies. For this activity, we will use 1) a comparative research design (George & Bennett, 2005) to review a set of international education research standards bodies, 2) a literature review to define previous finding on quality gaps and equity gaps in existing K-12 CS education research.

Phase III: Define key recommendations

In this phase, we will create a set of recommendations that align well with closing the gaps to bring education research. Using our knowledge of the gaps and the standards from Phase II, we will discuss and create a proposed set of recommendations for the K-12 CS education research community.

Phase IV: Vet recommendations

In this phase, we will vet key recommendations with experts in equity-focused research in K-12 computing education. For this qualitative study, we will develop a strata for identifying ten experts in the field. We will then use a focus groups design to solicit feedback on our comparative analysis and recommendations.

Phase V: Finalize manuscript

We will write parts of the manuscript during each phase. However, this phase will be focused on finalizing the manuscript text. We will then submit the manuscript for review.

Phase VI: Revise manuscript based on reviews

During this phase, we will revise the manuscript based on feedback received by reviewers.

Working group lead details

Dr. Monica McGill has co-led two ITiCSE working groups and has participated in two additional working groups, including one that was entirely virtual. She is well-versed in how working groups run. She understands the problems with scope creep in working groups and the need to have a well-defined plan to complete a successful study in a short-time frame with co-authors from multiple timezones. In addition to her involvement in ITiCSE working groups, she has published multiple articles investigating the state of K-12 CS education research and the gaps in quality among published articles (Decker & McGill, 2017; McGill, Decker & Abbott, 2018; McGill & Decker, 2020; Decker & McGill, 2019; Upadhyaya, et al, 2020). One of the missions of CSEdResearch.org is to build the capacity for CS education research that is both high quality and takes into consideration all K-12 students. She also serves on the ACM TOCE editorial board and is working on several National Science Foundation grants related to building capacity of K-12 CS education research in her role as Founder & CEO of CSEdResearch.org.

Dr. Sarah Heckman is new to ITiCSE working groups, but co-created and co-led the Designing Empirical Education Research Studies (DEERS) workshop for seven years. The DEERS workshops trained and mentored educators in education research with a key focus on narrowing the scope of a participant’s planned research study. The DEERS workshops and mentorship have led to several high-quality publications and furthered computing education research. She has published articles characterizing how well the computing education literature follows the standard norms for reporting empirical research (Heckman, et al, 2021). She serves on the SIGCSE Technical Symposium Steering Committee and was the Program Co-Chair of the SIGCSE Technical Symposium in 2019 and 2020. She is working on a National Science Foundation grant to build capacity for equitable K-12 CS education research. She is a Teaching Professor and Director of Undergraduate Programs for the Department of Computer Science at North Carolina State University (USA).

Organisational Details

Given that our working group is led by two people in the United States, we are seeking a diverse group of participants from a variety of backgrounds and geographic locations.

Time Zones. We will handle different time zones by choosing a variety of times to meet, so one set of individuals are not consistently having to join during their normal sleep hours. We will use an online tool to help us choose ideal times for all participants.

Suitable collaboration technology (and alternatives) and how this choice will allow for inclusive, engaged participation. We will likely use one of our Zoom channels for online meetings before and after ITiCSE with options for captioning turned on. We will also likely use Google docs with the final document published in overleaf (which has both Latex and non-Latex modes). We will use emails, and if people are comfortable, potentially use Slack for communication. We will seek to learn about any needs of the participants in the working group and work to accommodate those needs. Finally, we will engage in best practices for inclusive engagement, including using practices like “1, 2, 4” where ideas are first generated by individual participants, then those ideas are discussed in pairs, then in larger groups, then with the whole group. This is a method that helps ensure all voices are involved and heard.

Work prior to the conference. Work prior to the conference will be framed in a way that enables participants to work in pairs to accomplish specific tasks within a reasonable timeframe. This work will include exploring the research question, designing the research study, conducting a literature review, and engaging in the study. Our goal is not to create a 50-page paper, but to create a report that captures the integral parts of the research that are able to be completed within the allotted time frame.

Details of the introduction meeting for participants. Our introduction meeting will include an ice-breaking activity to build the team rapport and will define norms for our participation. We will also using this meeting to set expectations for participation and for being included as an author in the final report. We expect all participants to participate nearly equally.

Number of expected meetings (dates and details) prior to the conference. We will establish our plan with the number of expected weekly meetings and potential dates prior to advertising. By setting clear expectations early, potential participants will have the needed knowledge to fully commit to participation. We will start working in early May. We expect to conclude the research study prior to the conference.

Expected state of report (milestones) at multiple stages prior to the start of the conference and draft submission. Prior to the start of the conference, we expect to have the expert interviews coded and analyzed. We expect to use time at the conference to build a meaningful set of recommendations for the community based on our study.

Plan of work for the intensive three days, July 7-9. These days will be spent cleaning up the draft and building a meaningful set of recommendations for the community based on our study. This will allow all participants to attend ITiCSE stress-free.

Work after the conference. We expect limited work after the conference that will be primarily spent addressing reviewer comments.

Tentative meeting/work schedule for final WG report submission. We plan to hold only one to two meetings after ITiCSE to address reviewer comments. Most of this work will be done asynchronously.

To register, please complete the working group application form.

References

Decker, A., & McGill, M. M. (2017, March). Pre-college computing outreach research: Towards improving the practice. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (pp. 153-158).

Decker, A., & McGill, M. M. (2019). A systematic review exploring the differences in reported data for pre-college educational activities for computer science, engineering, and other STEM disciplines. Education Sciences, 9(2), 69.

Fincher, S., Lister, R., Clear, T., Robins, A., Tenenberg, J., & Petre, M. (2005, October). Multi-institutional, multi-national studies in CSEd Research: some design considerations and trade-offs. In Proceedings of the first international workshop on computing education research (pp. 111-121).

George, A. L., & Bennett, A. (2005). Case studies and theory development in the social sciences. mit Press.

Heckman, S., Carver, J. C., Sherriff, M., & Al-Zubidy, A. (2021). A systematic literature review of empiricism and norms of reporting in computing education research literature. ACM Transactions on Computing Education (TOCE), 22(1), 1-46.

McGill, M. M., & Decker, A. (2020, February). A gap analysis of statistical data reporting in K-12 computing education research: Recommendations for improvement. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 591-597).

McGill, M. M., Decker, A., & Abbott, Z. (2018, February). Improving research and experience reports of pre-college computing activities: A gap analysis. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (pp. 964-969).

McGill, M. M., Decker, A., McKlin, T., & Haynie, K. (2019, February). A gap analysis of noncognitive constructs in evaluation instruments designed for computing education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 706-712).

Sanders, K., Sheard, J., Becker, B. A., Eckerdal, A., & Hamouda, S. (2019, July). Inferential statistics in computing education research: a methodological review. In Proceedings of the 2019 ACM conference on international computing education research (pp. 177-185).

Upadhyaya, B., McGill, M. M., & Decker, A. (2020, February). A longitudinal analysis of k-12 computing education research in the united states: Implications and recommendations for change. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 605-611).