Category Archive: Research

Attending SIGCSE Technical Symposium 2023? We’ll be there!

We’ll be actively engaged in the 2023 SIGCSE Technical Symposium (TS) in Toronto.

If you’d like to learn more about the work we have recently been engaged in, be sure to stop any of our sessions. Only our workshop requires registration.

Day/TimeTypeRoomTitle

Community Listening Sessions

Given the recent discussions across asynchronous platforms this past week, many related to feedback from SIGCSE TS reviews, we have set up two listening sessions for community members to talk about their experiences and perspectives.

The listening sessions are an opportunity to share concerns about barriers within the community that prevent researchers from reaching their full potential (and even worse, be driven out of the community) AND ultimately helping the hundreds of thousands of teachers and millions of students that we aim to support.

October 12, 4-5pm CT

October 13, 1-2pm CT

The sessions will not be a time for anyone to offer excuses or rationale for these barriers. Also, you do not need to attend both sessions.

Sessions will be moderated and norms will be set to minimize disrespect and harm. You will be able to share concerns anonymously if you choose. Sessions will be closed captioned. For those with disabilities or conflicts and attending is not possible, please feel free to email me directly with concerns.

We offer this as a community place to recognize the harm as well as the trauma that has been experienced by members in an effort to help move the dialogue into forming actionable steps for improvement at a later time. As an aside, the SIGCSE TS committee has offered to share our aggregated results of these listening sessions on their website through a blog post or another mechanism in conjunction with other plans they have for addressing these issues. Though these listening sessions are independent of the SIGCSE board and SIGCSE TS Committee, we hope they can be used to help inform their future plans. However, we are also very open to hearing more broadly about barriers that go beyond SIGCSE conferences and community.

Connecting K-5 Students to Integrated Computer Science

We recently partnered with Code.org to conduct a national study that focuses on how K-5 teachers integrated computer science (CS) into their curriculum. Why? Well, Code.org is working on a new and unique CS curriculum called Computer Science Connections. 

The goal of their curriculum is to teach computing by making critical connections between learning CS and other subjects like math, language arts, science, and social studies. 

Presently, there is minimal research and knowledge available that discusses how and/or why teachers integrate CS into other subject areas. There is also minimal scholarship focused on the barriers teachers and administrators may face when attempting to integrate CS into other K-5 content areas. We believe this will be an important area to watch in the next few years as CS enters into more K-5 classrooms and teachers struggle to balance teaching a new subject without more hours in the day to do so.

Where do we come in? We will be reaching out to states all over the country – 29 in total – to get an overarching view of how and why CS is integrated (or not integrated) into K-5 classrooms. We will also be conducting a systematic literature review (SLR) to better facilitate conversations around promising practices integrating CS into K-5 learning environments. Overall, this information will be used as a launching point that Code.org will be able to use as they continue to expand their mission of teaching all students computer science. 

Are you interested in finding out more about our work? Watch our social media and look for our ongoing updates, or visit Code.org’s Computer Science Connections page and start integrating CS into your curriculum.

Computer Science Teachers’ Problems of Practice: Solve This!

In 2021 we received funding from a ACM SIGCSE Special Projects Grant, with our colleague Dr. Michelle Friend (University of Nebraska – Omaha) for a project we called: Solve this! Problems of practice teachers face in K-12 CS Education. Since then we have been working on gathering, analyzing, and disseminating the findings. Overall, our goal for this project is to provide a platform for researchers to understand authentic problems of practice that teachers face in order to bridge the gap between research and practice. 

What have we accomplished so far?

At the beginning of the project we designed a survey to be sent to teachers around the world. The survey included demographic questions about the teacher and their locale, but most importantly about the problems of practice they experience when planning, teaching, or attempting to plan/teach computer science in their school or classroom. Once the survey underwent internal and external face validity, we disseminated the survey. Our survey reached teachers in Ireland, Canada, and the United States. We opened it in July 2021 and closed it in October 2021, receiving over 700 responses.

After cleaning the data, we were left with 396 responses. We created over 40 codes as we  analyzed the data and several themes emerged. Although we are still in the process of data analysis, some of the initial findings include problems of practice such as a lack of teaching time or schedule availability to teacher CS, poor academic habits, and challenges related to student interactions or partner work. We have been able to share initial results at several conferences and our paper examining our initial set of data has been accepted to Koli Calling 2022. 

What is next? 

Our goal is to have our interactive K-12 CS education teaching problems of practice populated and ready for use by the end of this year. All of the problems of practice entered through this study will be added to our website and will be searchable by demographics of the teachers who submitted them (e.g., country, years teaching CS). 

For researchers, this site will provide you with the problems teachers are facing and can help inform your research agenda. 

Teachers will be able to upvote problems of practice that they experience and will be able to add their own problems. 

Watch our social media platforms for our Problems of Practice page announcement!

CS Teachers’ Reflection on the CSTA K-12 Teacher Standards

This past summer, IACE had the opportunity to partner with the Computer Science Teachers Association (CSTA) and CREATE, a research center at the University of California, San Diego, to develop an assessment of teachers’ understanding and use of the CSTA K-12 CS Teacher Standards. As part of this process, we wanted to understand how the Standards can help inform CS teachers’ professional reflection process and their professional development trajectory. With funding from E_CSPD_Week, a U.S. Department of Education EIR grant, CSEdResearch.org joined the partnership to break down the CSTA K-12 Teacher Standards to usable rubric language for personalized reflection and feedback. This summer we piloted a reflection-based assessment for Standards 2-5, with Standard 1 being piloted next summer. After piloting our designed process in two states, Indiana and South Carolina, we learned a lot and continue to improve the process. 

To provide a high level overview of the work that went into the process, our team, along with assistance from CSTA, dissected the CSTA K-12 Teacher Standards 2-5 to create 18 rubric items and scales across three main categories: 1) plan, 2) assessment, and 3) professional growth and development. We then created an entry form to collect the data from a group of teachers in Indiana and South Carolina who participated in the pilot of this work. We are currently undergoing the next phase, scoring and developing a process for external expert readers to provide feedback to the teachers who submit their information as part of this optional process. Our work has resulted in a set of recommendations on how to improve the process so teachers are able to more easily collect and enter their data, which we provided to CSTA and CREATE during a recent discussion. Once completed, this will be tested with a wider group of teachers in summer 2023 and go through a second revision process.

We are also in the process of starting work developing an assessment for Standard 1, CS content knowledge. Working with Dr. Adrienne Decker, we will be creating a brief assessment for AP CS A targeted to high school teachers. We will be piloting this assessment in summer 2023.

Stay tuned for updates on this project. 

Educate Maine: Decreasing Financial Barriers and Increasing Access to Coding

At IACE we find great value in raising up the voices of our partners who are doing great things in the computer science community. One of those partners is Educate Maine.

This summer Educate Maine’s signature project, Project>Login, hosted 5 Girls Who Code camps all over the state of Maine.

They were able to provide these camps for free, decreasing financial barriers and increasing access for all students.  The girls who participated were able to engage with industry professionals, learn from experienced teachers, and make memories to last a lifetime.

What was our role in this amazing experience? External evaluators. As part of this work, Educate Maine is continuing to reflect and improve their practices through evaluations. The evaluations focused on student and teacher experiences during the week long camps all over the state. Most of the girls who participated in Project>Login’s Girls Who Code camps do not have coding at their school or a Girls Who Code after school program, therefore this summer experience is truly increasing their knowledge of what it means to be a “coder” and, more widely, a “computer scientist”

WEX Industry Partner with Girls Who Code campers

Girls at their Girls Who Code camp

Partner work at one of the Girls Who Code camps

Interested in checking out our other projects? Click here

K-12 Computer Science Teachers Problems of Practice

Recently, K-12 teachers in Indiana spoke to us during #CSTAPDWeekIN (CSTA PD Week in Indiana) about problems of practice that they have experienced or witnessed during their time teaching CS. We highlight here a few of their thoughts.
—-

K-12 CS teachers from Indiana recently shared problems of practice with us that they have witnessed or experienced related to computer science education. Three of these addressed the lack of trained teachers and how that impacts schools and students, misconceptions about who belongs in CS, and the lack of funding for CS education despite state-wide mandates.

Lack of trained teachers

Danielle Carr, CS teacher at Lake Central High School and CS instructor for IndianaComputes!, said, “I wish they (researchers) knew that many of the teachers teaching CS don’t have a background in CS.” Echoing this, Jennifer Hanneken, school library media specialist at Lawrenceburg Community School Corporation, noted that, “Schools need to be required to have computer science taught by a qualified, certified teacher. They need to understand the enormity of the standards.” This is especially true if we are to achieve equitable learning outcomes so that all students receive quality computer science instruction.  Of the 35 teachers we spoke with, nearly one-third indicated that they were second career teachers who came from industry and were asked to teach CS specific courses. Nearly half of the teachers were asked or voluntold to teach CS in their schools with limited to no background in CS specific content. This phenomenon is not unique to Indiana and is a known issue in other places around the country and the world. It speaks to how important it is to consider this in our research, particularly how it relates to student outcomes.

Misconceptions about who belongs

Teachers were also aware of the critical aspect of student and teacher perceptions about “who” belongs in CS. Carr further said, “There is a stigma of who ‘should/can’ do computer science”, which inevitably impacts recruitment to courses. Carrie Koontz, 6th grade science teacher at Edgewood Junior High School, echoed this, “Students consider CS as a subject for certain people, not for everyone.” While national statistics (such as those provided by Code.org) display an increase of students studying CS who are from historically excluded populations, there is still a long way to go. Equitable CS education is and continues to be a critical goal that must be achieved both in Indiana and nationwide if we want to broaden participation. Overcoming the notion of who “belongs” is an important first step.

Funding for all aspects of CS education

One of the teachers, Kathryn Dunphy, a K-5 teacher at Avon Community School Corporation, said that “there is an assumption of what Computer Science is, so funding is not provided for what is needed, outside of maybe computers.” Several other teachers said that lack of funding for CS education programs, manipulatives for younger students, and related resources are often pushed to the bottom of budgets each school year. As the policies move forward in state after state and district after district, policymakers and legislatures must ensure that mandates are funded adequately to ensure the best learning outcomes for their students.

Jennifer Hanneken

Danielle Carr

Interested in learning about more problems of practice? Click here

A Conversation with Dr. Satabdi Basu

This post features Dr. Satabdi Basu, a Senior Education Researcher at SRI International, an independent, nonprofit research institute.

Dr. Basu joined SRI International in 2016 after receiving her Ph.D. from Vanderbilt University in Computer Science (CS) with a specialization in AI for Education. At SRI, she leads projects in K-12 CS and AI education that give her the opportunity to work with school districts and focus on assessment and curriculum design and teacher professional development. She currently is working on several projects, including developing middle school CS teachers’ understanding of CS standards and formative assessment practices at Milwaukee Public Schools and bringing the SPICE (Science Projects Integrating Computing and Engineering) project to Metro Nashville Public Schools and Charlottesville City Public Schools. She is also working on an international project with school systems in Hong Kong that are focused on promoting computational thinking in primary grades. 

Her research work also includes published articles summarized on our site:

• A Principled Approach to Designing a Computational Thinking Practices Assessment for Early Grades by Satabdi Basu, Daisy Rutstein, Yuning Xu, Linda Shear | ACM SIGCSE TS (2020)
• The Role of Evidence Centered Design and Participatory Design in a Playful Assessment for Computational Thinking About Data by Satabdi Basu, Betsy Disalvo, Daisy Rutstein, Yuning Xu, Jeremy Roschelle, Nathan Holbert | ACM SIGCSE TS (2020)
• What We Can Learn About Student Learning From Open-Ended Programming Projects in Middle School Computer Science by Shuchi Grover, Satabdi Basu, Patricia Schank | ACM SIGCSE TS (2018)
• Measuring Student Learning in Introductory Block-Based Programming: Examining Misconceptions of Loops, Variables, and Boolean Logic by Shuchi Grover, Satabdi Basu | ACM SIGCSE TS (2017)
• A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming Environments by Shuchi Grover, Satabdi Basu, Marie Bienkowski, Michael Eagle, Nicholas Diana, John Stamper | ACM ToCE (2017)

Dr. Basu found the resources at CSEdResearch.org about 4 or 5 years ago when searching for survey instrumentation. She has used it ever since. She says, “I use the filters to find what I need, especially when writing literature reviews or designing instruments.” Internally at SRI, she endorses the resources on CSEdResearch.org and is always looking through the site to find new and updated information.

When asked why she uses CSEdResearch.org, she responded, “If it wasn’t there, I would be using Google Scholar to find papers, search for the instruments used, reach out to authors – that takes a lot of time. Using the site is easier and cuts down on the time of putting together instruments for projects or literature reviews for papers.”

In addition to the website, she also enjoys the tweets CSEdResearch.org provides that focus on brief informative guidance on instrument creation.

 
Dr. Satabdi Basu is a Senior CS Education Researcher at SRI International. She has published numerous articles on CS education research, particularly focused on computational thinking and K-12 students. She has presented at national and international conferences, and also been invited as a keynote speaker.

Block-based Programming in Computer Science Classrooms

This week’s post features David Weintrop and his research on block-based programming. He shares three key points his research has discovered so far.

The first time I saw Scratch, I thought, “Wow! How clever! Is this the end of missing semi-colons errors!?” It was clear to me how the shape of the blocks, their easily understood behaviors, and the Sprites they controlled all worked together to make programming more accessible and inviting.

With my background in computer science, I could also see how foundational programming concepts were also present. I started with Scratch but then discovered a whole host of other environments, like Snap!, MIT AppInventor, Pencil Code, and Alice, that used a similar block-based approach. This got me thinking – do kids learning computer science with block-based tools? Should it be used in the Classroom? If so, what is the role of the Teacher? And finally, will block-based help kids learn text-based programming languages like Java and Python? My research seeks to try and answer these questions. Here is a bit of what I have found.

Kids think block-based programming is easier than text-based programming.

As part of my research on block-based programming in K-12 classrooms, I asked students what they thought about block-based programming. For the most part, students perceived block-based programming to be easier than text-based programming. They cited features such as the “browsability” of available commands, the blocks being easier to read than text-based programming, and the shape and visual layout of the blocks. It is also worth noting that some students viewed block-based programming as inauthentic and less powerful than text-based programming.

Kids do learn programming concepts with block-based tools.

My research found that students do in fact learn programming constructs when using a block-based tool. In fact, students who learned to program using a block-based tool scored higher on programming assessments compared to students who learned with a comparable text-based tool. I found a similar result in a different study looking at the AP Computer Science Principles (CSP) exam, which asked students questions in block-based and text-based pseudocode.

May help kids learn text-based languages, but it is not automatic.

I also investigated the transition from block-based to text-based programming in high school computer science classrooms. I found that there was no difference in student performance in learning text-based programming based on prior experience with block-based or text-based programming. In other words, students performed the same regardless of how they had learned programming up to that point. One thing to note is that in my study, the teacher provided no explicit supports to help students make connections between their block-based experience and the text-based language. I mention this only to say that there is still research to be done into how best to support the blocks-to-text transition.

Overall, my research is finding that block-based programming should have a role in K-12 computer science education. While there is still work to be done, what we know so far suggests that block-based programming can serve as an effective introduction to the field of computer science.

David Weintrop is an Assistant Professor in the Department of Teaching & Learning, Policy & Leadership in the College of Education with a joint appointment in the College of Information Studies at the University of Maryland. His research focuses on the design, implementation, and evaluation of accessible, engaging, and equitable computational learning experiences. His work lies at the intersection of design, computational science education, and the learning sciences. David has a Ph.D. in the Learning Sciences from Northwestern University and a B.S. in Computer Science from the University of Michigan.

Longitudinal Trends in K-12 Computer Science Education Research

In this post, Bishakha Upadhyaya provides highlights of our SIGCSE 2020 paper on trends in K-12 CS Education research (co-authored with Monica McGill and Adrienne Decker). For more details, watch her talk or read the paper.  

Research in the field of Computer Science education is growing and so is the data and results obtained from it. Without a comprehensive look at them collectively, it can be difficult to understand the current longitudinal trends in the field. In order to identify the trends in the K-12 computing education research in the US, we conducted a longitudinal analysis of data collected from five different publication venues over the course of 7 years.

For the purpose of this analysis, we looked at the manually curated dataset on csedresearch.org with over 500 articles that focused on K-12 computing education from years 2012 to 2018. As the majority of the articles in the dataset were from the US, we only looked at research papers whose participants were also from the US. We then ran SQL queries on the dataset in order to extract the subsets of data that were later analyzed in Tableau and presented visually using graphs and tables.

Some of the major trends that we were interested in examining were:

• Locations of students/interventions studied
• Type of articles (e.g., research, experience, position paper)
• Program data (e.g., concepts taught, when activity was offered, type of activity, teaching methods),
• Student data (e.g., disabilities, gender, race/ethnicity, SES)

Results revealed that there has been an increasing shift in classroom activities from informal curriculum to formal curriculum. This shift suggests that more research is being conducted within classes offered during school hours, increasing the reach to more students with the availability of more labs, lectures and other teaching methods.

Trends also revealed that the majority of the research papers had student participants based in California. While this may seem reasonable given California is the most populous state in the US, this trend doesn’t follow for Texas, the second most populous state. There were only 4 papers that represented participants from Texas. This suggests that policies and other standards may have an influence over the computing activities and research in the state.

Our analysis revealed another longitudinal trend, various disparities in reporting the student demographics, particularly the socio-economic status (SES) of the students. For the purpose of this analysis, we considered information about free/reduced lunch as low SES if not explicitly reported in the paper. Only 32 of the articles analyzed reported information about students’ SES. Despite previous evidence showing that the SES of the student affects their academic achievement, the underreporting suggests that it is still not being considered in many research studies.

In a field with increasing efforts to increase the number of students from different backgrounds studying computer science, our research has shown considerable disparity in the research landscape of computing education. The lack of reporting makes it difficult for everyone from researchers and educators to policymakers to understand the results of these efforts, especially what needs improving. It is crucial to see how different interventions play out amongst different populations in order to implement and achieve the goals of CS for All.

Bishakha Upadhyaya is a Senior at Knox College, majoring in Computer Science and minoring in Neuroscience. She was the President of ACM-W chapter at Knox for 2019/2020 school year and served as the CS Student Ambassador. She was involved in this research as a part of her summer research project. As a part of her senior research project, she was involved in exploring the enacted curriculum in Nepal, Pakistan, Bangladesh and Sri Lanka. She will be joining Bank of America as a Global Technology Analyst after graduation in Spring 2021.