When choosing methods, quantitative data excels in generalizing findings, identifying patterns, and exploring correlations and causation. Yet, it often forces data into fixed categories, potentially stripping away context by emphasizing the “what” and “how many.” Conversely, qualitative analysis delves into themes and meanings, providing context and exploring the “why” and “how” (Morgan, 1993). While it doesn’t lend itself to broad generalizations, it offers deep insights. Both methods, used separately or combined, are valuable.

Using Quantitative Methods

Quantitative research must align with established methods and quality standards (German Research Foundation, 2019). Common designs in education include descriptive studies, randomized controlled trials, and correlational research. Researchers should predetermine sample sizes, effect types, materials, procedures, and data sources before collecting data (American Psychological Association, 2021). The validity of metrics must be reassessed relative to the research questions and participants to maintain integrity (German Research Foundation, 2019).

Contextualization and Positionality. Quantitative methods often overlook researcher biases and positionality, which can skew data interpretation (Guba et al., 1994; Zuberi et al., 2008). Properly framing demographic variables is essential to uncover causal mechanisms driving inequalities and avoid deficit-based interpretations (Pearson et al., 2022). Addressing context and acknowledging limitations can improve study outcomes (Chouinard & Cousins, 2009; Garibay & Teasdale, 2019).

QuantCrit. QuantCrit, integrating critical race theory with quantitative data, promotes a holistic approach to understanding minoritized groups (Zuberi, 2001). It challenges biased perspectives and offers a fuller representation of racial and ethnic realities (Pearson et al., 2022).

Sample Sizes and Generalizability. Sample size considerations vary based on population size, measurement error, and effect sizes (Krejcie & Morgan, 1970; Ahmad & Halim, 2017). Smaller samples may yield larger effect sizes but are less generalizable (Slavin & Smith, 2009). Randomized-controlled and well-designed quasi-experimental studies are preferred for evaluating interventions (Slavin & Smith, 2009).

Advanced Statistical Models. Models like multilevel analysis account for data clustering within educational settings, allowing for equitable outcome comparisons (Ohland et al., 2011; Espinosa, 2011; Leath & Chavous, 2018). Disaggregating data categories enables more meaningful participant self-description and supports asset-based approaches (Bhatti, 2021; Pearson et al., 2022).

Using Qualitative Research

Qualitative research approaches often lend themselves to deeper understanding of a phenomenon related to participants and their environments by emphasizing context, perspectives, and experiences. Unlike quantitative methods that focus on numerical data and statistical trends, qualitative approaches delve into the ‘why’ and ‘how’ of human behaviors. This methodological flexibility allows researchers to employ techniques such as interviews, focus groups, ethnography, and case studies, each tailored to capture the richness of individual and collective experiences (Kvale, 2008; Krueger & Casey, 2015; Hammersley & Atkinson, 2019; Yin, 2018).

Central to qualitative research is its ability to contextualize findings within the socio-cultural settings of participants, thereby uncovering nuanced insights that quantitative data alone may overlook (Lincoln & Guba, 1985). It fosters empathy and trust, enabling researchers to engage deeply with participants and elicit candid responses that contribute to a more comprehensive understanding of their worldviews and realities (Tracy, 2020).

Despite its strengths, qualitative research is not without limitations. Findings are context-specific and may not be easily generalized beyond the studied population or setting (Stake, 1995). Moreover, interpretations can be subjective, influenced by researchers’ biases and perspectives (Braun & Clarke, 2006). However, these challenges are mitigated by rigorous methodological practices and transparent reporting, ensuring the reliability and validity of qualitative studies (Merriam & Tisdell, 2016).

In conclusion, qualitative research plays a crucial role in education and social sciences by offering in-depth insights into diverse populations and their environments. By embracing complexity and context, qualitative methods provide a holistic understanding that informs policy, practice, and further research efforts (Denzin & Lincoln, 2018; Creswell & Poth, 2018).

Using Mixed Methods Approaches

Mixed methods research combines quantitative and qualitative approaches to offer a comprehensive investigation of research phenomena. This integration allows for insights that are impossible to achieve through either approach alone, essentially making the sum greater than its parts (Fetters & Freshwater, 2015). The combination of methods allows for engaging diverse beneficiaries and creating opportunities for more thorough and inclusive research, especially in studies focused on social equity (Pearson et al., 2015).

The key to mixed methods is integration—merging quantitative and qualitative elements to produce richer insights (Creswell & Plano Clark, 2018; Johnson & Onwuegbuzie, 2004). Integration can happen at various study stages such as purpose, research question, design, method, results, and discussion (Love et al., 2022). Researchers should specify the phases where each methodology is applied, demonstrate how the design aligns with the research questions, and clarify the intent behind using mixed methods.

Mixed methods enhance understanding by capturing complex data and relationships, offering in-depth insights for inclusive and effective practices (Marx, 2016; Pearson et al., 2022). This approach is valuable in educational research for synthesizing findings and addressing inequities, particularly in supporting marginalized groups. Creswell and Plano Clark (2018) describe three main mixed methods designs:

  1. Convergent: Combining both methods simultaneously.
  2. Explanatory Sequential: Starting with quantitative, followed by qualitative to explain results.
  3. Exploratory Sequential: Beginning with qualitative to build to quantitative analysis.

Each design varies in how and when the different methods are used.

Mixed methods should not be confused with multiple methods research, which may use various qualitative approaches without combining them with quantitative methods (Roller & Lavrakas, 2015).

Strategies for Using Methods 

Determine appropriate methods; if new methods need to be created for the study, clearly state why known methods are not appropriate and clearly state the process for creating the new method.

Define the research design, sample size, materials, measures, procedures, data sources, and the study type, selecting appropriate methods of inquiry to answer the research question(s) (APA Jars). For quantitative methods, this includes descriptive, randomized controlled trial, quasi-experimental, correlational, time-series, longitudinal, or single-case designs and variants of each. For qualitative methods, this means study types may include ethnographic, grounded theory, phenomenology, or ethnography. For mixed methods, this includes convergent, explanatory sequential, and exploratory sequential designs and researchers should clearly plan which phases of the research project will use quantitative and qualitative methodologies (Creswell, 2012; Love et al., 2022).

Privilege the experiences and needs of non-dominant communities, centering asset-based approaches.

Acknowledge researcher’s positionality and its impact on the study, including how it mitigates biases, addresses limitations, impacts the data collection process, and interpretation of data.

Collect and continually assess evidence of validity and reliability. For quantitative data, this may mean conducting analysis using Cronbach’s alpha, confirmatory factor analysis, and exploratory factor analysis. For qualitative data, this may mean engaging in member checking, external audits, and triangulation.

Develop consistent data collection and analysis techniques when multiple researchers are engaged in collecting and analyzing qualitative data.

Disaggregate monolithic groupings for more contextualized and inclusive characteristics (e.g., collecting diversity dimensions that accurately reflect the participants, comparing groups using descriptive and inferential statistics (e.g., multilevel models)) (Bhatti, 2021; Sinclair et. al., 2018)

Acknowledge all changes to the procedures, including inconsistencies or deviations and their potential impact on findings.

Ensure the data adequately captures all relevant forms of diversity, adapting to arising needs for more or different data when needed for representation.

For Mixed methods: 

      Clearly communicate the purpose, intent, and rationale behind using a mixed methods approach, including how methodologies complement each other in addressing the research questions and objectives.

      Qualitative methods used in a mixed-methods study should meet the criteria for qualitative methods.

      Quantitative methods used in a mixed-methods study should meet the criteria for quantitative methods.

      Integrate findings from the quantitative and qualitative methods, investigating the meaning behind complementary and contradictory data.

      Reflect upon any challenges, biases, and trade-offs from integrating findings and provide a transparent account of how these limitations will be mitigated or managed.

Additional Resources

References

Ahmad, H., & Halim, H. (2017). Determining sample size for research activities. Selangor Business Review, 2(1), 20–34.

American Psychological Association. (2021). Journal Article Reporting Standards (JARS). Retrieved from https://apastyle.apa.org/jars

Armstrong, D., Gosling, A., Weinman, J., & Marteau, T. (1997). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31(3), 597–606.

Bhatti, H. A. (2021). Toward “inclusifying” the underrepresented minority in STEM education research. Journal of Microbiology & Biology Education, 22(3), e00202-21.

Byrd, W. C. (2021). Behind the diversity numbers. Harvard Education Press.

Chouinard, J. A., & Cousins, J. B. (2009). A review and synthesis of current research on cross-cultural evaluation. American Journal of Evaluation, 30(4), 457–494.

Covarrubias, A. (2011). Quantitative intersectionality: A critical race analysis of the Chicana/o educational pipeline. Journal of Latinos and Education, 10(2), 86–105. Taylor & Francis.

Cooper, S., Grover, S., Guzdial, M., & Simon, B. (2014). A future for computing education research. Communications of the ACM, 57(11), 34-36. ACM.

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson Education, Inc.

Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches (2nd ed.). Sage Publications, Inc.

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage Publications.

Denson, C., Austin, C., Hailey, C., & Householder, D. (2015). Benefits of informal learning environments: A focused examination of STEM-based program environments. Journal of STEM Education, 16(1). Laboratory for Innovative Technology in Engineering Education (LITEE).

Espinosa, L. (2011). Pipelines and pathways: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review, 81(2), 209–241.

Fetters, M. D., & Freshwater, D. (2015). The 1+ 1= 3 Integration Challenge. Journal of mixed methods research, 9(2), 115-117.

Garibay, C., & Teasdale, R. M. (2019). Equity and evaluation in informal STEM education. New Directions for Evaluation, 2019(161), 87–106.

German Research Foundation. 2019. Guidelines for Safeguarding Good Research Practice. https://www.htwg-konstanz.de/fileadmin/pub/forschung/Forschungsreferat/Guidelines_for_Safeguarding_Good_Researche_Practice_DFG__2019.pdf

Guba, E. G., Lincoln, Y. S., & Others. (1994). Competing paradigms in qualitative research. Handbook of Qualitative Research, 2(163–194), 105.

Hammer, D., & Berland, L. K. (2014). Confusing claims for data: A critique of common practices for presenting qualitative research on learning. Journal of the Learning Sciences, 23(1), 37-46.

Hazzan, O., Dubinsky, Y., Eidelman, L., Sakhnini, V., & Teif, M. (2006). Qualitative research in computer science education. ACM SIGCSE Bulletin, 38(1), 408-412. ACM.

Hood, S., Hopson, R. K., & Kirkhart, K. E. (2015). Culturally Responsive Evaluation. In K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of Practical Program Evaluation (4 ed., pp. 281-317). Jossey-Bass. https://doi.org/10.1002/9781119171386.ch12

Johnson, R. . B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014

Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607–610. Sage publications.

Leath, S., & Chavous, T. (2018). Black women’s experiences of campus racial climate and stigma at predominantly white institutions: Insights from a comparative and within-group approach for STEM and non-STEM majors. Journal of Negro Education, 87(2), 125–139.

Leung L. (2015). Validity, reliability, and generalizability in qualitative research. J Family Med Prim Care. 2015 Jul-Sep;4(3):324-7. doi: 10.4103/2249-4863.161306.

López, N., Erwin, C., Binder, M., & Chavez, M. J. (2018). Making the invisible visible: Advancing quantitative methods in higher education using critical race theory and intersectionality. Race Ethnicity and Education, 21(2), 180–207.

Love, H. R., Cook, B. G., & Cook, L. (2022). Mixed-Methods Approaches in Special Education Research. Learning Disabilities Research & Practice, 37(4), 314–323.

Mangahas, A. (2023). Issues of Diversity, Equity, and Inclusivity in Online Secondary STEM Education. Teaching and Learning Online: Science for Secondary Grade Levels, 31.

Marx, S. (Ed.). (2016). Qualitative research in STEM: Studies of equity, access, and innovation. Routledge.

McGee, E. O. (2021). Black, brown, bruised: How racialized STEM education stifles innovation. Harvard Education Press.

Miller-Cotto, D., & Lewis Jr, N. (2020). Am I a “math person”? How classroom cultures shape math identity among Black and Latinx students.

Morgan, D. L. (1993). Qualitative content analysis: a guide to paths not taken. Qualitative health research, 3(1), 112-121.

Ohland, M. W., Brawner, C. E., Camacho, M. M., Layton, R. A., Long, R. A., Lord, S. M., & Wasburn, M. H. (2011). Race, gender, and measures of success in engineering education. Journal of Engineering Education, 100(2), 225–252.

Parson, L. (2019). Considering Positionality: The Ethics of Conducting Research with Marginalized Groups. Research Methods for Social Justice and Equity in Education. Springer.

Patton, M. Q. (2002). Qualitative research & evaluation methods. SAGE Publications. https://books.google.com/books?id=FjBw2oi8El4C

Pearson, M. I., Castle, S. D., Matz, R. L., Koester, B. P., & Byrd, W. C. (2022). Integrating critical approaches into quantitative STEM equity work. CBE—Life Sciences Education, 21(1), es1.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. Guilford Publications.

Saldana, J. (2013). The coding manual for researchers: An introduction to codes and coding.

Slavin, R., & Smith, D. (2009). The relationship between sample sizes and effect sizes in systematic reviews in education. Educational evaluation and policy analysis, 31(4), 500-506.

Zuberi, T. (2001). Thicker than blood: How racial statistics lie. U of Minnesota Press.

Zuberi, T., Bonilla-Silva, E., & Others. (2008). White logic, white methods: Racism and methodology. Rowman & Littlefield Publishers.