Reflecting on Positionality and Biases in Analyzing and Interpreting Data

Researchers must reflect on their positionality, biases, and methodological shortcomings when analyzing and interpreting data  (American Psychological Association, 2021). Context, power dynamics, and participant diversity should be considered to minimize biases (German Research Foundation, 2019). Transparent data analysis procedures are crucial, including checking and reporting assumptions for quantitative analyses and addressing outliers and limitations (Vetenskapsrådet, 2017). Clear definitions of outcome constructs and measures help ensure reliable and accurate analysis, emphasizing theoretical or practical significance for educators and decision-makers.

Aligning Interpretation with Research Questions

Interpretation should align with research questions, study benefits, harms, and relevant evidence, clarifying the study’s implications (Schulz et al., 2010). Standards from the APA, AERA, What Works Clearinghouse (WWC), and CONSORT offer guidance on methodology, outcomes, statistical analysis, and interpretation (McGill & Decker, 2020). Quantitative data analysis should be executed properly, acknowledging that statistical measures are not entirely objective and considering context and power dynamics (Strunk & Hoover, 2019; Pearson et al., 2022).

Holistic and Reflexive Approaches to Data

Adopting a holistic approach to data analysis considers context, power dynamics, and community experiences (Strunk & Hoover, 2019; Pearson et al., 2022). Reflexive data analysis recognizes anomalies as representing individuals’ lived experiences and ensures interpretations reflect the communities studied (Strunk & Hoover, 2019). Challenging previous findings or expectations can provide insights into participants’ lived experiences (Vetenskapsrådet, 2017), with a focus on diversity dimensions improving research outcomes (German Research Foundation, 2019; U.S. Institute for Education Sciences, 2022).

Equity in Data Interpretation

Equitable data interpretation requires awareness of cultural influences and benefits from researchers who share commonalities with participants (Garibay & Teasdale, 2019; Pearson et al., 2022). Researchers should conduct asset-based interpretations of non-dominant communities, reflecting their own experiences and collaborating with others who share participant experiences (Asmal et al., 2022; Crespo et al., 2022). This approach respects participants and leverages collaborations to enhance the research process (Hood, 1998; Kirkhart, 1995).

Strategies for Analyzing and Interpreting Data

  • Ensure that the research methods used are adequate for addressing the study research questions and consider their limitations and the impact of available tools on addressing the research questions.
  • Assess relevant assumptions of each statistical analysis performed, and modify interpretations of results accordingly, if necessary.
  • Consider study context, power-dynamics, and situational contexts to inform the selection of data analysis techniques (Pearson et. al., 2022).
  • Ensure the interpretation matches the results of the study and assess benefits and harms that might occur while considering additional supporting evidence and crucial information (Schulz et al., 2010).
  • Utilise diversity dimensions when evaluating results to understand the impact on different groups (German Research Foundation, 2019; U.S. Institute for Education Sciences, 2022).
  • Consider alternative viewpoints or the individual’s experience when results do not match expectations.
  • Seek collaboration with diverse colleagues and study participants, and continuously reflect to limit biases and ensure a more reliable interpretation of results.

Additional Resources

References

American Psychological Association. (2021). Journal Article Reporting Standards (JARS). Retrieved from https://apastyle.apa.org/jars

Asmal, L., Lamp, G., & Tan, E. J. (2022). Considerations for improving diversity, equity and inclusivity within research designs and teams. Psychiatry Research, 307, 114295.

Crespo, S., Herbst, P., Lichtenstein, E. K., Matthews, P. G., & Chazan, D. (2022). Challenges to and opportunities for sustaining an equity focus in mathematics education research. Journal for Research in Mathematics Education, 53(2), 88-93. National Council of Teachers of Mathematics.

Garibay, C., & Teasdale, R. M. (2019). Equity and evaluation in informal STEM education. New Directions for Evaluation, 2019(161), 87–106.

German Research Foundation. 2019. Guidelines for Safeguarding Good Research Practice. https://www.htwg-konstanz.de/fileadmin/pub/forschung/Forschungsreferat/Guidelines_for_Safeguarding_Good_Researche_Practice_DFG__2019.pdf

Hood, S. (1998). Responsive evaluation Amistad style: Perspectives of one African American evaluator. Proceedings of the Stake Symposium on Educational Evaluation, 101–112. ERIC.

Kirkhart, K. E. (1995). 1994 conference theme: Evaluation and social justice seeking multicultural validity: A postcard from the road. Evaluation practice, 16(1), 1-12.

McGill, M. M., & Decker, A. (2020, February). A gap analysis of statistical data reporting in K-12 computing education research: recommendations for improvement. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 591-597).

Pearson, M. I., Castle, S. D., Matz, R. L., Koester, B. P., & Byrd, W. C. (2022). Integrating critical approaches into quantitative STEM equity work. CBE—Life Sciences Education, 21(1), es1.

Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340. doi:10.1136/bmj.c332

Strunk, K. K., & Hoover, P. D. (2019). Quantitative Methods for Social Justice and Equity: Theoretical and Practical Considerations. Research Methods for Social Justice and Equity in Education.

U.S. Institute for Education Sciences. 2022. Standards for Excellence in Education Research. https://ies.ed.gov/seer/index.asp

Vetenskapsrådet, S. (2017). Good research practice. Stockholm: Swedish Research Council.