Reflecting on Positionality in Analyzing and Interpreting Data

Researchers must reflect on their positionality and methodological shortcomings when analyzing and interpreting data  (American Psychological Association, 2021). Context, power dynamics, and participant information should be considered to minimize biases (German Research Foundation, 2019). Transparent data analysis procedures are crucial, including checking and reporting assumptions for quantitative analyses and addressing outliers and limitations (Vetenskapsrådet, 2017). Clear definitions of outcome constructs and measures help ensure reliable and accurate analysis, emphasizing theoretical or practical significance for educators and decision-makers.

Aligning Interpretation with Research Questions

Interpretation should align with research questions, study benefits, harms, and relevant evidence, clarifying the study’s implications (Schulz et al., 2010). Standards from the APA, AERA, What Works Clearinghouse (WWC), and CONSORT offer guidance on methodology, outcomes, statistical analysis, and interpretation (McGill & Decker, 2020). Quantitative data analysis should be executed properly, acknowledging that statistical measures are not entirely objective and considering context and power dynamics (Strunk & Hoover, 2019; Pearson et al., 2022).

Holistic and Reflexive Approaches to Data

Adopting a holistic approach to data analysis considers context, power dynamics, and community experiences (Strunk & Hoover, 2019; Pearson et al., 2022). Reflexive data analysis recognizes anomalies as representing individuals’ lived experiences and ensures interpretations reflect the communities studied (Strunk & Hoover, 2019). Challenging previous findings or expectations can provide insights into participants’ lived experiences (Vetenskapsrådet, 2017), with a focus on improving research outcomes (German Research Foundation, 2019; U.S. Institute for Education Sciences, 2022).

Interpreting Data

Data interpretation requires awareness of cultural influences and benefits from researchers who share commonalities with participants (Pearson et al., 2022). Researchers should conduct asset-based interpretations of non-dominant communities, reflecting their own experiences and collaborating with others who share participant experiences. This approach respects participants and leverages collaborations to enhance the research process (Hood, 1998; Kirkhart, 1995).

Strategies for Analyzing and Interpreting Data

  • Ensure that the research methods used are adequate for addressing the study research questions and consider their limitations and the impact of available tools on addressing the research questions.
  • Assess relevant assumptions of each statistical analysis performed, and modify interpretations of results accordingly, if necessary.
  • Consider study context, power-dynamics, and situational contexts to inform the selection of data analysis techniques (Pearson et. al., 2022).
  • Ensure the interpretation matches the results of the study and assess benefits and harms that might occur while considering additional supporting evidence and crucial information (Schulz et al., 2010).
  • Utilise participant charactersitics (e.g., grade levels) when evaluating results to understand the impact on different groups (German Research Foundation, 2019; U.S. Institute for Education Sciences, 2022).
  • Consider alternative viewpoints or the individual’s experience when results do not match expectations.
  • Seek collaboration with colleagues and study participants who can offer a broad perpsective, and continuously reflect to ensure a more reliable interpretation of results.

Additional Resources

References

American Psychological Association. (2021). Journal Article Reporting Standards (JARS). Retrieved from https://apastyle.apa.org/jars

German Research Foundation. 2019. Guidelines for Safeguarding Good Research Practice. https://www.htwg-konstanz.de/fileadmin/pub/forschung/Forschungsreferat/Guidelines_for_Safeguarding_Good_Researche_Practice_DFG__2019.pdf

Hood, S. (1998). Responsive evaluation Amistad style: Perspectives of one African American evaluator. Proceedings of the Stake Symposium on Educational Evaluation, 101–112. ERIC.

Kirkhart, K. E. (1995). 1994 conference theme: Evaluation and social justice seeking multicultural validity: A postcard from the road. Evaluation practice, 16(1), 1-12.

McGill, M. M., & Decker, A. (2020, February). A gap analysis of statistical data reporting in K-12 computing education research: recommendations for improvement. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 591-597).

Pearson, M. I., Castle, S. D., Matz, R. L., Koester, B. P., & Byrd, W. C. (2022). Integrating critical approaches into quantitative STEM equity work. CBE—Life Sciences Education, 21(1), es1.

Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340. doi:10.1136/bmj.c332

Strunk, K. K., & Hoover, P. D. (2019). Quantitative Methods for Social Justice and Equity: Theoretical and Practical Considerations. Research Methods for Social Justice and Equity in Education.

U.S. Institute for Education Sciences. 2022. Standards for Excellence in Education Research. https://ies.ed.gov/seer/index.asp

Vetenskapsrådet, S. (2017). Good research practice. Stockholm: Swedish Research Council.