Single Paper Session 35

Session Page

Session Information

This page shows the session details and the presentations assigned to this session.

Back to programme overview
Add this session to calendar

Assessing Higher-Order Writing Skills: Development and Validation of a Diagnostic Instrument

Keywords
evaluation and assessment of writing competence;text composition;writing at school;writing skills
Domain
Assessment and Evaluation

Abstract

Writing competence is central to academic success and participation beyond school. (Becker-Mrotzek, 2014). Current models conceptualize text production as a multilevel process, with higher-order composing skills—such as coherence and cohesion, audience awareness, and information management—being particularly important for text quality (Hennes, 2020). To support individualized instruction, teachers must accurately assess these subskills and identify student’s strengths and weaknesses (Graham et al., 2012). However, existing diagnostic instruments rarely target these higher-order composing competences in a differentiated way (Hennes, 2020). This study presents the development and validation of a writing test designed to assess four key dimensions of composing: global coherence, local cohesion, audience awareness, and information management. The instrument was developed for students in grades 4 to 9 and comprises ten tasks, each targeting one dimension. An extended text production task served as the criterion variable, with text quality evaluated globally using comparative judgments. Validation data were collected from students in grades 4 (N = 91), 6 (N = 135), and 9 (N = 65) in Germany; grade 9 was excluded from the analysis due to ceiling effects. For grades 4 and 6, regression analyses identified tasks that significantly predicted text quality; together, these explained substantial variance. Subsequent exploratory factor analyses – conducted to examine whether the remaining tasks reflected the hypothesized multidimensional structure – yielded a single-factor solution for both grade levels. These findings raise important questions regarding the relationship between statistical dimensionality and diagnostic utility, particularly as current models of text production assume a multidimensional structure. Implications for educational practice and theoretical models of text composition will be discussed. References Becker-Mrotzek, M. (2014). Schreibkompetenz. In J. Grabowski (Hrsg.), Sinn und Unsinn von Kompetenzen: Fähigkeitskonzepte im Bereich von Sprache, Medien und Kultur (1. Aufl., S. 51–72). Verlag Barbara Budrich. Graham, S., McKeown, D., Kiuhara, S., & Harris, K. R. (2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational Psychology, 104(4), 879–896. Hennes, A.-K. (2020). Schreibprodukte bewerten: Die Rolle der Expertise bei der Bewertung der Textproduktionskompetenz [KUPS (Kölner UniversitätsPublikationsServer)].

Effectiveness of single-case writing interventions (2008-2025): Preliminary meta-analysis findings

Keywords
intervention study;meta-analysis;writing at school;writing instruction
Domain
-

Abstract

Research Topic and AimThis presentation introduces a pre-registered meta-analysis examining the effectiveness of writing interventions tested through single-case experimental designs from 2008 to 2025. This work is being conducted under an EARLI-funded network composed of writing researchers from three countries.Theoretical Framework Building on Rogers and Graham’s (2008) and Casola’s (2023) works, the meta-analysis targets school-based writing interventions for Grade 1-12 students to estimate their impact on writing performance while identifying student- and intervention-level moderators of effectiveness.Methodological DesignDatabase searches conducted in June 2025 using PsycINFO, Education Source Ultimate, and Web of Science initially yielded 4,753 records. Four raters screened the abstracts of these records (95-96% of interrater agreement) and retained 198 papers for full-text screening. Of these, 135 fulfilled the following inclusion criteria: single-case experimental design, grades 1-12 students, included a baseline with at least three data points, reported at least one quantitative writing or motivational outcome, and provided sufficient information to compute effect sizes. Once the database searches are complemented with hand searches, the raw single-case data of the selected studies will be extracted using WebPlotDigitizer 4.6 and coded for key moderators at the student (e.g., grade level, educational status) and intervention levels (e.g., type of writing intervention, provider). Multilevel modeling will be used to estimate intervention effects.Preliminary FindingsPreliminary coding of the 135 studies identified so far indicated a predominance of primary-school and special education samples; frequent use of multiple-baseline and multiple-probe designs; researchers as the main intervention providers; and a firm reliance on writing quality, length, and genre elements as outcome measures, with relatively few studies including objective motivational measures. Preliminary statistical results will be presented at the conference.Relevance to the Writing DomainThis work will provide updated guidance for evidence-based writing instruction in Grades 1-12 and inform the design of single-case literacy interventions across three countries. ReferencesCasola, M. A. (2023). Single-subject writing strategy instruction: A meta-analysis. [Unpublished master’s dissertation]. The University of Western Ontario, Canada).Rogers, L. A., & Graham, S. (2008). A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology, 100(4), 879–906. https://doi.org/10.1037/0022-0663.100.4.879

Modelling the Subskills of Writing in Instructional Texts

Keywords
evaluation and assessment of writing competence;writing at school;writing models;writing skills
Domain
-

Abstract

The Cascaded Model of Writing (CASMOW) – a current writing model – shows that in lower secondary school, where lower-level skills are largely automated, these skills only contribute indirectly to text quality via higher-level writing skills such as cohesion and lexical diversity, which in turn have a direct impact on text quality. To date, CASMOW has only been validated for narrative texts (Philippek et al., 2025). However, studies examining individual writing skills independently of the model suggest that their influence vary depending on the text genre (Beers & Naggy, 2009). The present study therefore investigated the applicability of CASMOW to instructional texts.The sample comprised 150 students in grades 5 to 7, aged ten to thirteen (M(age) = 11.21, SD = 0.93; 67 girls). Participants wrote an instructional text, which was analysed for lexical diversity and text quality. Executive functions, handwriting fluency, spelling, grammatical skills and cohesion were assessed using standardised tests. All variables were transferred to a structural equation model according to the CASMOW structure.Preliminary results showed that lower-level skills mainly influenced text quality indirectly, which is consistent with the results for narrative texts. In contrast to Philippek et al. (2025), however, spelling had a direct influence on text quality. Higher-level skills also showed a different pattern: lexical diversity only indirectly influenced text quality via text length, while cohesion had no influence. Overall, the model explained 35% of the variance in text quality. Since a large part of the variance remains unexplained, there must be other higher-level writing skills that are more relevant to writing instructions and should be added to the model. Furthermore, the results emphasise that effective writing instruction should be genre-specific and not generalised across all text types. References Beers, S. F., & Nagy, W. E. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre? Reading and Writing, 22(2), 185–200. https://doi.org/10.1007/ s11145-007-9107-5. Philippek, J., Kreutz, R. M., Hennes, A.‑K., Schmidt, B. M. & Schabmann, A. (2025). The contributions of executive functions, transcription skills and text-specific skills to text quality in narratives. Reading and Writing (38), 651–670. https://doi.org/10.1007/s11145-024-10528-5