Inglés (pdf)
Articulo en XML
Referencias del artículo
Traducción automática
Permalink
The effect of sex, repetition history, and level of education are considered both in the assessment of self-regulated learning (SRL), applied at the interface between the areas of Psychology and Education. These variables are examined considering the functioning of SRL, which mobilizes psychological, cognitive, metacognitive, motivational, affective, behavioral, and environmental aspects in the planning and achievement of academic goals, as well as in the students’ self-reflection regarding the results obtained (Paiva & Lourenço, 2012; White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997).
This research focused on establishing standards for the Multidimensional Battery of Self-Regulation for Reading Comprehension (BAMA-Reading; Ferraz & Santos, 2019). This undertaking considered the three variables mentioned above, focusing on investigating measurement invariance as they are aspects inherent to BAMA-Reading target audience-students from Middle School in Brazil. The BAMA-Reading is composed of nine scales that assess the key processes of SRL from a multidimensional perspective. These dimensions include motivation, method, time management, self-perceived behavior, and physical and social environments (Ferraz, 2022; Ferraz & Santos, 2019).
Three BAMA-Reading scales focus on motivation for activities linked to reading comprehension. The Achievement Goals Scale for Reading Comprehension (Escala Metas de Realização para a Compreensão de Leitura; EMR-CL) assesses motivational orientation by achievement goals, considering the learning goal, the performance-approach goal, and the performance-avoidance goal. The Self-Efficacy Scale for Reading Comprehension (Escala Autoeficácia para Compreender a Leitura; EA-CL) assesses self-efficacy for reading comprehension, characterized by students' perception of competence in understanding textual material. In turn, the Causal Attributions Scale for Reading Comprehension (Escala Atribuições de Causas para a Compreensão de Leitura; EAC-CL) assess intrapersonal causality attributions for situations of success and failure in the aforementioned cognitive-linguistic skill, centered on intelligence, effort, difficulty of the text, and luck, as well as the classification of these causes according to the psychological locus, stability and controllability dimensions (Ferraz & Santos, 2019).
In BAMA-Reading, the SRL method dimension is represented by the Reading Comprehension Strategies Scale (Escala Estratégias para Compreender a Leitura; EE-CL), which measures reading strategies. Time management is assessed using the Reading Time Organization Scale (Escala Organização do Tempo para Leitura; EOT-L), which assesses planning, deadline management, and procrastination. The self-perceived behavior dimension is measured by the Self-Monitoring for Reading Scale (Escala Automonitoramento para Ler; EAu-L), which measures students' ability to perceive their performance while reading, and by the Self-Reactions for Reading Comprehension Scale (Escala Autorreações para a Compreensão de Leitura; Ear-CL), which measures adaptive and maladaptive self-reactions for reading comprehension. The Environments for Reading Scale (Escala Ambientes para a Leitura; Eam-L) assesses the physical environment dimension, referring to environmental self-selection and instructional resources to access and perform reading. Finally, the Social Environment for Reading Scale (Escala Ambiente Social para Ler; EAS-L - Ease of Getting Help and Difficulty of Getting Help factors) assesses the social environment dimension, relating to identifying models of good readers and searching for selective help (Ferraz & Santos, 2019).
Except for the EAC-CL, the BAMA-Reading scales have a four-point Likert-type response key (Ferraz & Santos, 2019). The EAC-CL, in turn, has two situations (A and B), referring to whether or not the student is successful in reading comprehension. From them, students indicate a cause that explains them: intelligence, effort, difficulty of the text, or luck. After choosing the cause, they are asked to classify it according to the psychological dimensions of intrapersonal causality attributions - locus, stability, and controllability (for more information about the EAC-CL, see Ferraz et al., 2023).
Returning to this research, Zeidner and Stoeger (2019) suggest that sex variable is essential to comprehend SRL differences. These researchers found that girls used more strategies centered on the course of action (monitoring) and more optimization of personal regulation (planning and structuring of the environment) than boys. However, there were no distinctions for behavioral strategies between female and male students. Focusing on metacognitive skills, Ciascai and Haiduc (2011) identified differences between girls and boys in planning, related to thinking about what is needed to solve a problem and monitoring the task's progress, aimed at evaluating the estimates for completion within of the deadline. Regarding the self-regulation of the social environment, Serafim and Boruchovitch (2010) study showed that in the early school years, girls were more predisposed to ask for help than boys.
In what alludes to the motivational constructs present in SRL, Li (2017) stated that girls apply more strategies linked to the learning goal orientation. It should be mentioned that this personal accomplishment goal is characterized by valuing the intellectual gains obtained with formal education and by creativity and proactivity to carry out tasks (Urdan & Kaplan, 2020). Li (2017) also highlighted that the girls showed greater interest in studies and mastery of SRL in the physical environment. When considering both the effects of the school environment and those of sex, Vantieghem and van Houtte (2015) found that, at the end of the school year, 7th-grade boys who had experienced the pressure of sex conformity, defined as demands from significant people for the students to emit gender-congruent behaviors, had lower academic self-efficacy than girls in a similar situation. In turn, Ferraz et al. (2019) found that the causal attributions reported by girls indicated greater accountability and perceived control over school success and failure outcomes.
Regarding the history of having to repeat school years, in the study conducted by Boruchovitch (2001), this variable was related to learning strategies. The repeating students reported more the absence of strategies to solve a problem. In contrast, those who had no history of repetition reported more attempts to solve problems alone, seeking help only when they considered it necessary. In terms of motivation, Garcia and Boruchovitch (2015) found that students who had not repeated a school year indicated more causal attributions aimed at school success. The same result was found in the study by Ferraz et al. (2019), which also found more causal attributions for school failure on the part of non-failing students, indicative of the notions of commitment to academic performance, be it school success or failure, from the manifestation of controllable causes of internal locus.
Ferraz et al. (2019) also verified the presence of the interaction of the history of repetition and education levels (Elementary School and Middle School) with the causal attributions for school success, as well as for sex and the history of repetition, which encompassed both attributional beliefs for school success and failure. Concerning this last finding, the boys who had repeated a school year scored higher in the causes to explain the positive and negative results through the less controllable psychological dimensions of an external locus, in contrast to the girls who had repeated a school year and the boys and girls without a history of repetition.
Concerning formal instruction, Skibbe et al. (2019) indicated that SRL development is observed from the beginning of schooling, with its mastery being gradual and occurring concomitantly with literacy and reading comprehension. White and DiBenedetto (2015) clarified that, with changes in the education levels, a drop in motivation is commonly identified, which, in turn, can negatively reflect on academic self-efficacy beliefs, due to the particularities of a new environment, where students need to have greater independence and self-management.
Focusing on intrapersonal causal attributions, which prospectively interfere with motivational quality, Miranda et al. (2012) found that the explanatory contribution of this construct for students’ performance from Portugal in Portuguese language and mathematics was greater for Primary Education (33 %) than for Secondary Education (21 %). In the studies by Garcia and Boruchovitch (2015) and Ferraz et al. (2019), there was a greater indication of causal attributions for general school success than in the evaluation of samples from the first stage of Middle School to the second stage. Regarding SRL in the social sphere, at the beginning of schooling, Serafim and Boruchovitch (2010) found more negative perceptions about strategies to ask for help, while in the final years of Middle School, these differences were not observed, indicating that during the schooling, students become aware of the need to establish support networks as a way to optimize their learning.
The studies above suggest that the functioning of SRL may differ due to sex, the history of school year repetition, and the particularities of school years. In the present study, it was assumed that the possible differentiation of SRL according to these variables is not restricted to their specificities and may be associated with the measurement non-invariance of the self-report instruments applied to measure them. Therefore, there is a need to be sure about the measurement invariance before proceeding with the comparison of groups (Cheung & Lau, 2012; Rutkowski & Svetina, 2017), with this verification being one of the practices that give fairness to testing in the process of psychological assessment (American Educational Research Association (AERA) et al., 2014).
Accordingly, the general aim of this research was to propose a preliminary standardization of BAMA-Reading, considering sex, repetition history, and the school years of Middle School. The specific objectives of the study were (a) to investigate the measurement invariance of the BAMA-Reading scales that have a Likert-type response key (EMR-CL, EA-CL, EE-CL, EOT-L, EAu-L,Ear-CL, Eam-L, and EAS-L), considering sex, repetition history and school year; (b) to analyze the interaction effect of EAC-CL due to the variables mentioned above.
Materials and Method
Participants
The sample was composed of 910 Middle School students, (n = 206, 6th year; n = 268, 7th year; n = 258, 8th year; n = 177, 9th year) - one student did not report the school year. The students were from six state public schools, five of them located in the state of São Paulo (n = 829; 91.10 %) and one in Rio Grande do Sul. The students’ ages ranged between 10 and 17 years (M = 12.99; SD = 1.29). Of these students, 470 were female (51.7 %), and 156 indicated having a history of school year repetition (17.3 %), with the reported number years repeated ranging from one to three.
Instruments
Identification questionnaire. This instrument was designed to collect personal information and the student's school records, with an emphasis in this study on age, sex, school year, and the presence of a history of school year repetition.
Multidimensional Battery of Self-Regulation for Reading Comprehension, BAMA-Reading (Bateria Multidimensional da Autorregulação para a Compreensão de Leitura, BAMA-Leitura; Ferraz & Santos, 2019). The BAMA-Reading comprises 116 items distributed across nine scales presented in the introduction to this research. The battery assesses self-regulation for reading comprehension of Brazilian Middle School students. Its construction is based on the six-dimensional conceptual model of SRL of Zimmerman and Risemberg (1997).
Data collection procedure
A Research Ethics Committee of the University of São Francisco (Brazil) approved this study under authorization No. 3.263.350. The school issued the terms of the consent form, as the educational institution had prior authorization from the parent/guardian for this type of project. The students, in turn, registered their acceptance by signing a consent form. Data collection took place in person during class time. The BAMA-Reading was applied in pencil and paper format, and the students took, on average, 40 minutes to complete it.
Data analysis procedure
Softwares used to data analyses: (SPSS, version 22.0) and JASP (Goss-Sampson, 2020). The deviation from the normality of the data was verified using the Shapiro Wilk test, which showed p < .001 in all the BAMA-Reading scales.
In investigating the invariance of the BAMA-Reading scales EMR-CL, EA-CL, EE-CL, EOT-CL, EAu-CL, Ear-CL, Eam-CL, and EAS-L, firstly, the plausibility of the models was investigated through confirmatory factor analysis (CFA). The Robust Diagonally Weighted Least Squares (RDWLS) estimator was recommended for sample data with deviation from normality (Damásio & Dutra, 2017). The internal consistency of the scales was also verified based on McDonald’s alpha (α) and Omega (ω) coefficients, with values > .70.
To assess the invariance of configural, metric, and scalar measures of the BAMA-Reading scales for the sex, repetition history, and school year variables, we used the Multigroup Confirmatory Factor Analysis (MGCFA) - plausibility reference for models: Root Mean Square Error of Approximation (RMSEA; 90 % confidence interval) and Standardized Root Mean Square Residual (SMSR) ≤ .08; Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) ≥ .90 (Brown, 2015). To assess the measurement invariance, we calculated the ∆CFI from the difference between the CFI values of Model 2 (metric) and Model 1 (configural), in which the reference was .01. Upon verification of the measurement invariance in the metric model, the scalar model was evaluated through the difference between the CFI of Model 3 (scalar) and Model 2 (metric) (Cheung & Rensvold, 2002).
Because the CFA result for the EAS-L was not plausible, a new analysis was carried out (CFA 2) considering the modification indices (Table 1). The EAS-L invariance analysis also considered the indices described in the results section (Table 2).
To assess the associations between the EAC-CL with the sex, repetition history, and school year variables, we verified the chi-square test values (χ2). Differentiation between groups was assessed through the value of adjusted residuals, AR ≥ 2.0 (p < .05). In the comparisons involving sex and repetition history, the size of the effect was verified using Phi (f). For the school year using Cramer’s V. In both measures, values close to 0 suggest a weak effect and close to 1 a strong effect (Field, 2009).
The interpretation of the scores of the BAMA- Reading EMR-CL, EA-CL, EE-CL, EOT-CL, EAu-CL, Ear-CL, Eam-CL, and EAS-L scales was based on percentile values, generated by the analysis of quartiles that accounted for 16 groups that considered the set of students based on sex, repetition history and school year. This procedure is indicated for sample data that show deviations from normality (Pasquali, 2019). For the EAC-L, the interpretation of the causal attribution for situations of success and failure in reading comprehension, considering the causes of intelligence, effort, text difficulty, and luck, linked to the locus, stability, and controllability psychological dimensions, is based on the study by Ferraz et al. (2023).
Results
Specific objective a
Table 1 shows that the models tested from the CFA were plausible, except the EAS-L, in which only the CFI achieved the pre-established value. The modification indices indicated the presence of residual covariance between the EAS-L items: item 1, “I look for my classmates to help me”, and item 5, “I think of people who are good at reading comprehension to help me”; and among the item 9, “I thank anyone who helps me understand the text” and item 10, “I praise the person who helped me understand the text”. That said, the result of the second CFA for the EAS-L shown in Table 1 indicates an improvement in the model's fit indices after considering the errors between these pairs of items. Regarding the internal structure of the scales, only the factors Performance Avoidance Goal (EMR-CL) and Time Organization (EOT-L) did not present α and ω coefficients classified as adequate. In turn, the Ear-CL Maladaptive Self-reactions factor presented an α value less than .70. The other unifactorial scales and factors of the BAMA-Reading scales presented appropriate internal structure indices.
Table 2 shows that for sex, the EA-CL, EE-CL, EOT-L, EAu-CL, and EAS-L presented configural, metric, and scalar measurement invariance. These same measurement invariance parameters were identified for repetition history in the EMR-CL and EAS-CL and the school year in the EA-CL and EAS-CL. Considering the sex variable, in the EMR-CL, only measurement invariance in the configural and metric parameters was identified. Still, regarding the sex variable, only the configural invariance parameters for Ear-CL and Eam-CL were verified. Configural invariance was the only parameter identified in the variable repetition history in the EA-CL, EE-CL, EOT-CL, Eau-CL, Ear-CL, and Eam-CL scales, and in the school year variable - EMR-CL scales, EE-CL, EOT-CL, Eau-CL, Ear-CL, and Eam-CL.
Table 2: Measurement Invariance Analysis via MGCFA for BAMA-Reading: Sex, Repetition History and School Year
Notes: ΔCFI value highlighted in bold indicates the measurement invariance of the configural, metric and scalar type. Upon reaching values of ΔCFI > .01, the calculation of the next category of measurement invariance was not performed.
Specific objective b
We analyzed the effect of comparing the Causal Attribution Scale for Reading Comprehension Scale (EAC-CL) items with the sex, repetition history, and school year variables. Regarding sex, the following results were found: Situation A (being successful in reading comprehension), causes, χ2 (3) = 13.437 (p< .01), Cramer’s V = .12 (differences identified in the intelligence and effort causes, both with AR = 2.9, and the luck cause, with AR = 2.0); psychological dimensions, locus, χ2(1) = 2.055 (p = .15), φ = .05; stability, χ2(1) = 0.406 (p = .52), φ = .02; and controllability, χ2(1) = 0.104 (p= .75), φ = .01. Situation B, (being unsuccessful in reading comprehension), causes, χ2(3) = 1.204 (p = .75), Cramer’s V = .04; psychological dimensions, locus, χ2(1) = 2.104 (p = .15), φ = .05; stability, χ2(1) = 3.794 (p = .05), φ = .06 (AR = 1.9); and controllability, χ2(1) = 0.671 (p = .41), φ = .03.
Comparison of the EAC-CL and repetition history resulted in: Situation A, causes, χ2(3) = 14.601 (p < .01), Cramer’s V = .13 (differences identified in the intelligence, with AR= 2.7; text difficulty, AR = 2 .1; and luck causes, AR = 2.0); psychological dimensions, locus, χ2(1) = 6.377 (p = .01), φ = .08 (AR = 2.5); stability, χ2(1) = 4.146 (p = .04), φ = .07 (AR = 2.0); and controllability, χ2(1) = 2.358 (p = .12), φ = .05. Situation B, causes, χ2(3) = 5.141 (p = .16), Cramer’s V = .07; psychological dimensions, locus, χ2(1) = 1.002 (p = .32), φ = .03; stability, χ2(1) = 4.448 (p = .03), φ = .07 (AR = 2.1); and controllability, χ2(1) = 0.147 (p = .70), φ = .01.
For the group comparisons involving the school year and EAC-CL, in Situation A, causes, χ2(9) = 16.236 (p = .06), Cramer’s V = .06; psychological dimensions, locus, χ2(3) = 4.170 (p = .24), φ = .07; stability, χ2(3) = 2.185 (p = .53), φ = .05; and controllability, χ2(3) = 3.534 (p = .32), φ = .06 were verified. For Situation B, the following results were identified: causes, χ2(9) = 21.897 (p < .01), Cramer’s V = .15, related to intelligence with the 6th year (AR = 2.8) and 9th year (AR = 3.0), text difficulty with the 6th year (AR = 2.2), and luck with the 9th year (AR = 2.3); psychological dimensions, locus, χ2(3) = 7.267 (p = .06), φ = .09; stability, χ2(3) = 13.905 (p < .01), φ = .12, referring to the 6th year (AR = 3.6) and 8th year (AR = 2.3); and controllability, χ2(3) = 5.930 (p = .11), φ = .08.
Subsequently, four cut-off points were established through the analysis of quartiles to propose a preliminary version of intragroup standardization for the BAMA-Reading scales (general aim of this research) - EMR-CL, EA-CL, EE-CL, EOT-CL, EAu-CL, Ear-CL, Eam-CL e EAS-L. This distribution considered the sex, repetition history, and school year variables. This proposal for the interpretation of the BAMA- Reading is presented in Appendices A, B, C, and D. The interpretation of the 24th percentile or below presupposes low self-regulation for reading comprehension, taking into account the construct evaluated by each scale of the BAMA- Reading; between the 25th and the 49th percentiles, it was allocated medium-low scores; between the 50th and 74th percentiles, medium-high scores; and for the 75th percentile or above, there is high self-regulation in reading comprehension. The classification proposed in Appendix E suggests an interpretation of the categories indicated by the students in the EAC-CL.
Discussion
In order to meet the general objective of this research, appendices A to D present a first proposal for interpreting the scores, supported by the percentile values that weigh the variables sex, repetition history, and school year. The interpretation of the scores is supported by the percentile values that weight these three variables that did not have measurement invariance confirmed for all of them, considering the three tested parameters (configural, metric, and scalar).
The classification into four groups, involving the low, medium-low, medium-high, and high portions for reading comprehension self-regulation are corroborated by a study conducted by Ferraz (2022), which evaluated the effect of comparing the BAMA-Reading scales with the low and high performance of students in this cognitive-linguistic skill. The researchers found a convergence between low proficiency in reading comprehension and lower battery scores, and vice versa.
In this sense, the results of the investigation of measurement invariance suggest caution in carrying out group comparison analyses of the BAMA-Reading scales with the variables tested for measurement invariance. However, it was recognized that MGCFA might not be the most appropriate method to investigate the measurement invariance for groups containing more than two categories, such as the school year (Kim et al., 2017). Therefore, the performance of a new investigation is recommended, applying, for example, the Multiple Indicators Multiple Causes (MIMIC) method to assess whether these results are repeated, especially regarding the school year, which is characterized as a categorical and continuous variable. Additionally, other invariance analysis procedures for groups containing a disproportionate number of participants are recommended for the history of repetition variable since the number of repeating students was lower than those who had not repeated a school year (17.3 %). From this perspective, an expansion of the sample is also suggested to increase the representativeness of the groups (Rutkowski & Svetina, 2017).
Another way of proceeding with measurement invariance from the MGCFA perspective is to investigate partial invariance, emphasizing scales that do not present the metric and scalar type, to identify the characteristics of the items that tend to favor one group to the detriment of another. Accordingly, the cultural issues present in the school climate that involve gender roles (Vantieghem & van Houtte, 2015) can be considered, as well as the stigma of school failure inherent in the phenomenon of repetition, combined with the school’s criteria for making students repeat a year (if applied at any time or by continuous progression) (Ferraz et al., 2019), and to assess whether the particularities of the functioning of SRL linked to the procedures that lead to the reading comprehension exposed in the BAMA-Reading items are, in fact, representative and accessible for students in the initial and final grades of Middle School in public schools (AERA et al., 2014).
In turn, in the EAC-L, the results showed that sex, repetition history, and school year have, to some extent, an effect on students’ beliefs of causal attribution to situations of success and failure in reading comprehension, as indicated by Garcia and Boruchovitch (2015), Graham (2020), Ferraz et al. (2019) and Miranda et al. (2012). Specifically, it was evidenced that the attribution of the intelligence, effort, and luck causes in the context of being a successful student in reading comprehension differed due to the sex variable, with this finding supported by the study by Ferraz et al. (2019), which focused on general causal attributions for school performance.
Also, within the scope of success for reading comprehension, it was found that the history of repetition acts differently in the attribution of the intelligence, text difficulty, and luck causes. It was found that the perception of students who repeated or not a school year can be different in the locus and stability psychological dimensions. This last result alludes to the impact of successive experiences of failure on students’ motivation to adhere to reading activities, which can have a negative impact on the development of proficiency in reading comprehension. A drop in motivational quality is assumed when a scenario is identified in which students classify their success as caused by an external locus. Therefore, they do not give merit to their achievements; in failure situations, distorted notions regarding the stability of the cause, assessed as immutable, reduce the possibility of overcoming difficulties (Ferraz et al., 2019; Graham, 2020).
Concerning the condition of being successful, presented in Situation A of the EAC-CL, the attribution of intelligence, text difficulty, and luck differed in the 6th and 9th years and the stability dimension in the 6th and 8th years. Miranda et al. (2012) found distinctions in the contribution of causal attributions to school performance considering education levels. In the present study, it is conjectured that something similar occurred since the causal attributions differed at the beginning and the end of Middle School, except for the 8th year, which is an intermediate year of this cycle. However, caution should be taken with this type of assumption, as the research method used in both studies was cross-sectional. Therefore, the results represent isolated moments of each school year and not a continuum that could be better evaluated in studies with longitudinal designs (Stocker & Faria, 2018). The proposed interpretation of the EAC-CL (Appendix E) comes from the research carried out by Ferraz (2022). It was theoretically based on frameworks that established and revised the theory of intrapersonal causal attributions (Graham, 2020; Weiner, 2010).
The recommendations and implications of this research indicate that comparisons between groups involving sex, repetition history, and school year for the BAMA-Reading scales should only be carried out through new measurement invariance analyses. This warning extends to establishing procedures for analyzing data from studies that propose to carry out this type of investigation (Cheung & Lau, 2012; Kim et al., 2017; Rutkowski & Svetina, 2017). This care is a way to avoid unfairness with those involved in psychological testing applied in the educational context since the variables investigated here are part of the battery’s target audience (AERA et al., 2014).
Therefore, we recognized that the initial proposal for interpreting the scales, despite the differences in scores regarding sex, repetition history, and school year, may still contain the interference of metric and scalar type invariance. New studies with the BAMA-Reading also serve to evaluate the relevance of this proposal and, in addition, to verify whether the structuring of intra-group standards is the most appropriate procedure when considering potential advances in establishing standards based on statistical techniques from Item Response Theory. In this sense, there is a need to expand the sample of students to other regions of Brazil to increase sample representation and enable more robust data analysis. These considerations are intended to formulate a research agenda with the BAMA-Reading to make it consistent, both in terms of the theoretical contribution of the constructs evaluated and in meeting the psychometric assumptions, which can also be a valuable and efficient tool from the point of view of the practice.
