 Research
 Open access
 Published:
The factor structure of the Motivated Strategies for Learning Questionnaire (MSLQ): new methodological approaches and evidence
Psicologia: Reflexão e Crítica volume 36, Article number: 38 (2023)
Abstract
Background
The area of selfregulated learning integrates the fields of metacognition and selfregulation and assumes that the student is an active processor of information capable of selfregulating his learning by putting together the cognitive, metacognitive, and motivational components. The Motivated Strategies for Learning Questionnaire (MSLQ) is a benchmark for the measurement of selfregulated learning. However, the field of study does not show adequate evidence of its structural validity. The vast majority of studies involving this question present serious methodological mistakes, compromising the evidence of validity.
Objective
Our study investigates the structural validity of MSLQ including all 15 scales and corrects relevant mistakes in the previous studies.
Method
We tested different models through item confirmatory factor analysis in a convenience sample of 670 college students (M = 22.8 years, SD = 5.2) from a public Brazilian university in the technological area. The models with the ML, MLR, MLM and WLMSV estimators.
Results
Only WLSMV produced models with acceptable fit. The final model has a bifactor structure with a general factor (selfregulated learning), 15 components as firstorder factors, and four broad components as secondorder factors. Twelve firstorder components, all secondorder components and the general factor had acceptable reliability. The components’ elaboration, intrinsic goal orientation and metacognitive selfregulation, did not show acceptable reliability, in terms of McDonald’s omega.
Conclusion
Considering the worldwide importance of the MSLQ, we do not recommend the use of the measurement of these components for clinical practice and psychoeducational diagnosis until new studies show that this low reliability only occurs in our sample. Our study shows new evidence, correcting many previous methodological mistakes and producing initial evidence favorable to the factor structure of the MSLQ.
The construct selfregulated learning emerged in the 1980s when researchers interested in integrating knowledge from the fields of metacognition and selfregulation studies decided to deepen their understanding of academic learning (Schunk & Greene, 2017). Mannion (2018, Impact Articles section) proposes that “selfregulated learning is the application of metacognition and selfregulation to learning”, on the basis that “metacognition is monitoring and controlling your thought processes” and “selfregulation is monitoring and controlling your emotions and behaviors”. Selfregulated learning is defined by the area as the dynamic interaction personenvironmenttask, necessarily involving three broad components: cognition, metacognition and motivation. The area stipulates that through these components the subject interacts with the objects of knowledge and selfregulates, considering the context of the environment and the demands of the task (Dinsmore et al., 2008).
The Motivated Strategies for Learning Questionnaire (MSLQ) is a traditional, consolidated, and widely used measure of selfregulated learning and has been translated into several languages and applied in various cultural contexts (Duncan & McKeachie, 2005). The reviews by Broadbent and Poon (2015) and Roth et al., (2016), show that the MSLQ was the most widely used selfregulated learning assessment instrument in higher education. The first review shows that the MSLQ was employed in 75% of the 12 studies that assessed selfregulated learning in the context of virtual learning, while the second shows that the MSLQ was used in 62% of the 152 studies reviewed.
The MSLQ is theoretically underpinned by sociocognitive perspective. According to this view, the students are responsible for their own learning process. This conception assumes that the students regulate their environment, emotions, motivations, and strategies to optimize the way they learn (Duncan & McKeachie, 2005; Pintrich et al., 1991).
The MSLQ follows the basic guidelines of the field of studies in selfregulated learning and focuses its measurement on motivational, cognitive, and metacognitive components. The MSLQ has 6 motivation scales and 9 scales of cognitive and metacognitive strategies (see Table 1). These scales are conceptually based on previous theories, such as Bandura’s selfefficacy, Eccles and Wigfield’s expectancyvalue, and Deci and Ryan’s selfdetermination (Nanol, 2015).
Psychological services aiming to help college students overcome learning difficulties or enhance selfregulatory processes use the MSLQ as a clinical diagnostic tool and intervention assessment (Casali et al., 2022). MSLQ is also a predictor of academic performance. Credé and Phillips’ (2011) metaanalysis shows that the MSLQ scales that best predict performance in college subjects are effort regulation (ρ = .40), selfefficacy for learning and performance (ρ = .37), and time and study environment (ρ = .31).
Despite its importance, the field of studies shows no evidence that the factor structure of the MSLQ is valid (Liu et al., 2019). Almost all the studies on its factor structure show evidence that refutes the factor structure of the MSLQ and its validity. In addition, almost all studies show methodological inconsistencies that compromise the evidence. The most important methodological inconsistencies are classified into three categories (Table 2). Given that inspection of the structural validity of the MSLQ requires the inclusion of all 81 items in their original form, we disregarded evidence from studies that removed items or tested alternative models derived from exploratory factor analysis (Hands & Limniou, 2023; VillarrealFernández & ArroyaveGiraldo, 2022; Wang et al., 2022; Zhou & Wang, 2021).
The first inconsistency involves the analysis of the general factor, which represents the selfregulated learning construct itself. This factor is rarely investigated by studies examining the structural validity of the MSLQ. This is an unjustified situation, since the measure of selfregulated learning itself demands the assessment of this construct. Hilpert et al. (2013) and Jackson (2018) are the only studies on the factor structure of the MSLQ that included components of motivation and strategy in the same model, making it possible to inspect the general factor of selfregulated learning. However, only Hilpert et al. (2013) inserted the general factor in the tested models (Table 2). Unfortunately, their study has important limitations regarding the evidence on the general factor. They did not use the 81 questionnaire items as observable variables, but the summed scores of the 15 scales. In summary, they tested two models with the general factor that did not have acceptable fit rates. In one of the models, the general factor was the only latent variable that explained the variance of the scores (onedimensional model, CFI = .57; RMSEA = .20 and SRMR = .13) while in the other model, the general factor was a secondorder latent variable explaining four firstorder latent variables (hierarchical model, CFI = .73, RMSEA = .16 and SRMR = .13).
The second inconsistency concerns the use of the correct estimator to carry out the factor analysis of items. To choose the estimator correctly is a central methodological decision since a wrong choice substantially reduces the accuracy of the parameters and increases the likelihood of the model being wrongly rejected (Li, 2016). To properly select an estimator, it is mandatory to inspect the multivariate normality of the items (Li, 2016), and we can see in Table 2 that this is not done or reported by most studies. The maximum likelihood estimator is suitable only when the items follow a multivariate normal distribution, but it is often used by studies without testing for the normality of the data (Li, 2016). Hamilton and Akhter (2009) and Liu et al. (2019) are the only ones who performed this analysis. Although both do not present the statistical results, they report that the data did not exhibit multivariate normality. Even though, Hamilton and Akhter (2009) mistakenly selected maximum likelihood for their analyses. Liu et al. (2019), on the other hand, are the only ones who used a suitable estimator for nonnormal data, the maximum likelihood robust estimator.
The third inconsistency refers to the authors’ conclusion that these MSLQ studies show favorable evidence of validity, while almost all the studies show inadequate fit rates for the models tested. Only three studies show models with acceptable fit (Table 2). However, two of these studies exhibit relevant methodological problems that compromise their evidence. Rotgans and Schmidt (2010) found an acceptable fit (CFI = .94 and RMSEA = .05) for a model with the following correlated motivation components: intrinsic goal orientation, extrinsic goal orientation, control of learning beliefs, and selfefficacy for learning and performance. On the other hand, they changed the original MSLQ response scale, so their evidence cannot be transposed to the original MSLQ. Whereas Şen et al. (2014) found an acceptable fit for two models: the motivation model of Rotgans and Schmidt (2010) and the correlated strategy components model (organization, elaboration, metacognitive selfregulation, effort regulation, and time and study environment). Both models showed CFI = .99 and RMSEA = .07. However, this study did not perform factor analysis of items because the authors used the summed scores of the scales as observable variables. In turn, Chow and Chapman (2017) found an acceptable fit for the motivation model, including all six components of the MSLQ correlated (CFI = .95, SRMR = .076 and NNFI = .95). While the evidence from this study is not incorrect, the failure to include the strategy components in the same model compromises the evidence because conceptually, selfregulated learning is an articulation between motivations and strategies.
Therefore, our study aims to correct those inconsistencies and provide solid evidence about the factor structure of the MSLQ. Furthermore, we hope to change this path of inadequate methodological practices by suggesting a correct way to conduct confirmatory factor analysis of items for an adequate analysis of the MSLQ. Also, by pointing out the inconsistencies, we do not intend to deride previous studies. We recognize that in the early 1990s, factor analysis of items was not a wellknown technique, with very few software to suitably conduct it. However, this context has changed considerably since the 2010s, and the field of study of selfregulated learning needs to use better methodological practices if it is to build better evidence about its constructs.
Method
Participants
The participants in this study come from a convenience sample of 670 students from a Brazilian public university in the field of technology. The majority is male (N = 436, 65.1%), with a mean age of 22.8 years (SD = 5.2), minimum age of 17 years and maximum age of 52 years, with an interquartile range of 4 years. Considering the age classification suggested by Rae Simpson (2018), this sample has 23 (3.4%) adolescents (< 18 years), 548 (81.8%) young adults (18–25 years), and 99 (14.8) adults (> 25 years). This age variation is typical of many Brazilian universities, in which the majority are young adults, but with the presence of both older adolescents and adults who enter university later in life.
Instrument
The MSLQ is a selfreport questionnaire about selfregulated learning in the context of higher education. It comprises 15 scales, six of them belong to the domain of motivations and nine to the domain of learning strategies. The questionnaire has 81 items: 31 concerning motivations and 50 concerning strategies. Table 1 describes the 15 firstorder components of the MSLQ measured by the 15 scales. Each item presents a statement that describes a selfregulatory behavior in the university academic context. The respondent must rate each statement by choosing one option from a sevenpoint scale: the lowest value indicates that that behavior is not representative of their behavior, and the highest value expresses that that behavior is very representative.
Our study uses a version of the MSLQ with items in Brazilian Portuguese. The motivation items were translated by Ruiz (2005) and are used with the author’s consent. As we did not find a translated version of the strategy items, the 50 items in this domain were translated by the authors of this article following the procedures recommended by the International Test Commission (2017).
Data collection
The data came from two samples collected in 2018 and 2019 at a public university with students from technology departments. Three instruments were applied to the first sample, in the following order: (1) Metacognitive Monitoring Test, approximately 40 minutes (Gomes et al., 2021b); (2) TAbVideogame, approximately 10 minutes (Gomes et al., 2020a); (3) MSLQ, approximately 15 minutes. Five instruments were applied to the second sample: (1) Metacognitive Monitoring Test, approximately 40 minutes (Gomes et al., 2021b); (2) TAbVideogame, approximately 10 minutes; (3) Learning Approaches Scale, approximately 5 minutes (Gomes et al., 2011); (4) MSLQ, approximately 15 minutes; and (5) SLATThinking, approximately 40 minutes (Gomes, 2021a). Data collection followed ethical guidelines (CAAE: 73453317.1.0000.0118). This study was fully supported by the university, which included the director of education and several professors. One of the researchers was invited to give a lecture about the study to the university’s students. Internal disclosures were made by the university, via emails and social networks, presenting the purpose of the study and a link that allowed them to answer the instruments on the SurveyMonkey platform. Upon opening the link, the participant only answered the instruments if he or she consented to the Informed Consent Form. All participants were volunteers and did not receive financial compensation. Participants did not have a time limit for answering the tests, so they could rest between them. The answers were automatically saved when the students submitted their answers as they advanced to the next page.
Data analyses
To correct the inconsistencies of the previous studies, we applied some methodological steps. First, we inspected the multivariate normality of the 81 items of the MSLQ and reported the results. Furthermore, we used item confirmatory factor analysis when assessing the factor structure of the MSLQ, including all 81 items encompassing the six motivations components and the nine strategies components. Finally, we tested models with the general factor representing selfregulated learning.
The data analysis involved planning a few steps. In the first step, we tested two fundamental models stipulated by the theory, including only firstorder factors. According to the theory, selfregulated learning can be conceived in terms of 15 specific components or in terms of two broad components, motivation and strategy. So, the first step involves testing two models that represent these two possibilities. The 15component specific firstorder model defines that each of the 15 components in Table 1 respectively explains the items on each scale. This model is represented in Fig. 1 (Model 1A). For spacesaving reasons, there are only two rectangles representing the items of each scale; however, it is important to note that each scale has a broad number of items. The firstorder two broad factor model, on the other hand, assumes that all 31 motivation items are explained by the broad motivation component while the 50 strategy items are explained by the broad strategy component (Fig. 1, Model 1B).
The second step of the analysis would be performed only if the specific 15 firstorder component model had a better fit than the two broad firstorder component model and then, the 15 specific components would be explained by different secondorder factors. The first model assumes that motivation and strategy, the two correlated secondorder factors, explain the firstorder factors (Model 2A) and is supported by the arguments of Pintrich et al. (1991). Figure 1 shows how the secondorder factors explain the firstorder factors. The second model assumes that three correlated secondorder factors explain the firstorder factors (Model 2B) and is supported by the arguments of Pintrich et al. (1991). The third model assumes that four correlated secondorder factors explain the firstorder factors (see Fig. 1, Model 2C), which is supported by the metaanalysis of Credé and Phillips (2011).
The third step includes two possibilities. If the second step had been performed, then the third step would evaluate all the models from step 2, adding a firstorder general factor orthogonal to the other components (Fig. 2, Models 3A, 3B and 3C). Otherwise, if the firstorder broad twofactor model showed the best fit in the first step analysis, then the general factor would be added to this model and orthogonalized with the other two components (Fig. 2, Model 3D). The third step is crucial, as it assesses the empirical plausibility of the selfregulated learning construct itself.
Since studies on the structural validity of the MSLQ usually use the maximum likelihood (ML) estimator to run confirmatory factor analyses of items, we will also use this estimator to run our models. However, if the result of the Mardia test has a pvalue well below .05, the weighted least squares means and variance adjusted (WLSMV) estimator, as well as two robust maximum likelihood estimators, the HuberWhite maximum likelihood robust (MLR) and SatorraBentler’s maximum likelihood mean adjusted (MLM) will be used (Satorra, 2000). Li (2016) argues that the WLSMV estimator is the most appropriate for data where multivariate normality is strongly violated, while robust maximum likelihood estimators would be appropriate for milder violations.
Confirmatory item factor analysis and factor reliability analysis were performed using the packages lavaan, version 0.6.11 (Rosseel et al., 2022), and semTools, version 0.5.5 (Jorgensen et al., 2021) of the R software, version 4.2 (Core Team, 2022). The tested models would be rejected if they had CFI < .90 or RMSEA ≥ .10 (Thakkar, 2020). One model would be considered better than the other when it showed a statistically significant difference in terms of chisquares and degrees of freedom (Δχ^{2} [Δdf]).
The reliability of the factor scores was assessed using Cronbach’s alpha and McDonald’s omega for the firstorder factors, and the general factor as well as Cronbach’s alpha and hierarchical omega (reliabilityL2 semTools R package function) was used for the secondorder factors. Factor scores with alpha ≥ .60 and omega ≥ .50 would be considered reliable (Gomes et al., 2020, b, c; Valentini et al., 2015). It should be noted that the cutoff point of .50 for omega was not created for bifactor models. We know that bifactor models in which the general factor is orthogonal to the other factors are very demanding models that tend to decrease the factor load of the specific components and consequently decrease their omega. In this case, if the usual cutoff point is .50, it would be plausible to assume that, approximately, an omega of .40 would still be acceptable for bifactor models.
Results and discussion
The results shown in Table 3 indicate that the only models tested exhibiting an acceptable fit were those estimated by the WLSMV. These results are not surprising, as the Mardia test suggests a strong violation of multivariate normality of the 81 items of the MSLQ for the sample in this study (skewness = 136,363.4, p < .001; kurtosis = 98.7, p < .001). Li (2016) had already argued that robust ML estimators are not appropriate for data that strongly violate multivariate normality, while WLSMV would be the appropriate estimator. The violation of multivariate normality is also not surprising. This result is congruent with the only study that has examined the multivariate normality of the MSLQ (Hamilton & Akhter, 2009; Rotgans & Schmidt, 2010). In the introduction, we showed that only a single study used a robust version of the ML estimator for nonnormal data (Liu et al., 2019). Although many studies did not report the estimator (Table 2), it is very likely that maximum likelihood was used since this is the default estimator in many software programs. Considering that model fit tends to worsen with the use of inadequate estimators (Li, 2016), it is very likely that the use of maximum likelihood in the MSLQ structural validity analysis is the main reason for the models’ rejection.
Only some models tested showed an acceptable fit (Table 3). In the first step of model analysis, only the 15factor specific model (Model 1A) showed acceptable fit (Table 3). This result is relevant from a theoretical standpoint because it supports unprecedented evidence on the structural validity of the MSLQ with the presence of all the components proposed by Pintrich et al. (1991) in the same model. Given the theoretical importance of this model and the fact that its fit was acceptable, we present its factor loadings in Tables 5, 6, 7 and 8, along with the factor loadings of the final model, which will be presented later. The factor loadings of this model can be seen in the third column of Tables 5, 6, 7 and 8. The fourth column of Tables 5, 6, 7 and 8 displays the means, the standard deviations of the factor loadings of each component, as well as their alpha and omega. The results of the 15factor specific model (Model 1A) suggest that the items of each of the 15 scales of the MSLQ are relevant for gauging the 15 components, as the lowest mean factor loading of the components on the items was .49 and the highest was. 82 (see Tables 5, 6, 7 and 8). The items 33 and 57 of the metacognitive selfregulation factor are exception, as they exhibited a factor loading of less than .30. All other items exhibited a loading equal to or greater than .40. All the MSLQ components are reliable in the 15factor specific firstorder model, showing alpha ≥ .60 and omega ≥ .50. Although alpha is the most widely used reliability index, the psychometric literature has recommended the use of omega or similar since it produces a more accurate estimate of the reliability of latent variables by considering their factor loadings (Cho, 2016). To our knowledge, only one study investigated the reliability of the MSLQ using the omega (Liu et al., 2019). However, the authors found no evidence of structural validity of the instrument. As such, our study is the first to show that components of the MSLQ are reliable via McDonald’s omega. Despite the promising evidence found in the 15 componentspecific model, it is relevant to point out that it is not the final model, which would best represent the factor structure of the MSLQ.
In the second step, the fourfactor model (Model 2C) was the only one not to be rejected, despite its moderate fit (Table 3). The results of the second step of the model analysis are relevant from a theoretical point of view, as they indicate that the traditional model of two broad components, one of motivation and the other of strategy, does not hold empirically. In short, the results of the second step suggest that the broad component of strategy should be divided into three distinct parts: learning strategy, regulation strategy, and social strategy, according to the theoretical proposition of Credé and Phillips (2011).
In the third step of the model analysis, there was a need for four constraints in the four broad factors model (Model 3C). The factor loadings of items 33, 57, and 62 were constrained to zero on the specific factors because the general factor explained all the common variance of these items. Constraining to zero certain loadings on the specific factors is common in bifactor models since in these models, the general factor and the specific factors compete to explain the common variance of the items. In this context, when the general factor explains all the item’s common variance, the factor loading of the item on the specific factor can become negative. This does not mean that this item has a negative relationship with the specific factor, but only that all the common variance of this item was explained by the general factor. To correct this negative loading, one should constrain it to zero (Reise et al., 2010). In addition to the constraint applied to the three items, the factor loading of the specific component of critical thinking had its factor loading constrained to zero on the broad component of learning strategies.
The models with the general factor and two and four broad factors (Models 3A and 3C) showed acceptable fit, while the model with the general factor and three broad factors (Model 3B) did not converge. Model 3C, among all models with acceptable fit, was the one with the highest CFI and lowest RMSEA. When we compare this model with the other models with acceptable fit, Models 1A, 2C, and 3A, we can see that Model 3C is the model with the best fit and so, it is considered the final model (Table 4).
The factor loadings of the firstorder components of the final model can be seen in the fifth column of Tables 5, 6, 7 and 8, while the factor loadings of the general factor of this model can be seen in the seventh column of Tables 5, 6, 7 and 8. The factor loadings of the four broad factors in relation to the 15 specific components appear in Table 9.
The factor structure of the final model provides new evidence of the empirical identification of the construct of selfregulated learning through the insertion of the general factor in this model. The general factor exhibited a mean factor loading of .39 (SD = .15), which is a good average for bifactor models, considering that in these models the general factor and the specific factors compete to explain the common variance of the items. However, this does not mean that the general factor had a relevant role in explaining the variance of all MSLQ items. The literature usually points out that an item is relevantly loaded by a factor when it has at least a factor loading of .40 (Howard & Henderson, 2023). Nevertheless, we understand that this cutoff point is not relevant for bifactor models, given that we have already explained the factor competition to explain the common variance of the items. We did not find in the literature any suggested cutoff point and, therefore, we proposed a cutoff point of .20 to consider that the item had a relevant factor loading. We identified that nine items had a loading of less than .20 on the overall factor (Tables 5, 6, 7 and 8), which is equivalent to about only 10% of the MSLQ items. It should be noted that these items include all five items in the test anxiety component. This result is interesting because it brings a new challenge to the field. From a theoretical perspective, it has been expected by theory that test anxiety items would load negatively on the overall factor, as theory states that the test anxiety component is counterproductive to selfregulated learning (Chow & Chapman, 2017). Regarding the other four items that did not show a factor loading greater than or equal to .20 in the overall factor, it is relevant to perform a future inspection of the content of their sentences to verify whether this content may have contributed to this low loading.
In addition, the overall factor proved to be reliable, as its Cronbach’s alpha was .93 and its McDonald’s omega was .81. From a theoretical point of view, the presence of a valid and reliable overall factor empirically supports the theoretical argument of Pintrich et al. (1991) about the existence of the selfregulated learning construct. In practical terms, the valid and reliable overall factor indicates that the user can apply the MSLQ to perform the measurement of the selfregulated learning construct.
It is interesting to note that the factor loadings of the specific components on the items of each scale remained relevant, despite the presence of a general orthogonalized factor in relation to all other components. This is noteworthy because it indicates that the specific components are important selfregulatory processes that explain some of the variance of the MSLQ items. This is true for 13 of the 15 specific components, of which only items 21 and 58 have factor loadings less than .20. The exceptions are the components of elaboration and metacognitive selfregulation because many of their items are loaded in a relevant way only by the general factor, with loadings lower than .20 on the specific factors. These two components showed inadequate reliability, taking McDonald’s omega as a reference (Table 6). Although the intrinsic goal orientation component has no items with loadings lower than .20, it has few items, also causing inadequate reliability via McDonald’s omega (Table 5). A technical procedure to solve the unacceptable reliability of these three components is to increase the number of items in the scales aimed at measuring them. It is relevant to add items with similar characteristics to those items that load on these components in a relevant way. Until this is done, it is relevant to be cautious about using the scores of these components for clinical practice and psychoeducational diagnoses. We emphasize that future analysis is needed on the sentence contents of items that have low loadings on specific factors, to understand why these items do not adequately represent their components in the presence of the overall factor.
The fact that the final model presents four broad components and not two broad components as strongly claimed by the theory (Pintrich et al., 1991), in part refutes the theory. What would be the broad component strategy is actually three distinct strategies, namely learning strategies, regulation strategies and social strategies. In other words, in the final model there is no broad component strategy and this is what refutes part of the theory.
The factor loadings of the broad components on their specific components support part the theory (see Table 9). The theory states that the specific components of help seeking and peer learning are positive selfregulated learning strategies. The results corroborate this theoretical claim when they show that the broad component of social strategies loads both specific components positively and with strong loadings. The same occurs with the specific components of time and study environment and effort regulation in relation to the broad component of regulation strategies. Regarding the broad component of motivation, it loads positively on the specific components that the theory states are positive selfregulatory motivations. Similarly, it loads negatively on the two selfregulatory motivations that the theory claims are negative.
The factor loadings of the broad component learning strategies on the specific factors partly refute the theory. The specific component rehearsal is loaded in a positive and relevant way by the broad component learning strategies, contrary to the theory’s claim that rehearsal is a strategy negatively associated with the other strategies. This negative relationship between rehearsal and the other MSLQ strategies are declared by selfregulated learning theory because rehearsal is defined by the theory as a low cognitive processing strategy, while the others are defined as highlevel cognitive processing (Chow & Chapman, 2017). A possible reason why we found a positive and relevant factor loading between rehearsal and learning strategies in our study may be that students do not perceive and evaluate rehearsal items as mechanical and low processing strategies. The rehearsal items describe memorization and repetition behaviors, but do not necessarily mandate that repetition be exclusively mechanical and unreflective. This is evidenced in the following items from this component: (1) “I memorize keywords to remind me of important concepts in this class” and (2) “When I study for this class, I practice by saying the material to myself over and over”. Although memorization by repetition is often conceived as rote learning in education, it is very possible that repetition is involved with active processes of relationship formation and meaning construction. For example, even by just repeating information, the student may, while repeating the information, articulate the repeated information, integrate it in some way with what he already knows. In short, the student can read the rehearsal MSLQ items and assess that even in “learning by heart” there is highorder cognitive processing. Students might be correct in their selfreport. Active subject theories may underestimate information repetition behaviors, making a perhaps naive assumption that if there is repetition, then there is no production of relationships and construction of meaning. There is much evidence from studies on memory in tasks of mere memorization by repetition of stimuli that an individual’s memorization, even by repetition, is involved in information association strategies and that better strategies generate better memorization performance (Carroll & Harris, 2020).
Still on the broad and specific components, we found a peculiar result that the broad component learning strategies have zero loading in relation to the specific component critical thinking. Moreover, the items of the critical thinking scale are well loaded by the general factor. This result suggests that critical thinking is only articulated to the general factor, that is, to selfregulated learning. Of course, we need to understand better why the specific component of critical thinking is not within the scope of direct relations with some broad component but relates only directly to the general factor of selfregulated learning. We do not have a hypothesis about this. Further studies are needed to understand whether this result is a specific feature of our sample or it is a generalizable result.
Our results need to be replicated in other samples for the new evidence to become robust. Despite this warning, the new evidence could have some clinical implications. Throughout the presentation of the results and discussion, we have highlighted several of these implications, but two remain to be discussed. The first of these involves the underutilization of the MSLQ measure. The metaanalyses by Credé and Phillips (2011) and Broadbent and Poon (2015) show that researchers have only used the scores of the 15 specific component measures of the MSLQ. Our evidence shows that both researchers and clinicians can use MSLQ scores for the measure of four broad components of selfregulated learning in addition to the general factor, and need not be restricted to the specific components. The second implication refers to the insufficient reliability of the specific components of metacognitive selfregulation, elaboration, and intrinsic goal orientation. This compromises the evidence for the prediction of these components relative to academic performance. Moreover, it hinders the clinical use of these components, making their diagnosis imprecise. This low reliability is even more serious in the metacognitive selfregulation component since it is one of the three basic components that the field defines as constitutive of selfregulated learning.
Conclusion
This study brings relevant contributions to the field of selfregulated learning, as it shows new evidence of structural validity of the MSLQ with all its scales in a single model (Pintrich et al., 1991). This study also provides advances for selfregulated learning theory in that it empirically identifies not only the 15 specific components of the MSLQ, but also four broad components and the general factor of selfregulated learning. The results indicate that both the general factor of selfregulated learning and the 12 components of the MSLQ are reliable. It also highlights the relevance of correct estimator selection for confirmatory factor analyses of items. The results showed that the maximum likelihood and its robust estimators are not appropriate for the MSLQ data used in this study. Following Li′s (2016) argument, confirmatory factor analyses of items are most likely to be well estimated by WLSMV when the data strongly violate multivariate normality. It was previously common to assume that if a scale had 7 or 8 points, it would tend to generate a normal distribution, consequently producing a multivariate normal distribution for the items in an instrument. However, this belief did not prove to be adequate. Having 7 points on a scale, as is the case with the MSLQ, is no guarantee that item responses will show multivariate normality. On the contrary, the results found in the literature on the MSLQ indicate multivariate nonnormality whenever inspected by researchers. Therefore, it is possible that most studies have been using an incorrect estimator.
In addition to those contributions, the present study also highlights theoretical aspects of selfregulated learning that should be investigated in the future. For example, there is a need for a better understanding of the relationships between the general factor of selfregulated learning and the broad and specific components. So far, we have no information about this, considering the absence of models that include the general factor in the analysis. Besides, we will need to investigate the predictive role of the general factor of selfregulated learning about educationally relevant outcomes such as performance and dropout, as well as the convergent and divergent validity of the general factor regarding certain constructs. We will also need to understand whether the componentspecific items of critical thinking and elaboration are really explained by the general factor alone, or whether our result is restricted to the characteristics of our sample. In any case, this result will have to be better understood. The results on the specific component of rehearsal indicate that memorization through repetition strategies is not perceived as superficial by students. This is a result that refutes an important statement of the theory and therefore needs further research.
Although the evidence from this study is promising, it is just the beginning. To the best of our knowledge, our results are the first to bring forward models with acceptable fit to the factor structure of the MSLQ. Furthermore, our results come from a convenience sample consisting of students from the same institution, most of whom are male and young adults. Results based on convenience samples weaken the external validity of the results, but in current practice they often turn out to be the only possible way to do. This limitation in the external validity of our study also occurs in all the studies that examined the factor structure of the MSLQ, which we cite in Table 2. To improve external validity, more studies with diversified samples need to be conducted in order to generate more robust evidence regarding the structural validity of the MSLQ and the reliability of its components. It is relevant that other researchers employ our data analysis strategies, particularly the analysis of multivariate normality, the use of the WLSMV estimator whenever multivariate normality is lacking, and the testing of bifactor models in which the general factor is present. If our results are reproduced, then the evidence from our study will prove more robust.
This study makes a practical contribution to users of the MSLQ by identifying inadequate reliability for three specific component measures. We do not encourage the use of these scales for clinical practice and psychoeducational diagnosis until the reliability issue is resolved. We recognize that our sample is limited because it is a convenience sample. However, the MSLQ is a widely used test in many parts of the world, serving as a tool for clinical practice and educational diagnosis. Given its importance, it is our point of view that the results of our study are sufficient for researchers to rethink about the use of elaboration, metacognitive selfregulation, and intrinsic goal orientation component measures.
In conclusion, in this paper, we show in detail the mistakes or the limitations present in previous works that invalidated an adequate analysis of the factor structure of the MSLQ. We used a data analysis strategy capable of correcting the mistakes made. We found different models with acceptable fits and defined the best one, calling it the final model. We hope this work can be a guide to the researcher, in order to avoid past mistakes on the structural validity of the MSLQ of being constantly repeated. This study presented a final model that deserves to be investigated in different contexts and samples.
For researchers wishing to test the models applied in our study, we caution that the MSLQ models are complex in terms of the number of parameters, so we suggest using a sample size of at least approximately 1000 students. In addition to the complexity of the models, the WLSMV estimator itself demands a large sample size when run on scales with as many points as in the MSLQ, which has seven points in its scale. Those interested in replicating the tested models can contact the first author of this article and request the scripts run in R software in order to reproduce them.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon request.
Abbreviations
 MSLQ:

Motivated Strategies for Learning Questionnaire
 ML:

maximum likelihood
 MLR:

maximum likelihood robust
 MLM:

maximum likelihood estimator with robust standard errors
 WLMSV:

weighted least squares mean and variance adjusted
 CFI:

confirmatory fit index
 RMSEA:

rootmeansquare error of aproximation
 SRMR:

standardized root mean square residuals
 MND:

multivariate normal distribution
 NR:

not reported
 SLR:

selfregulated learning
 IGO:

intrinsic goal orientation
 EGO:

extrinsic goal orientation
 TVA:

task value
 CLB:

control of learning beliefs
 SLP:

selfefficacy for learning and performance
 TAN:

test anxiety
 REH:

rehearsal
 ELA:

elaboration
 ORG:

organization
 CTH:

critical thinking
 MSR:

metacognitive selfregulation
 TSE:

time and study environment
 ERE:

effort regulation
 PLE:

peer learning
 HSE:

help seeking
 MOT:

motivation
 STR:

strategies
 LST:

learning strategies
 RST:

regulation strategies
 SST:

social strategies
References
Adesope, O., Hunsu, N., Van W., Bernard J., Austin, B., Richards, R.F. & Dutta, P. (2017). Work in progress: Assessing engineering students’ motivation and learning strategiesa psychometric analysis of the motivated strategies for learning questionnaire. ASEE Annual Conference and Exposition, Conference Proceedings, 2017June. https://doi.org/10.18260/12%2D%2D29144
Alkjarusi, H., Neisler, O., AlBarwani, T., Clayton, D., Khan, M., AlYahmadi, H., & AlKalbani, M. (2012). Psychometric properties of the motivated strategies for learning questionnaire for. College Student Journal, 46(3), 567–581 https://www.researchgate.net/publication/256349252_Psychometric_properties_of_the_Motivated_Strategies_for_Learning_Questionnaire_for_Sultan_Qaboos_University_students.
Broadbent, J., & Poon, W. L. (2015). Selfregulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007.
Carroll, K. A., & Harris, C. M. (2020). Using A repetitive instructional intervention to improve students’ higherorder thinking skills. College Teaching, 69(2), 82–90. https://doi.org/10.1080/87567555.2020.1823310.
Casali, N., Ghisi, M., Rizzato, R., Meneghetti, C., & De Beni, R. (2022). Validation of the “studyanxiety” questionnaire: A scale for the initial assessment of university students seeking psychological help. Journal of Psychopathology and Behavioral Assessment, 44, 1158–1168. https://doi.org/10.1007/s10862022099828.
Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational research methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239.
Cho, M. H., & Summers, J. (2012). Factor validity of the motivated strategies for learning questionnaire (MSLQ) in asynchronous online learning environments (AOLE). Journal of Interactive Learning Research, 23(1), 5–28 https://arizona.pure.elsevier.com/en/publications/factorvalidityofthemotivatedstrategiesforlearningquestion.
Chow, C. W., & Chapman, E. (2017). Construct validation of the motivated strategies for learning questionnaire in a Singapore high school sample. Journal of Educational and Developmental Psychology, 7(2), 107. https://doi.org/10.5539/jedp.v7n2p107.
Cook, D. A., Thompson, W. G., & Thomas, K. G. (2011). The motivated strategies for learning questionnaire: Score validity among medicine residents. Medical Education, 45(12), 1230–1240. https://doi.org/10.1111/j.13652923.2011.04077.x.
Credé, M., & Phillips, L. A. (2011). A metaanalytic review of the motivated strategies for learning questionnaire. Learning and Individual Differences, 21(4), 337–346. https://doi.org/10.1016/j.lindif.2011.03.002.
Dayel, S. B., Al Diab, A., Abdelaziz, A., Farghaly, A., & Al Ansari, A. (2018). Validity of the motivated strategies for learning questionnaire in Saudi Arabia. International Journal of Medical Education, 9, 309–315. https://doi.org/10.5116/ijme.5bec.81cf.
Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, selfregulation, and selfregulated learning. Educational Psychology Review, 20, 391–409. https://doi.org/10.1007/s1064800890836.
Duncan, T. G., & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational Psychologist, 40(2), 117–128. https://doi.org/10.1207/s15326985ep4002_6.
Dunn, K. E., Lo, W.J., Mulvenon, S. W., & Sutcliffe, R. (2012). Revisiting the motivated strategies for learning questionnaire: A theoretical and statistical reevaluation of the metacognitive selfregulation and effort regulation subscales. Educational and Psychological Measurement, 72(2), 312–331. https://doi.org/10.1177/0013164411413461.
Gomes, C. M. A., Araujo, J., & CastilloDíaz, M. A. (2021b). Testing the invariance of the metacognitive monitoring test. PsicoUSF, 26(4), 685–696. https://doi.org/10.1590/141382712021260407.
Gomes, C. M. A., Araujo, J., & Jelihovschi, E. G. (2020a). Approaches to learning in the nonacademic context: Construct validity of learning approaches test in video game (LATvideo game). International Journal of Development Research, 10(11), 41842–41849. https://doi.org/10.37118/ijdr.20350.11.2020.
Gomes, C. M. A., Golino, H. F., & de Souza Peres, A. J. (2020b). Fidedignidade Dos Escores Do Exame Nacional Do Ensino Médio (ENEM). Psico, 51(2), e31145. https://doi.org/10.15448/19808623.2020.2.31145.
Gomes, C. M. A., Golino, H. F., Pinheiro, C. A. R., Miranda, G. R., & Soares, J. M. T. (2011). Validação da Escala de Abordagens de Aprendizagem (EABAP) em uma amostra Brasileira. Psicologia: Reflexão e Crítica, 24(1), 19–27. Portuguese. https://doi.org/10.1590/S010279722011000100004.
Gomes, C. M. A., Linhares, I. S., Jelihovschi, E. G., & Rodrigues, M. N. S. (2021a). Introducing rationality and content validity of SLATthinking. International Journal of Development Research, 11(1), 43264–43272. https://doi.org/10.37118/ijdr.20586.01.2021.
Gomes, C. M. A., Quadros, J. S., Araujo, J., & Jelihovschi, E. G. (2020c). Measuring students’ learning approaches through achievement: Structural validity of SLATthinking. Estudos de Psicologia, 25(1), 33–43. https://doi.org/10.22491/16784669.20200004.
Hamilton, R. J., & Akhter, S. (2009). Construct validity of the motivated strategies for learning questionnaire. Psychological Reports, 104(3), 711–722. https://doi.org/10.2466/pr0.104.3.711722.
Hands, C., & Limniou, M. (2023). Diversity of strategies for motivation in learning (DSML)—A new measure for measuring student academic motivation. Behavioral Science, 13(4), 301. https://doi.org/10.3390/bs13040301.
Hilpert, J. C., Stempien, J., Der Hoeven, V., Kraft, K. J., & Husman, J. (2013). Evidence for the latent factor structure of the mslq: A new conceptualization of an established questionnaire. SAGE Open, 3(4). https://doi.org/10.1177/2158244013510305.
Howard, M. C., & Henderson, J. (2023). A review of exploratory factor analysis in tourism and hospitality research: Identifying current practices and avenues for improvement. Journal of Business Research, 154, 113328. https://doi.org/10.1016/j.jbusres.2022.113328.
International Test Commission. (2017). The ITC guidelines for translating and adapting tests, 2, 143–171. http://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
Jackson, C. R. (2018). Validating and adapting the motivated strategies for learning questionnaire (MSLQ) for STEM courses at an HBCU. AERA Open, 4(4), 233285841880934. https://doi.org/10.1177/2332858418809346.
Jorgensen, T.D.; Pornprasertmanit, Sunthud; Schoemann, Alexander M. & Rosseel, Yves. (2021). semTools: Useful tools for structural equation modeling (0.5–5) [R package]. https://CRAN.Rproject.org/package=semTools
Karadeniz, S., Buyukozturk, S., Akgun, O. E., Cakmak, E. K., & Demirel, F. (2008). The Turkish adaptation study of motivated strategies for learning questionnaire (MSLQ) for 1218 year old children: Results of confirmatory factor analysis 1. The Turkish Online Journal of Educational Technology, 7(4), 108–117 https://eric.ed.gov/?id=ED503477.
Li, C.H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s1342801506197.
Liu, J., Xiang, P., McBride, R., & Chen, H. (2019). Psychometric properties of the cognitive and metacognitive learning strategies scales among preservice physical education teachers: A bifactor analysis. European Physical Education Review, 25(3), 616–639. https://doi.org/10.1177/1356336X18755087.
Mannion, J. (2018). Metacognition, selfregulation and selfregulated learning: what’s the difference? My College. https://my.chartered.college/impact_article/metacognitionselfregulationandselfregulatedlearningwhatsthedifference/
Nanol, A. A. J. (2015). Motivation and its implication on academic performance. School of Graduate Studies Research Journal Liceo de Cagayan University, 15(1). https://doi.org/10.7828/sgsrjldcu.v15i1.771.
Pintrich, P., Smith, D., García, T., & McKeachie, W. (1991). A manual for the use of the motivated strategies for learning questionnaire (MSLQ). Ann Arbor, MI: University of Michigan https://eric.ed.gov/?id=ED338122.
R Core Team. (2022). R (4.2). [Computer software]. R Foundation for Statistical Computing. https://cran.rproject.org/bin/windows/base/
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477.
Rosseel, Y., Jorgensen, T.D., Rockwood, N., Oberski, D., Byrnes, J., Vanbrabant, L., Savalei, V., Merkle, Ed., Hallquist, M., Rhemtulla, M., Katsikatsou, M., Barendse, M., Scharf, F. & Du, H. (2022). lavaan: Latent Variable Analysis (0.6.11). [R package]. https://cran.rproject.org/web/packages/lavaan/index.html
Rotgans, J. I., & Schmidt, H. G. (2010). The motivated strategies for learning questionnaire: A measure for students’ general motivational beliefs and learning strategies? The AsiaPacific Education Researcher, 19(2), 357–369. https://doi.org/10.3860/taper.v19i2.1603.
Roth, A., Ogrin, S., & Schmitz, B. (2016). Assessing selfregulated learning in higher education: A systematic literature review of selfreport instruments. Educational Assessment, Evaluation and Accountability, 28, 225–250. https://doi.org/10.1007/s1109201592292.
Ruiz, V. M. (2005). Aprendizagem em universitários: variáveis motivacionais, (p. 195). [Doctoral dissertation, Pontifícia Universidade Católica de Campinas], Centro de Ciências da Vida.
Satorra, A. (2000). Scaled and adjusted restricted tests in multisample analysis of moment structures. In R. D. H. Heijmans, D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multivariate statistical analysis. A festschrift for Heinz Neudecker, (pp. 233–247). London: Kluwer Academic Publishers.
Schunk, D., & Greene, J. (2017). Handbook of selfregulation of learning and performance, (2nd ed., ). Routledge: New York.
Şen, Ş., Yilmaz, A., & Yurdugül, H. (2014). An evaluation of the pattern between students’ motivation, learning strategies and their epistemological beliefs: The mediator role of motivation. International Association on Science Education International, 24(3), 312–331 https://www.researchgate.net/publication/286199174_An_evaluation_of_the_pattern_between_students%27_motivation_learning_strategies_and_their_epistemological_beliefs_The_mediator_role_of_motivation.
Simpson, A. R. (2018). Young adult development project. MIT https://hr.mit.edu/static/worklife/youngadult/changes.html.
Thakkar, J. J. (2020). Structural equation modelling application for research and practice (with AMOS and R). Singapore: Springer.
Valentini, F., Gomes, C. M. A., Muniz, M., Mecca, T. P., Laros, J. A., & Andrade, J. M. (2015). Confiabilidade dos Índices Fatoriais da WaisIII Adaptada para a População Brasileira. Psicologia  Teoria e Prática, 17(2), 123–139. https://doi.org/10.15348/19806906/psicologia.v17n2p123139.
VillarrealFernández, J. E., & ArroyaveGiraldo, D. I. (2022). Adaptación y validez de la escala de motivación del Motivated Scale Learning Questionnaire (MSLQ) en universitarios colombianos. Electronic Journal of Research in Educational Psychology, 20(56), 119–150. https://doi.org/10.25115/ejrep.v20i56.4394.
Wang, F., Jiang, C., King, R. B., & Leung, S. O. (2022). Motivated Strategies for Learning Questionnaire (MSLQ): Adaptation, validation, and development of a short form in the Chinese context for mathematics. Psychology in the Schools, 6, 2018–2040. https://doi.org/10.1002/pits.22845.
Zhou, Y., & Wang, J. (2021). Psychometric properties of the MSLQB for adult distance education in China. Frontiers in Psychology, 12, 620564. https://doi.org/10.3389/fpsyg.2021.620564.
Acknowledgements
Cristiano Mauro Assis Gomes: Productivity Fellowship, CNPq, Brazil.
Jhonys de Araujo: Doctoral scholarship, CAPES, Brazil.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or notforprofit sectors.
Author information
Authors and Affiliations
Contributions
We confirm that all authors have contributed significantly and are in agreement with the content of the manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Data collection followed ethical guidelines and was approved by CEP/UDESC (73453317.1.0000.0118).
Consent for publication
Not applicable.
Competing interests
None.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
de Araujo, J., Gomes, C.M.A. & Jelihovschi, E.G. The factor structure of the Motivated Strategies for Learning Questionnaire (MSLQ): new methodological approaches and evidence. Psicol. Refl. Crít. 36, 38 (2023). https://doi.org/10.1186/s41155023002800
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41155023002800