Portuguese validation of the Social Competence and Behavior Evaluation Scale (SCBE-30)

Abstract

Aim

The Social Competence and Behavior Evaluation Scale (SCBE-30) is a 30-item questionnaire designed to assess preschoolers’ behavioral problems (externalizing and internalizing) and social competence. It is widely used in the developmental research. This study aims to contribute to the adaptation and validation of the Portuguese version of the teachers’ short form of the SCBE-30 (Psychol. Assess. 8:369–777, 1996).

Method

Participants were 361 children from 3 to 6 years old whose preschool teachers completed the SCBE-30. For external validation purposes children completed arithmetic and theory of mind tasks, and parents completed a socio-demographic questionnaire.

Results

Confirmatory factor analysis did not confirm the 10-item-subscale solution of the original SCBE-30 version. Instead, a five-item-subscale was found as a better solution.

Discussion

The reduced 15-item version replicates the three-factor structure, shows good psychometric properties and meets external validation criteria. Further research should focus on the invariance of the factor structure of the SCBE-30 between cultures.

Background

The Social Competence and Behavior Evaluation Scale (SCBE) is a widely used instrument designed to evaluate patterns of social competence, emotion regulation and expression, and adjustment difficulties in children aged 30 to 78 months (LaFreniere et al. 1992). The teachers’ short form (SCBE-30) assesses the development of social competencies or maladaptive behaviors such as chronic aggression and social withdrawal (Bigras and Dessen 2002; LaFreniere, & Dumas, 1996). Social competence is the children’s ability to respond to and to deal with challenging social interactions/interpersonal situations. Since teachers spend a significant amount of time with children on a daily basis, they are considered a reliable and efficient source of information (LaFreniere & Dumas, 1996).

The SCBE-30 was designed to capture the child’s relationships with teachers and peers by providing: (a) a standardized description of affect and behavior in context; (b) a screening instrument that distinguishes specific types of behavioral-emotional problems; (c) an assessment of children’s positive social adjustment or competence; (d) a sensitive measure with high internal consistency, reliability, and stability; and (e) a sensitive measure of the behavioral change over time to evaluate short-term treatment and programs outcomes (LaFreniere & Dumas, 1996). The SCBE-30 is composed of three factors: (a) anger-aggression (externalizing behavior); (b) anxiety-withdrawal (internalizing behavior) and (c) social competence (pro-social behaviour). This factor structure has been found in at least eight countries, supporting cross-cultural validation of the instrument (LaFrenière et al. 2002). As far as we know, however, no confirmatory factor analysis (CFA) of the 3-factor model has ever been published.

We highlight three main aspects of the knowledge derived from the SCBE-30. First, the subscales of the SCBE-30 show very good internal consistency, and the three-factor solution has been found in many samples. Nonetheless, only in samples of US and Quebec did all 30 items load higher than .40 using exploratory factor analyses (EFA). In the Brazilian version there was one item with low loading (i.e., minor to .40) and another item loading simultaneously on two subscales. In the Russian version, some items didn’t reach expected loadings or showed cross-loadings (e. g., three items of the externalizing behaviour subscale, Butovskaya & Demianovitsch, 2002). Inter-correlations among externalizing behavior, internalizing behavior, and prosocial behaviour, typically show that: (a) anger-aggression and anxiety-withdrawal are relatively orthogonal; and (b) pro-sociality and each of the previous two are moderately negatively correlated (LaFrenière et al. 2002).

Secondly, sex, age, and socio-economic status (SES) effects have also been found. Girls are usually reported to be more pro-social and less aggressive when compared to boys (LaFrenière et al. 2002; LaFreniere & Dumas, 1996). Studies have also found age effects; as preschool children grow older, they usually became more prosocial. Results are mixed, however, regarding the pre-schoolers’ development of internalizing and externalizing behaviors (LaFrenière et al. 2002). In some samples there is no age association. The subscales of the SCBE-30 have been related to parental socio-economic status (SES). Children from lower SES tend to score higher in both externalizing and internalizing behaviors.

Finally, some research supports the association between cognitive measures and SCBE-30 scores. For example, scores on subscales of the SCBE have been associated with performance in the WPPSI-R and the Lollipop test (Venet et al. 2002). Peer acceptance, a construct closely associated with social competence, also has been found to be associated with theory of mind scores (Slaughter et al. 2002). Additionally, behavioral regulation, an executive skill that underlies social competence and aggression, has been associated with literacy, vocabulary, and math performance (McClelland et al., 2007).

The goal of the current study was to present an initial validation of the Portuguese version of the teachers’ form of the SCBE-30. Our goal was to (a) confirm the three factor structure in a Portuguese sample, and (b) provide initial evidence for the validity of the instrument using discriminant and external criteria.

Method

Participants and procedure

Twenty-two teachers completed the SCBE-30 for 361 children (168 boys) from the greater Porto area, Portugal. Sixty-five were age three (M = 43,67 months, SD = 2.01 months; range = 39–47 months), 135 were age four (M = 51.94 months, SD = 3.69 months; range = 48–59 months), 103 were 5-year-olds (M = 64.44 months, SD = 3.59 months; range = 60–71 months), and 50 were 6-year-olds (M = 74.50 months, SD = 74.50 months; range = 72–83 months). For eight children age was not provided so they were excluded of age effects analyses. Mean level of parental education was 11.39 years (SD = 4.37) for mothers and 10.28 years (SD = 4.04) for fathers. These figures are similar to the average educational level for the Portuguese population aged 25–44 years-old, which is 10.9 years for women, and 9.9 years for men (Conselho Nacional da Educação, 2014).

A subsample of 96 children (43 boys) was selected for external validation purposes. This subsample took part in a larger project, and was composed of children aged three and four (M = 49.82, SD = 6.43). Schools were contacted using the snowball method, while trying to maintain a balance between public and private centers. There are 678 kindergartens (51 % public, 23 % private for profit and 26 % private non-profit) in the greater Porto area. Fourteen kindergartens agreed to join the study, of which eight were public, three were private for-profit, and three were private non-profit. The sample corresponds approximately to 2 % of the total number of registered kindergartens. After obtaining the centers’ agreement, teachers informed parents about the study and asked to sign a consent form and to complete the socio-demographic questionnaire. Teachers completed approximately 15–20 questionnaires SCBE-30 each, at least 3 months after the beginning of the school year.

Instruments

The social competence and behavior evaluation scale

The SCBE-30 for teachers was originally developed by LaFreniere and Dumas (1996). It includes 30 items rated on a 6-point Likert-type scale that describe the frequency of the behavior (1 = never, 6 = always). Items are organized in three subscales: anger-aggression, anxiety-withdrawal, and social competence.

This study used the Brazilian version of the SCBE-30 (Bigras & Dessen, 2002). A revision of item formulation was performed due to linguistic differences between the European versus Brazilian Portuguese mainly in the colloquial terms. This revision was undertaken by two European Portuguese native speaker researchers. A PhD senior researcher checked the appropriateness of the changes to the original English version. Finally, a back translation was performed by a bilingual Portuguese-English person. The translated English version, when compared to the original one (LaFreniere and Dumas 1996), proved to be semantically equivalent. Thus, we concluded that the Portuguese version of the SCBE-30 is equivalent to the original English version.

External criteria

Socio-demographic questionnaire

This questionnaire requested information about sex and age of the children, parental education, and family income. Data from eight mothers and 52 fathers was either illegible or not provided.

Arithmetic

Children’s early mathematical ability was measuredwith the Arithmetic subtest of the WPPSI-R (range 0–23) (Weschler, 2009).

Theory of mind

Two tasks were used to assess children’s ToM: (a) unexpected content task and (b) wrong localization task (Gopnik & Astington, 1988; Wimmer et al. 1988). Overall three questions were made, one of representational chance and two of false belief (range 0–3).

Results

A Confirmatory Factor Analysis (CFA) was performed to test the fit of the original model to data, using the software AMOS (v. 20 SPSS, Inc.). Goodness of fit was evaluated using: ratio X2/df, Comparative Fit Index (CFI), and Root Mean Square Error of Approximation (RMSEA). Ratio X ²/df under 2, a CFI over .90 and a RMSEA under .10 indicate good fit (Bollen, 1989).

The initial 30 items model did not show an acceptable fit: X ²/df = 3.69, CFI = .79; NFI = .74, RMSEA = 0.09. Consequently, we randomly split the sample in halves to perform new factorial analyses. In subsample A, an exploratory approach was used to seek an acceptable factorial solution that did not compromise the theoretical original structure with three factors. We removed the items with lower factor loadings in each subscale. We then tested the model in the sub-sample A. Thus, we developed, analyzed and compared successive new factorial models.

Table 1 presents the main steps of this exploratory process. We removed the two items with lower factor loadings in each factor, which proved to be insufficient to achieve a good fit (see model 2 in Table 1). We then deleted three additional items in each factor, and tested the new 15-item solution that showed an acceptable fit (X ²/df = 2.08, NFI = .86, CFI = .92, RMSEA = 0.80). We then tested this model in subsample B. The CFA revealed an acceptable fit also in this subsample, thus confirming its structure: X ²/df = 2.45, NFI = .84, CFI = .90, RMSEA = 0.09. The items included in each factor are depicted in Table 2, along with loadings for subsample B, and descriptive statistics for the total sample. The Cronbach’s alpha from the original 30-item version and from the 15-items version are highly similar (77 and .80 for the anxiety-withdrawal subscale, 89 and .88 for the anger-aggression subscale, and .85 and .88 for the social competence subscale.) Appendix provides the items included in and excluded from the 15 item scale.

Table 1 Factor loadings for the main models tested using an exploratory approach (subsample A)

Table 2 Factor loadings (CFA) in subsample B, and descriptive statistics for the total sample

Correlations between subscale scores from the 30-item version and from the 15-item version ranged from .90 (anger-aggression) to .96 (anxiety-withdrawal). Further analyses comparing patterns for external validation criteria for scores in the 15-item version and in the 30-item version showed no differences in effects. Due to the psychometric limitation of the 30-item scale, and the approximate equivalence regarding external validation criteria, subscales scoreswere computed using only the 5 items per subscale depicted on Model 3.

Correlations between subscales were r = – .37, p < .001 for social competence and anger-aggression, r = – .34, p < .001 for anxiety-withdrawal and social competence, and r = .03, n.s., for anxiety-withdrawal and anger-aggression.

The concurrent validity of the shortened version of the SCBE-30 was analyzed through the correlations with four criteria variables: children’s sex, children’s age, children’s cognitive outcomes and family SES. Table 3 shows the descriptive statistics of each subscale for children’s age and sex. Boys presented higher scores on anger-aggression, F (1, 360) = 17.23, p < .001. We found a main effect of age, with younger children being less pro-socially competent, F (3, 349) = 3.74, p = .011. Post-hoc analyses revealed significant differences between scores of children aged three and children aged four, five, and six. No effects of age or sex were found on the measure of internalizing behaviour.

Table 3 Mean scores (standard-deviations) in the SBCE (5 items subscales) for age and sex

Table 4 presents the correlations between SCBE subscales and child cognitive outcomes and family SES indicators. Higher family income and, to a lesser extent, greater parental education were significantly related to lower scores on the anger-aggression and anxiety-withdrawal subscales. Social competence subscale scores were positively correlated with arithmetic and theory of mind scores. Correlations of the excluded items with external validation criteria were usually lower or similar to those of included items. For the anxiety-withdrawal subscale, the range of correlations between excluded items and income was r = -.05 to r = -.14. For the anger-aggression subscale, the range of correlations between excluded items and income ranged from r = -.05 to r = – .30. The item 17 is the only item excluded that showed a pattern of correlation with SES indicators similar to items included. In the social competence subscale, mean correlation of the items included was r = .17 with Arithmetic and r = .16 with ToM, versus r = .14 and r = .17 for items out, respectively. Two items out (14 and 23) displayed higher correlations with these criteria.

Table 4 Correlations between the SCBE (5-item subscales) and criteria variables

Discussion

The goal of this study was to attempt to confirm the three factor structure of the SCBE-30 in a Portuguese sample of preschool children and provide initial evidence for the validity of the adapted instrument using discriminant and external criteria.

We did not replicate the 10-item per subscale solution in the present sample. This was not surprising given that many items showed low or problematic loadings in international adaptation studies, even using EFA (e. g., LaFrenière et al. 2002). Although previous adaptations used EFA, the current study used a CFA. The best model found was a three-factor structure with 15 items—five per subscale—that proved to be valid for the Portuguese sample. The reliabilities of the subscales obtained with 15-item solutionwere slightly higher than the reliabilities obtained with the SCBE-30 and almost identical with the third one. Given the association between number of subscale items and magnitude of alpha coefficients, the higher values of reliability found in this study provide additional support for the decision to reduce the number of items in the scales.

We further performed a content analysis of the included and excluded items. On the anxiety-withdrawal subscale, items related to facial expressions, inaction and retraction for social interactions are present both in the included and in the excluded items. On the anger-aggression subscale, items referring to aggression, anger, frustration and opposition are both in the included and the excluded items. The three excluded items, however, deal with children aggressively confronting adult. This behaviour is not captured by the included items. Nevertheless we consider that the exclusion of these items has no effect on the construct representation since these behaviours can be also measured by the included items; that is, if children are angry when interrupted, they will likely show reactions when teacher interrupts them (usually teachers interrupt children in the classroom setting), if children shout or speak with bad manners, they will usually do that also with the teacher or other adults. Finally, on the social competence subscale, items included have a broader formulation, i.e., they refer to cooperation, help, work in groups, consolation. Items excluded refer to more specific behaviours (“Negotiates solutions”) or are culturally inappropriate. For example, “Helps younger children” is not applicable to children in Portugal because most preschool centres in Portugal do not put younger children and older children together in the same classroom.

The results show that the correlations of excluded and included items with external criteria, are quite similar or even lower in the case of excluded items. This suggests that the differences in factor structure could be due both to classical measurement error and cultural differences in the interpretation of items.

As expected, intercorrelations among subscales, were similar to those found in other studies, with anxiety-withdrawal and anger-aggression subscales remaining orthogonal and showing a moderate correlation with social competence. The discriminant analyses confirmed the tendency found in previous studies that girls are less aggressive than boys. A main effect of age, with older children being more prosocial, was found for social competence, which is in line with previous research (LaFrenière et al. 2002). We did not find, however, an effect for age on anger-aggression. Although a decrease in externalizing behaviour has been found in some international samples, this is not a universal trend (e.g., LaFrenière et al. 2002). External validation criteria showed modest to moderate correlations with subscales. Also as expected, anxiety-withdrawal and anger-aggression subscales were negatively correlated with family income and parental education. These results can be explained by the effects of lower SES on parental stress, which leads to more negative familiar interactions, which may then have an impact on the display of child externalizing and internalizing behaviours in the classroom (Bigras & Dessen, 2002; Venet et al., 2002). Of the child cognitive measures, only the social competence subscale was significantly correlated with child arithmetic and theory of mindscores. Research showed that both competencies were related to better self-regulation, which seems to be a key skill underlying social behavior and cognitive performance. The association between social competence and theory of mind may be the result of greater understanding of social perspective-taking promoting greater caring about peers (Slaughter et al. 2002).

This study had two main limitations. The sample is not representative of the Portuguese population aged 3 to 6 and the criteria measures did not include other social adjustment indicators that have been used in previous studies (e.g., inhibition or peer evaluation). However, the CFA is the statistical procedure that better inform of the composition of latent variables in psychometric research (Bollen, 1989).

Conclusions

A shortened version of the SCBE-30 offers some advantages in research. It can be useful for researchers who cannot use the longer version because teachers are too busy to complete the long form on a large number of children. In classes between 20 to 25 children (common in Portugal and Iberoamerican countries), the estimated completion time can be reduced up to 15 min, making teachers more receptive to use it. Moreover, test length does not guarantee psychometric superiority (Burisch, 1997). Some factors could explain the better solution we found for a shortened version of the SCBE-30: teachers’ fatigue accumulation, frustration produced by redundancy and excessive working memory load during completion of a single questionnaire. Because we did not confirm the original 30-item structure, we believe further research should: (a) deepen reliability and external validity information in other samples, (b) focus on cross-cultural questions about the factor structure of the SCBE-30 and the 15-item version presented here, and (c) measure invariance of the factor structure of the scale between cultures and age groups.

Appendix

Table 5 Items included and excluded from the SCBE 15-item solution