Deviant behavior variety scale: development and validation with a sample of Portuguese adolescents

Abstract

This study presents the development and analysis of the psychometric properties of the Deviant Behavior Variety Scale (DBVS). Participants were 861 Portuguese adolescents (54 % female), aged between 12 and 19 years old. Two alternative models were tested using Confirmatory Factor Analysis. Although both models showed good fit indexes, the two-factor model didn’t presented discriminant validity. Further results provided evidence for the factorial and the convergent validity of the single-factor structure of the DVBS, which has also shown good internal consistency. Criterion validity was evaluated through the association with related variables, such as age and school failure, as well as the scale’s ability to capture group differences, namely between genders and school retentions, and finally by comparing a sub-group of convicted adolescents with a group of non-convicted ones regarding their engagement in delinquent activities. Overall, the scale presented good psychometric properties, with results supporting that the DBVS is a valid and reliable self-reported measure to evaluate adolescents’ involvement in deviance.

Background

Youth involvement in behaviors that violate social and/or legal norms, rules or conventions has been for long the subject of interest and research, within the most diverse theoretical approaches. Despite the existence of several Portuguese studies on this topic (e.g. Ferreira 1997; Fonseca 2004; Lemos 2010; Pechorro et al. 2013, 2014; Sanches et al. 2012), we have done an extensive literature review and we could not find any publication addressing the validation or the analysis of the psychometric properties of a Portuguese scale to measure the deviant or delinquent behavior of adolescents. Our aim with this study was to fill this gap.

According to the APA Dictionary of Psychology (VanDenBos 2007), deviance is defined as “any behavior that deviates significantly from what is considered appropriate or typical for a social group” (pp.276), while delinquency is defined as a “behavior violating social rules or conventions” (pp. 265). There is an apparent overlap between these two concepts, although the same dictionary defines juvenile delinquency as an “illegal behavior by a minor (usually identified as a person under age 18), including behavior that would be considered criminal in an adult” (pp. 510). This conceptual overlap is confounding and is frequently reflected in the measurement instruments used in research, with delinquency scales and deviance scales frequently having identical contents. Although the term delinquency is the most often used in the literature, it is closely associated with law-breaking behaviors. Given that our aim was to develop a scale that includes illegal behavior but also rule-breaking behavior that is not illegal, we considered that the term deviance was broader and therefore more adequate to our purposes.

Deviance or delinquency are commonly measured in two ways: through official records concerning convictions and through self-reported measures. Official records have the advantage of being more reliable, but the disadvantage of over-representing the most serious offenses and the most serious offenders (those who have been caught and convicted). Self-reports instead, have the advantage of detecting a wider range of behaviors, both in terms of variety, frequency and seriousness, but the disadvantage of being less reliable, since they might be affected by memory, bias and concealment. Despite this disadvantages, several studies have accounted for the validity of self-reports (e.g. Farrington 1999; Jolliffe et al. 2003; Webb et al. 2006) and this is the method most often used in psychological research to measure delinquent and deviant behavior.

Self-reported scales can be of three types, depending on whether they are focused on the frequency, the seriousness, or the variety of the behaviors. Frequency scales measure the number of times that each deviant act has been committed in a certain period of time. Seriousness scales are based on severity ratings of behaviors, usually developed by experts in the field. They are frequently divided in two or three seriousness levels, such as minor and serious offenses, or minor, moderate and serious offenses, or even high and low prevalent offenses, and individuals are labeled according to the level of the most serious offense committed. Variety scales in turn, measure the range or number of different deviant acts that have been committed in a certain period of time, so in these scales each item is assumed to represent a different type of infraction.

Although frequency scales are the most commonly used in deviance and delinquency research, a comparative study by Bendixen et al. (2003) has shown that variety scales are superior to frequency scales in what concerns their psychometric properties: they have higher internal consistency, higher stability over time, higher group differences, and stronger associations with conceptually related variables. Moreover, variety scores tend to be less skewed than frequency scores and they assign equal weight to all offenses, while frequency scores tend to overweight minor offenses, given that they usually occur more frequently. Variety scales also have some practical advantages over frequency scales: they have a simpler answering format, which decreases both the probability of respondent guessing, as well as the time necessary to answer the questions (Bendixen et al. 2003). Although variety scales are considered a reliable and valid measure and have been used for long (e.g. Bendixen et al. 2003; Bendixen and Olweus 1999; Caspi et al. 1994; Junger-Tas and Marshall 1999; Weerman and Bijleveld 2007), they seem to be gaining an increasing relevance recently, given the growing number of publications using this kind of scale (e.g. Brown and Jennings 2014; Cohn et al. 2012; Donner et al. 2014; Flexon, and Meldrum 2013; Hirtenlehner et al. 2014; Intravia et al. 2012; Malouf et al. 2014; Megens and Weerman 2012; Meldrum et al. 2012; Peck 2013; Trinkner et al. 2012; Yu et al. 2013).

Taking all these reasons into account, our aim was to develop a variety scale intended to measure the adolescents’ involvement in deviance–the Deviant Behavior Variety Scale (DVBS)–and to study its psychometric properties with a sample of Portuguese adolescents. Two criteria were taken into account during scale development. First, the scale should be as simple and short as possible, so that it could be easily understood and quickly answered, given that young people involved in deviance frequently have poor reading skills,. Second, the scale should be sufficiently varied, both in terms of the type of behavior, and in terms of its severity, so that it would be able to detect different types and levels of deviant engagement.

Two alternative models were analyzed using Confirmatory Factor Analysis, to see which one best represented the internal structure of the scale. The first one was a single-factor model, frequently used in deviance and delinquency research, namely when a variety scale is used (e.g. Brown and Jennings 2014; Hirtenlehner et al. 2014; Sanches et al. 2012; Smith and McVie 2003; Sutton and Winnard 2007; Trinkner et al. 2012; Weerman 2011; Yu et al. 2013). The second one was based on the seriousness of the infractions and is a two-factor model composed by minor and serious infractions (which correspond to high and low prevalence rates respectively), that is also quite common, namely in research using variety scales (e.g. Bendixen and Olweus 1999; Peck 2013; Weerman and Bijleveld 2007). Although the factorial structure of most deviance scales is organized around different categories of infractions (e.g. thefts, drugs use, aggressive behavior, vandalism, etc.), this is not appropriate for a variety scale, where each item is assumed to represent a different type of infraction.

Methods

Participants

Participants were 861 Portuguese adolescents (46 % male) aged between 12 and 19 years old (M = 15.7; SD = 2.05). School attendance varied from the 5th grade to the 2nd year of college, with 51 % of the participants having already failed at least once. The majority of participants (51.6 %) frequented public schools, 40.5 % frequented a private school and 7.9 % a private university. Participants were distributed in two sub-groups based on their answer to the following question included in the questionnaire: “Have you ever been convicted by the court to fulfill a sentence due to your involvement in illegal or criminal activities?”. Those who answered “yes” (N = 84) were assigned to the “convicted” group, while the remaining (N = 777) were assigned to “non-convicted” group. This small sub-group of convicted adolescents was mainly composed by boys (75 %) and it was slightly older (M = 16.2; SD = 1.45) than the non-convicted group (M = 15.72; SD = 2.10). All convicted adolescents frequented public schools, from the 5th to the 12th grade, and 95 % of them reported having failed at least once.

Measures/instrument

Item generation

An initial pool of 34 items was compiled from different sources, including literature review on pre-existing self-reported measures for delinquent behavior (e.g. Bendixen and Olweus 1999; Junger-Tas et al. 2003; Smith and McVie 2003), and a Portuguese self-reported frequency scale measuring adolescents’ deviant behavior (see Sanches et al. 2012). Items had different severity levels and pertained to 11 categories: thefts, alcohol and drugs consumption, verbal and physical aggression, possession of weapons, vandalism, truancy, driving without a license, assault, use of public transport without paying, lies and defiance of authority, and selling drugs.

Item selection

Items were evaluated by 81 professionals (13 Judges, 9 Prosecutors, 21 Social Workers, 17 Psychologists and 20 Teachers) belonging to several organizations in the Lisbon area that work with youth at-risk and involved in deviance. Item evaluation was based on two criteria: type of infraction (answers given on a 3-point scale: 1) behavior that does not violate a law, nor violates a social norm; 2) behavior that does not violate a law, but that violates a social norm; or 3) behavior that violates a law; and severity of the behavior (answers given on a 4-point scale: from 1 = non-serious behavior; 4 = very serious behavior). The 14 items that were classified by at least 90 % of the professionals as a violation to a social norm or, alternatively, as a violation to a law were selected, accounting for 10 of the 11 categories of behaviors initially included. Truancy, alcohol consumption and the use of soft drugs were left out. However, given that these behaviors are present in most deviance and delinquency scales and are relevant for the age group to which the scale is intended, five more items pertaining to these three categories were included. The 19 behaviors selected were then sorted according to the judges’ evaluation of their severity level, which ranged between 2.4 and 4 (M = 3.41; SD = 0.35). Table 1 presents a complete list of the 19 items composing the final version of DBVS.

Table 1 Percentage of positive answers on the scale items

Question format and score calculation

For each of the 19 deviant behaviors composing the DBVS, participants are asked whether they have committed it during the last year. Answers are given in a yes or no format. A participants’ variety score, which is a sum score, is calculated by summing the dichotomous scores on each individual item, and it ranges between 0 and 19, with higher scores indicating a wide variety of deviant behaviors committed.

Procedure

This study was approved by the General Education Directorate of the Ministry of Education and Science, as well as by the ethical committee of the ISPA–Instituto Universitário. Participants were selected by convenience. Data were collected in seven public schools, three of them being vocational schools, one private school and one private university, all in the Lisbon area, after obtaining the consent from school boards. Parental consent was also obtained, except for university students. Questionnaires were administered collectively in classrooms, in the exclusive presence of the researcher. All participants took part on a voluntary basis. They were informed that the questionnaire was anonymous and that the data was strictly confidential and no one they knew would have access to their answers. Questionnaires took about 5–10 min to be completed.

Data analysis

Statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS, version 22.0) (IBM SPSS, Chicago, IL). Descriptive statistics were used to analyze the distribution responses of each item. Construct validity was examined through factor, convergent and discriminant validity. The factorial structure of the scale was evaluated using Confirmatory Factor Analysis (CFA), but since the scale items were dichotomous, the model was estimated using tetrachoric correlations and the weighted least squares mean and variance adjusted estimation procedure implemented in Mplus 6.1 (Muthén & Muthén, Los Angeles, CA). The goodness-of-fit of the factorial models tested was evaluated through the following indices, with the reference values generally assumed in CFA (Byrne 2001; Marôco 2014): χ ² /df (~2–3); CFI and TLI (> 0.9); RMSEA (< 0.05) and WRMR (<1). Convergent validity was evaluated through the average variance extracted (AVE), which accounts for the proportion of variance in the items that is explained by the underlying factor, and is considered appropriate when AVE >0.5 (Fornell and Larcker 1981; Marôco 2014). Discriminant validity was analyzed comparing the squared correlation between factors with the AVE of each factor. In order to have discriminant validity, the association between factors should be smaller than the individual AVE (Fornell and Larcker 1981; Marôco 2014). The internal consistency of the scale was examined using both the Cronbach’s Alpha and the Composite Reliability (CR), where values ≥0.70 are considered adequate. Finally, criterion validity was evaluated through Person’s Correlations between the scale and associated variables, while group differences were examined using the Student-t test and ANCOVA. The Welch’s Student-t test was used when the Levene test revealed heterocedasticity of variances for the groups under study (p < 0.05).

Results

Item analysis

We started by analyzing the distribution response for each dichotomous item (coded as 0 or 1). Results are presented in Table 1.

Prevalence rates ranged between 1.6 and 74.5 % and, as expected, serious infractions displayed lower prevalence rates than minor infractions. Seven items had prevalence rates <10 %, three of which <5 %, 10 items had prevalence rates ranging between 10 and 50 % and only 2 items had prevalence rates >50 %. This wide range of prevalence rates reflects the different severity level of the behaviors included in the scale.

Construct validity

Factor validity

Two models were tested using CFA. The first model was a one-factor structure, commonly used in delinquency research, especially when using variety scales (e.g. Bendixen and Olweus 1999; Trinkner et al. 2012; Weerman 2011). The second alternative model tested was based on the seriousness of the behaviors and it was a two-factor structure, composed by serious and minor infractions (see Table 1 for information on the items’ composing each of the factors), which is also commonly used (e.g. Bendixen and Olweus 1999; Peck 2013; Weerman, and Bijleveld 2007). The results of the CFA showed adequate goodness-of-fit indices for both models, with the two-factor structure showing a slightly better fit to the observed data in all the indices considered (χ ²/df = 2.437, p <0.001; CFI = 0.951; TLI = 0.944; RMSEA = 0.041, I.C.]0.036–0.047[; WRMR = 1.290) than the one-factor structure (χ ²/df = 2.591, p <0.001; CFI = 0.945; TLI = 0.938; RMSEA = 0.043, I.C.]0.038–0.049[; WRMR = 1.345). Despite these results, the correlation between serious and minor infractions in the two-factor model was very strong (r = 0.895, p <0.001), suggesting the absence of discriminant validity.

Convergent and discriminant validity

Convergent and discriminant factor validity were evaluated using the Average Variance Extracted (AVE, see Marôco 2014; Fornell and Larcker 1981). The AVE for the total score was 0.49, for serious infractions 0.54 and for minor infractions 0.48. These results indicate that both the total score and the two factors have appropriate convergent validity. However, as we suspected, the squared correlation between serious infractions and minor infractions was 0.80, which is larger than their individual AVE, and therefore indicates the absence of discriminant validity between these two factors in the present sample. Given these results, the one-factor structure is the most parsimonious and the one that more adequately fits the data. The standardized factor weights, as well as the items’ squared multiple correlations for the one-factor solution are presented in Fig. 1.

Fig. 1

Standardized regression weights (displayed on the left side) and squared multiple correlations (displayed on the right side) for the one-factor model

Reliability

The internal consistency of the DBVS was assessed using Cronbach α and Composite Reliability (CR). Cronbach α for the 19 items composing the scale was 0.829 and no significant improvements were found excluding any item. Following the indications of Fornell and Larcker (1981), the internal consistency of the scale was also examined using CR, which in the present sample was 0.90. Finally, the item/domain discriminating power was assessed through the corrected item-total correlations. All the items were significantly (p ≤ 0.001) and positively related with the total score, with correlation coefficients ranging between 0.20 and 0.52, although more than 2/3 of the items presented coefficients ≥0.40. These results strongly support the adequacy and reliability of the single-factor solution of the DBVS in the present sample.

Group differences and associations with related variables

Criterion validity was assessed through the scale’s ability to identify group differences, as well as its association with variables that have been consistently related in the literature with deviance and delinquency involvement, such as age and gender (e.g. Emler and Reicher 1995; Farrington et al. 2013; Gottfredson and Hirschi 1990; Hansen 2003; Junger-Tas, et al. 2003; Junger-Tas et al. 2004; Smith and McVie 2003), or school failure (e.g. Defoe et al. 2013; Emler and Reicher 1995; Farrington 2005; Hansen 2003) Starting with gender differences, the results of the Welch’s t test showed a statistically significant difference in deviant involvement (t _(675,10) = 6.824, p < 0.001), with boys reporting having engaged in a higher number of deviant acts (M = 4.59; SD = 3.84) than girls (M = 3.05; SD = 2.58). The results of the Welch’s t test also revealed a significant difference regarding school failure (t _(725,32) = 11.431, p < 0.001), with those who had been retained at school at least once reporting having engaged in a higher number of deviant acts (M = 4.92; SD = 3.72) than those who had never been retained (M = 2.54; SD = 2.24). In the same line, Pearson correlation coefficients showed that the deviance variety scores were significantly and positively related with the number of school retentions (r = 0.18; p < 0.001). Also as expected, age was positively associated with deviance (r = 0.23; p < 0.001). According to the literature (e.g. Emler and Reicher 1995; Farrington et al. 2013; Gottfredson and Hirschi 1990; Hansen 2003; Junger-Tas, et al. 2003), the engagement in deviant and delinquent activities escalates from pre-adolescence to mid-adolescence, when it reaches its peak (generally around 16 years old), and then starts to stabilize and decrease in the subsequent years. To see if we were able to find this developmental pattern using the DBVS, the average deviance variety score was calculated for each age cohort, separated by gender, and is graphically represented in Fig. 2. Results clearly show that the developmental pattern found with the DBVS is similar to the one found in previous research, except that girl’s involvement in deviance after 16 years old kept more or less stable instead of decreasing.

Fig. 2

Mean of the variety deviance score for each age-cohort, separated by gender

Finally, we wanted to examine the DBVS’s ability for detecting significant differences between the convicted and the non-convicted adolescents regarding their involvement in deviant activities and again the results of the Welch’s t test showed a statistically significant difference between the two groups (t _(91,11) = 5.84, p ≤ 0.001). Those who had already been convicted reported having been engaged in a higher number of deviant acts during last year (M = 6.43; SD = 4.53) than those who had never been convicted (M = 3.47; SD = 3.01). Since the composition of these two groups was significantly different in what concerns age and gender, and given the association between these variables and delinquency, it was important to control for their effect, including them in the analyses as covariates. So, although there was no homogeneity of variances between the two groups, given the robustness of the F statistic and the impossibility of including covariates in a non-parametric test, we performed an ANCOVA, with age and gender as covariates. The results showed that the difference between the two groups was still statistically significant (F ₍₃₈₅₆₎ = 40.89, p < 0.001, η ² _p  = 0.05), even after controlling for age (F ₍₃₈₅₆₎ = 58.72, p < 0.001, η ² _p  = 0.06) and gender (F ₍₃₈₅₆₎ = 47.80, p < 0.001, η ² _p  = 0.05).

Discussion

The purpose of this study was to develop a scale intended to measure the variety of deviant behavior–the DBVS–and evaluate its psychometric properties with a sample of Portuguese adolescents. To date, we haven’t found studies reporting the development of a Portuguese deviance variety scale, nor have we found studies reporting the psychometric properties or specifically addressing the validation of a deviance scale with Portuguese adolescents. With this study we intended to contribute to fulfill this gap. Although frequency scales are the ones most often used in deviance and delinquency research, our decision about developing a variety scale was based on the results of a comparative study by Bendixen et al. (2003), which has shown that variety scales have better psychometric properties as well as some practical advantages over frequency scales.

In order to analyze the construct related validity of the DBVS, two alternative models were tested using CFA: a single factor model frequently used in delinquency research (e.g. Sanches et al. 2012; Smith and McVie 2003; Sutton and Winnard 2007), particularly when a variety scale is used (e.g. Brown and Jennings 2014; Hirtenlehner, et al. 2014; Trinkner et al. 2012; Weerman 2011; Yu et al. 2013); and a two-factor model organized around two seriousness levels (minor and serious infractions) which is also common in variety scales (e.g. Bendixen and Olweus 1999; Peck 2013; Weerman and Bijleveld 2007). Overall, both models showed a good fit in our data, and although the two-factor model presented a slightly better fit, subsequent analysis showed that there was no discriminant validity between the two dimensions, therefore indicating that the single-factor solution was the most appropriate. Convergent validity of the single factor structure of DBVS was also confirmed, with AVE values on the threshold of adequacy.

Item analysis also revealed the overall psychometric adequacy of the items of the scale. Despite the expected low response frequencies of some of the more serious infractions (e.g. 3 items had response frequencies <5 %), which consequently displayed lower correlations with the total score, these items were not excluded. They presented appropriate factor weights and keeping them did not compromise the overall fit of the single-factor model. Furthermore, this pattern of results was already expected: serious infractions have much lower prevalence rates than minor infractions (see e.g. Bendixen and Olweus 1999), and it was our aim to develop a scale sufficiently varied, both in terms of the type of behavior, and in terms of its severity, so that it would be able to detect different levels of engagement in deviant activities. The DBVS has also shown good reliability, either through the Cronbach’s α or through the composite reliability indices. Finally, the scale’s association with related variables, such as the number of school years failed and age, which displayed the developmental pattern expected according to the literature, along with the expected group differences in deviant engagement, namely in what concerns gender and official convictions, attested for the criterion validity of the scale.

Although the DBVS has presented good psychometric properties, some limitations must be addressed. First, participants were selected by convenience and, despite our efforts in order to collect data in the most varied contexts (public, private and vocational schools), we cannot state that our sample is representative of the Portuguese adolescent population. Some age groups (the youngest and the oldest) are also underrepresented in our sample. This happened especially in the convicted group, which had no 19-year-old participants, and no female participants with 12 or 13, although this probably reflects the quite low prevalence of convicted girls at this age. Second, adolescents were assigned to the convicted group, according to a self-reported answer and it would be more reliable to compare the results of convicted adolescents, based on official records. Also stability over time, which is a good reliability indicator, was not examined in this study, and it would have been interesting to analyze the scale’s association with other related variables, such as self-control or the association with delinquent peers. Finally, the analysis of the cross-cultural factorial invariance of the scale would provide stronger evidence of its construct validity. More studies are therefore necessary to overcome these limitations and provide further validity evidence to the scale.

Conclusions

The results obtained in this study support the factorial, convergent and criterion validity of the Deviant Behavior Variety Scale, suggesting that this simple and short scale is a reliable measure to evaluate adolescents’ involvement in deviant activities.