Non-verbal intelligence outperforms selective attention in a visual short-term memory test

Abstract

Short-term memory is a dynamic psychological process that operates within a network in which non-verbal intelligence and attentional domains are connected. However, no consensus has been reached about which process has the greatest effect on this memory ability, which was the main objective of the present study. A sample of 1448 Brazilian participants (mean age = 26.62 years, standard deviation = 9.97 years; 53.9% females) were collectively tested on pen-and-paper standardized and validated measures of selective (ROTAS-C), alternating (ROTAS-A), and divided (ROTAS-D) attention. They also performed the R1 Non-verbal Intelligence Test and a visual short-term memory test (Memória Visual de Curto Prazo [MEMORE] test). The statistical analyses consisted of a data mining procedure, in which exhaustive automatic selection screening was performed. The results were compared with Corrected Akaike Information Criteria. The linear model met the classic assumptions of ordinary least squares and only included main effects of selective attention (standardized β = 0.39) and non-verbal intelligence (standardized β = 0.37) as main predictors (F_2,39 = 7.01, p < 0.01, adjusted R² = 24%). The results are discussed within a cognitive psychology framework.

Short-term memory (STM) is a vital neuropsychological process that refers to the ability to retain small amounts of information for a short period of time (Camina & Güell, 2017). Two main aspects of STM are the presence of (1) temporal decay and (2) a chunk capacity limit (Cowan, 2008). Much evidence suggests that STM is an important mediator of visual awareness, long-term phonological learning, and communication (Hambrick, Kane, & Engle, 2004; Leclercq & Majerus, 2010; Maljkovic & Martini, 2005). Accumulating evidence indicates that STM is required for almost every cognitive ability and plays a critical role in intelligence and attention (Fukuda & Vogel, 2010).

In turn, intelligence is one of the most studied psychological processes. There are many ways to conceptualize and define intelligence. Studies of intelligence were one of the landmarks of psychometric modeling in psychology. In 1904, Charles Spearman examined correlations among different ability tests and found positive intercorrelations among sets of test items. Spearman labeled this phenomenon “positive manifold” (Borg, 2018). According to Spearman, all human abilities comprise universal factors (g-factor) and specific factors (s factors). Recent findings in the literature have demonstrated the plausibility of this structure (Castejon, Perez, & Gilar, 2010; Cucina & Byle, 2017).

According to Spearman’s theory, intelligence can be viewed as an ability to efficiently adapt to the environment. Therefore, it involves learning from experience and recognizing and solving problems. Data support the stability of intelligence across the lifespan (Ramsden et al., 2011). Non-verbal intelligence refers to abstract reasoning, and its ability is partially independent of language (Anagnostou et al., 2013).

Attention is a multidimensional ability that involves the ability to focus selectively on some things while diverting focus away from others (Gazzaniga & Halpern, 2015). Its process allows attending to, filtering, and selecting among a continuous stream of information (Richard et al., 2020). Some controversies in the recent literature can be found with regard to taxonomic classifications (Posner & Boies, 1971). Nonetheless, there is growing agreement that selective, alternating, and divided attention are found in attentional processes. Selective attention refers to the ability to select and focus on particular inputs while simultaneously suppressing irrelevant or distracting information (Stevens & Bavelier, 2012). Alternating attention is frequently defined as the rapid shifting of attentional focus (Commodari, 2017). Divided attention is the ability to simultaneously produce competing responses to multiple cognitive inputs (Buchin & Mulligan, 2019; Moore Sohlberg, McLaughlin, Pavese, Heidrich, & Posner, 2000; Naveh-Benjamin, Guez, Hara, Brubaker, & Lowenschuss-Erlich, 2014).

Although evidence indicates interrelationships among these cognitive skills, the networks that underlie the structure of STM are still under debate (Engle, Laughlin, Tuholski, & Conway, 1999). Psychometric and statistical studies have explored relationships between memory and other psychological attributes but have not reached a consensus about which predictor has the greatest effect on STM (Anunciação, Portugal, Rabelo, Cruz, & Landeira-Fernandez, 2020; Richard et al., 2020). This gap in the literature impacts the theoretical building of cognitive processes and indirectly impacts clinical procedures that are related to STM.

The present study sought to determine the best predictor of visual STM, operationalized as the results of psychological tests of visual STM, attentional processes, and non-verbal intelligence. The methodological approach included a machine learning-based exploratory method that implemented a data mining algorithm.

Methods

The present study was part of a long-term research project that integrates the psychometric and computational modeling of memory and other cognitive skills. The present study involved observational/correlational research that implemented a relational and predictive design.

Participants

The sample comprised 1448 participants, 14–65 years of age (M = 26.62 years, SD = 9.97 years). Women comprised 53.9% (n = 781) of the sample (χ²₁ = 8.98, p < 0.01), and men comprised 46.1%. With regard to level of education, 4.1% (n = 60) of the participants had elementary school as their highest level, 34.6% (n = 501) had high school as their highest level, and 61% (n = 883) had incomplete or complete higher education (χ²₂ = 704.8, p < 0.01). Most of the participants were Brazilian (89%, n = 1289) from the southeast region (59.4%, n = 860; χ²₆ = 2701.8, p < 0.01). Table 1 shows the demographics of the respondents.

Table 1 Demographic data of the sample

Procedure

Data collection occurred between 2013 and 2019. This procedure mainly relied on the voluntary participation of undergraduate and graduate students, and compulsory assessment of individuals who applied for the national driver’s license or were undergoing psychological evaluation for civilian gun ownership.

Because of Brazilian federal legislation, a psychological assessment is compulsory for everyone who applies for a driver’s license and gun ownership. This assessment is performed within 1 day. It occurs in specialized clinics or centers and should be administered by a certified psychologist. According to current legislation, psychologists administer a clinical interview and also administer psychological tests to assess non-verbal intelligence, attention and memory skills, and personality traits.

The participants in the present study were recruited from these clinics and invited to participate. All of the tests were collectively administered in pen-and-paper format. The time spent to perform a task did not last more than 30 min.

All researchers’ e-mail addresses were available to the participants if they had any questions, but no events were reported. This study was approved by the ethical committee of São Francisco University (USF), Brazil (protocol no. 0058.1.142.186-11).

Instruments

Visual short-term memory test (Memória Visual de Curto Prazo [MEMORE] test)

The MEMORE test evaluates the ability to memorize, recall, and discriminate a previously seen stimulus and then retain it for a brief period of time. Participants are first required to memorize 12 colored circles for 1 min. After a divergent interference task that lasted about three minutes, in which the participant provided personal details on the first page of the activity, the participant was then asked to recall the 12 circles among a list of 24 circles (12 of the circles were previously seen) in any order for 2 min. No cues were presented. Figure 1 shows some of the circles that were used in the test. The test scoring procedures were based on signal detection theory, in which hits (true positive), misses (false negative), false alarms (false positive), and correct rejections (true negative) are possible. The number of false negatives (i.e., a previously presented circle was not checked) and false alarms (i.e., an absent circle was checked) are subtracted from the number of hits (i.e., a previously presented circle is checked) and correct rejections (i.e., an absent circle was not checked). The results were on a gradual ordinal scale and could vary from − 24 to + 24, in which higher scores indicated better performance.

Fig. 1

Colored circles used in the test (adapted stimuli are shown because of copyright restrictions)

Psychometric analysis MEMORE data suggested a one-dimensional structure (χ²₂₅₂ = 1255.530, p < .001, root mean square error of approximation = 0.048, GFI = 0.860, MIREAL = 0.287), with adequate internal consistency (ordinal Cronbach’s α = 0.76) and high stability (test-retest correlation = 0.82). Other statistical information is available in Anunciação et al. (2020).

The selective (ROTAS-C), alternating (ROTAS-A), and divided (ROTAS-D) attention battery (Rabelo, Cruz, & Castro, 2020)

This pen-and-paper battery consists of three cancelation tests. All of the tasks evaluate an individual’s attentional capacity. Participants need to follow a specific pattern that is drawn on paper in 2 min. The Selective Attention Test (ROTA-C) investigates the participant’s selective attention by asking the participant to select only one target stimulus among various distracting stimuli. The Alternating Attention Route Test (ROTA-A) evaluates the participant’s ability to switch attentional focus from one stimulus to another. The Divided Attention Routes Test (ROTA-D) provides a measure of an individual’s ability to share attention, defined as the ability to mark more than one different stimulus in different ways simultaneously. The theoretical model of this task is based on Sohlberg and Mateer (Moore Sohlberg et al., 2000; Sohlberg & Mateer, 1987). The scoring of this battery is performed by subtracting the errors plus the omissions from the number of correct responses

The Brazilian validation studies relied on a sample of 1251 people who were assessed between 2013 and 2018 throughout Brazil. Psychometric analyses suggested an excellent test-retest correlation (r = 0.9) and high internal consistency (Cronbach’s α = 0.80). The three domains of attention were significantly correlated (selective vs. alternating, r = 0.55; selective vs. divided, r = 0.61; alternating vs. divided, r = 0.54). A linear negative trend was found between age and performance. Older participants had lower results (r = 0.41). In turn, highly educated participants had higher scores than participants with only an elementary or high-school education. Females scored higher than males. Other statistical information is found in Rabelo et al. (2020). Figure 2 describes the tasks.

Fig. 2

Cancelation tasks for attentional measure (adapted stimuli are shown because of copyright restrictions)

R1 Non-verbal Intelligence Test (da Silva, 2014)

This is a Raven’s Matrices-based test that is widely used to measure an individual’s intellectual capacity. The task was developed based on the g-factor theory of intelligence and is frequently used in Brazilian specialized traffic clinics with the purpose of selecting amateur and professional drivers. The test takes approximately 30 min to complete. Figure 3 shows an approximation of the activity.

Fig. 3

Intelligence tasks (adapted stimuli are shown because of copyright restrictions)

Statistical analyses

All of the data were initially checked to ensure consistency. Outliers were not removed, and missing data were not at random and thus were not imputed. Categorical variables are expressed as frequencies and percentages. Continuous variables are expressed as the mean (M) and standard deviation (SD), quartiles, and median absolute values. In accordance with the respective user guides, the raw scores of each cognitive measure were transformed to composite scores. This transformation is described in the instrument’s subsections.

We relied on automatic selection based on Corrected Akaike Information Criteria (AICc), implemented in the glmulti package, to study the relationship between visual short-term memory (i.e., the dependent variable, considered interval-level for the purposes of data analysis) and results of the selective, alternating, and divided attention and non-verbal intelligence tasks. GLMulti heuristics involve an exhaustive screening that combines all independent variables and then compares the AICc to choose the best predictors (i.e., the model with the lowest AICc). When the tested model included an interaction term (i.e., selective attention × divided attention), the corresponding main effects were also included in accordance with the principle of “marginality” (Calcagno & de Mazancourt, 2010).

The regression assumptions were tested via visual inspection, in addition to the Shapiro-Wilk test for normality of the residuals, the Breusch-Pagan test for heteroskedasticity, and the variance inflation factor (VIF < 10) for multicollinearity.

An alpha level of .05 was used to avoid type 1 errors. The analyses were performed using R 4.0 software with the tidyverse (Wickham, 2016), glmulti (Calcagno & de Mazancourt, 2010), and olsrr (Hebbali, 2018) packages. Codes are available at https://osf.io/wyevt/

Results

The means, SDs, and other statistics for the observed data for all of the instruments are presented in Table 2.

Table 2 Means, standard deviations, and other descriptive statistics of raw scores for all instruments

Exhaustive screening among all possible linear models to fit the relationship between the results of the visual short-term memory [MEMORE] test and all other measures was performed using the glmulti genetic algorithm (Table 3). By default, this method uses all possible combinations, compares each computed AICc and its related weight, and then ranks the results in accordance with the lowest AICc.

Table 3 Top ranked predictors of visual short-term memory results

The algorithm computed all sets of combinations of candidates. The model fit from the best ranked models slightly favored the first (visual memory ~ intelligence + selective attention). Despite the similarity between the two well-ranked models, the first was substantially more parsimonious and included results from the non-verbal intelligence test and selective attention test (ROTAS-C; AICc = 244.05, weight = .14).

The estimated importance value for a particular predictor is regarded as the overall support for each variable across all models in the candidate set. Because its results are equal to the sum of the weights/probabilities for the models in which the variable appears, a predictor that appears in several models with large weights will receive a high importance value. The vertical dashed line at .80 in Fig. 4 is often used as a cut-off to differentiate between the important and less important variables.

Fig. 4

Model-averaged importance of predictors

Table 4 presents the results of the final regression model, in which ~ 28% of the variance of the results of the visual short-term memory test (MEMORE) was accounted for by the predictors (F_2,39 = 7.01, p < 0.01). The normality of the residuals was visually examined through a Q-Q plot and the Shapiro-Wilk test (W = .97, p = .6), suggesting that normality was not violated. The Breusch-Pagan test for heteroskedasticity was performed to check whether variance was constant. The results suggested homoscedasticity of the model (p = .15). The VIF values were < 5 (tolerance = .998), indicating that no multicollinearity was present.

Table 4 Final multiple linear regression model

The standardized coefficient enables comparisons of the effect size of each predictor. A larger value denotes a greater effect on the dependent variable.

Discussion

Short-term memory is one of the most important neuropsychological processes. The maintenance of information over a short period of time is needed in almost every cognitive process. However, only a few studies have investigated the role of attention and intelligence in visual short-term memory ability, especially in non-clinical samples. The present study took advantage of a series of psychometric validation studies of cognitive measures that are used in the context of compulsory psychological assessment. Our results indicated that the unique contributions of selective attention and non-verbal intelligence are the best candidates over a set of different models.

Previous studies established a relationship between memory and other cognitive skills, such as intelligence and attention (Engle et al., 1999; Veer, Luyten, Mulder, van Tuijl, & Sleegers, 2017; Zanto & Gazzaley, 2009). Non-verbal intelligence and selective attention are both crucial constructs that are used to predict performance on various complex cognitive tasks. The present results indicated that intelligence had a greater effect size, which was partially consistent with the findings of Alosco et al. (2014), Fenn and Hambrick (2015), Miller, Spitznagel, Hughes, Rosneck, and Gunstad (2018), and Haavisto and Lehto (2005), although the first studies were performed with clinical samples, and the latter studies relied on data from Air Force recruits. In our study, we also used validated measures in which the sample size was equal or larger than the original studies.

Alternating and divided attention were shown to be predictors that were not selected from the data mining procedure. This result can be justified by previous neuroscientific studies. Although there are close links between attentional mechanisms, alternating and divided attention are known to disrupt memory encoding (Buchin & Mulligan, 2019; Naveh-Benjamin et al., 2014).

Considering the previous findings, this outcome corroborates the multidimensionality of attentional processes. Therefore, although correlations between attentional abilities are substantial, their effects on STM are dissimilar, stressing differences between these aspects of attention.

These outcomes are not universal or free of academic dispute. Other findings suggest that selective attention can be a moderator of the relationship between intelligence and short-term memory (Unsworth & Engle, 2005). These authors proposed that directed attention assumes a pivotal role in orienting memory maintenance (Oren et al., 2016) and activating higher-order visual areas that are needed to process stimuli (Lepsien, Thornton, & Nobre, 2011).

When focusing on intelligence as a predictor of visual short-term memory, a partial explanation can be provided by the g-factor theory. Evidence from several research fields indicates that the g-factor has high explanatory power in a wide range of dimensions of behavior and across diverse mental tasks (Kärner, 2017). Thus, higher intelligence can lead to better strategies during the encoding phase of the short-term memory process, improve memory storage capacity, and enhance mental speed that is needed to perform memory tasks (Colom, Abad, Quiroga, Shih, & Flores-Mendoza, 2008).

The present study had two characteristics that limited a deeper understanding of the interplay with the broader literature. First, our results were based on psychometric modeling and thus were gathered from cognitive tests that were designed to operationally analyze a particular theory. The cognitive mechanisms that underlie the relationships among psychological constructs are still unknown, and multiple and competing definitions of the same constructs can be found (Jurado & Rosselli, 2007). As often reported in psychology, slight changes in the way one conceptualizes and measures these constructs tend to lead to substantial differences (Sijtsma, 2012). Thus, the results of the previous studies on which we relied in the present study may also reveal theoretical assumptions that are different from our assumptions, thereby limiting the conclusions that can be drawn. However, we stress that this limitation is frequently found in psychometric studies.

We consider that visual short-term memory is still available after 1–3 min, which is a known attribute of tests that are used among clinical neuropsychologists and neurologists in screening procedures (Buschke et al., 1999; Kuslansky, Buschke, Katz, Sliwinski, & Lipton, 2002). However, this contrasts with some assertions that the duration of short-term memory is only 15–30 s when no form of memory rehearsal is present (Alvarez & Cavanagh, 2004; Jiang, Olson, & Chun, 2000). Moreover, definitions that derive from multiple attention models in the literature, in which selective attention is considered to involve the maintenance of a cognitive/behavioral set despite competing stimuli, are not necessarily consensual (Sohlberg & Mateer, 1987; Stevens & Bavelier, 2012).

The statistical approach that was used to guide the results relied on a computer-aided algorithm, based on the notion of parsimony that is achieved by a lower AICc. Parsimony argues that a model with fewer predictors is preferred over a model with numerous predictors if the latter yields a nonsignificant, negligible, or trivial increase in explanatory power. Despite the use of the AICc to define the model that fit the data best and that is typically implemented in linear models (Calcagno & de Mazancourt, 2010; Hasan Örkcü, 2013), the specific criterion that is applied by the researcher is arbitrary. In the fields of psychology and other empirical sciences, this is still an open research question, and many argue that these models can give markedly different results based on the same data. We are also aware that statistical models help the understanding of psychological processes, with varying degrees of fit from reality.

However, these issues are not particular features of only the present study and instead are a frequent condition in other psychometric endeavors (Sijtsma, 2012). Thus, the instruments that were used in the present study have sound psychometric studies to minimize these limitations (including the clinical utility), and the automatic procedure exhaustively tested all predictor combinations (Calcagno & de Mazancourt, 2010).

The present results shed further light on cognitive processes and may be of interest especially to cognitive psychologists and neuropsychologists. Unveiling the best predictors of STM could aid the theoretical building of cognitive processes and indirectly impact clinical procedures that are related to STM, which plays an important role in psychological processing.

Conclusion

The present study sought to integrate psychometric and statistical modeling to identify and compare effect sizes of results from attentional and non-verbal intelligence tests on the results of a visual short-term memory test. Despite the methodological challenges of studying relationships between latent variables, computer-aided model selection was based on the lower value of the AICc, suggesting that the main effects of non-verbal intelligence and selective attention were the best predictors of visual short-term memory results.

The effect size of non-verbal intelligence was higher than the effect size of selective attention when both processes were compared. Alternating and divided attention were also explored, but they were not significantly related to the outcome. Further research is needed to determine the reliability and generalization of these findings.