The incidental binding of color and shape is insensitive to the perceptual load

Abstract

The binding of information in visual short-term memory may occur incidentally when irrelevant information for the task at hand is stored together with relevant information. We investigated the process of the incidental conjunction of color and shape (Exp1) and its potential association with the selection of relevant information to the memory task (Exp2). The results in Exp1 show that color and shape are incidentally and asymmetrically conjugated: color interferes with the recognition of shape; however, shape does not interfere with the recognition of color. In Exp2, we investigated whether an increase in perceptual load would eliminate the processing of irrelevant information. The results of this experiment show that even with a high perceptual load, the incidental conjunction is not affected, and color remains to interfere with shape recognition, suggesting that the incidental conjunction is an automatic process.

Background

Working memory is organized in terms of dissociated storage systems controlled by the central executive, an attentional supervisor. The storage of verbal information is performed by a system that is independent of the one responsible for visual-spatial storage (Baddeley and Hitch 1974; Baddeley 2000; 2007). Evidence also suggests the dissociation between storage systems for visual and spatial information (Klauer and Zhao 2004; Logie 1995). However, the specialization in information storage according to the modality of stimulus does not eliminate the possibility that there is an interaction between systems, making it possible to associate a name with an object, an object with a place, and so on. A model based on dissociated memory systems requires a system capable of storing integrated representations, or at least links between the different dimensions stored in different systems. These integrated representations or association of links define an object as it is presented to our consciousness when we recall, for example, that we saw the car key on the white office desk.

The binding or conjunction of information in the short-term memory allows a single multimodal representation to contain or to refer to representations present in different storage modalities. The binding of information of different modalities on the short-term memory is of fundamental importance for most cognitive functions such as associative memory (Craik 2006), planning and execution of sequences of movements (Annett 1995), face and name association (Godoy and Galera 2011), or shape and color association (Parra et al. 2011). Despite its importance, little is known about the nature of the information integration process of different modalities in the short-term memory.

One issue that has received attention in the study of the integration of information in short-term memory is the dependence of this process on central executive resources. The involvement of these attentive resources in information conjunction in the working memory has been investigated through experimental paradigms that use the strategy of increasing cognitive load during the performance of a main task that requires information conjunction. In this paradigm, the main task is performed simultaneously with a competing task that demands attentive resources for its concretization. Studies that have used increased cognitive load during conjunction tasks have found that increasing the cognitive load does not affect voluntary binding. The performance of tasks that require conjunction is no more affected by the completion of a competing task that demands attention than is the performance of memory tasks for isolated characteristics. This suggests that the conjunction occurs automatically, independent of extra attention resources that may assist the association of separate characteristics into a single representation of an object, event, or chunk (Allen et al. 2006; Allen et al. 2009; Luck and Vogel 1997).

Starting from the concept that the conjunction of an object is automatically performed, Morey (2009) suggests that there are two possible explanations for the retention of integrated information in the memory. The first is that the association of different characteristics or dimensions is performed by a unified structure and stored as a unit of information in the working memory. An example is the association between a name and an object. The second hypothesis assumes that the storage of conjugated characteristics is performed in parallel, and both integrated and isolated information are kept in the memory through different mechanisms.

The hypothesis that the conjunction of characteristics creates a representation of a single object in the memory has been investigated through intervening tasks, tasks which have the same nature as the relevant object to be stored. In Morey (2009), for example, the main task required the memorization of spatially arranged letters on a computer screen. The main task was accompanied by a secondary task of articulatory suppression, which interferes with the verbal rehearsal of the letters. These experiments demonstrated that the storage of verbal and spatial information occurs separately because negative effects on storage were observed in the verbal task but not in the spatial task in trials were the articulatory suppression was performed. However, the discrimination of bound objects was better than the discrimination of separated objects, suggesting that both the unitary and separated information can be stored simultaneously.

The process of conjunction, regardless of being automatic or dependent on attentive resources, can occur either as voluntary or involuntary, or incidental. The incidental conjunction can be qualified as a concurrent storage of neutral or irrelevant information to perform cognitive tasks. Several studies that have addressed incidental conjunction indicate that it occurs asymmetrically (Elsley and Parmentier 2015; Jiang et al. 2000; Maybery et al. 2009; Olson and Marshuetz 2005). The asymmetry of the incidental conjunction lies in the relationship in which the memorization of a first characteristic (e.g., location) implies in the incidental codification and retention of a second characteristic, for example, of the consonant presented at the location to be memorized. However, the reverse does not happen. The consonant codification does not imply in the incidental codification of the location (Maybery et al. 2009). Santana and Galera (2014), in a study of visual, verbal, and spatial conjunction information in the working memory, also found evidence of an asymmetric effect of irrelevant dimension on the recognition of locations and visual stimuli; these authors suggest that the information about the object and its location are automatically integrated into a unitary representation.

Incidental conjunction occurs when an irrelevant characteristic of an ongoing task is encoded together with the relevant characteristic. We consider that this situation can be regarded as a failure in the selection process of relevant information that could be explained by the load theory proposed by Lavie (1995). According to this theory, the selection of information in the environment is affected by the perceptual load the participant is exposed to during the selection of relevant information. The theory of perceptual load assumes that perception has a limited capacity but operates automatically and involuntarily, respecting these limits on all information available in the environment. In tasks carried out under high perceptual load, the system’s capacity is filled to the limit, eliminating unattended information from the highest processing level. Conversely, in tasks performed under low cognitive load, even irrelevant information may be admitted at high processing levels (Lavie 1995). With respect to selection models of relevant information, the early selection occurs when the level of perceptual load required by the processing of relevant information is sufficiently high to exhaust the perception capacity, leaving no resources for processing distractors. On the other hand, the delayed selection of relevant information occurs in cases of low perceptual load, wherein attentional resources not used for the processing of relevant stimuli are used to process less relevant information for the task, among them, distractors (de Fockert et al. 2001).

The effects of perceptual load have been demonstrated in situations in which irrelevant items are added to a task, making the selection of the relevant stimuli more difficult. The increase in the number of irrelevant stimuli or in perceptual demands related to these stimuli saturate the processing capacity, thereby blocking the access to irrelevant information. The involvement of the perceptual load was demonstrated by Forster and Lavie (2008) in a study were the irrelevant stimuli, evoked responses that competed with those required by the main task. Their results showed that both relevant and irrelevant stimuli are selected and interfere with the processing of relevant information.

In this study, we sought to evaluate the effect of perceptual load on the incidental conjunction of shape and color in a recognition task based on Treisman and Zhang (2006). These authors used a detection task to determine the interference of changing an irrelevant characteristic of an object, for example, its spatial location, on the recognition of a relevant characteristic, for example, the object’s shape or color. The authors’ assumption was that if it were possible to demonstrate that changing the position of an object interferes with its recognition, one could conclude that the object and its position were memorized in an integrated representation formed by both the “what it is” and the “where it is”. Indeed, the results indicated that the integration of visual characteristics is mediated by the conjunction between position and visual characteristics. Because location is not relevant to the task, these results suggest that this combination is automatically or incidentally performed. The same logic was used by Maybery et al. (2009).

Using an adaptation of the experimental paradigm of Maybery et al. (2009), we to investigate the influence of irrelevant information on the memory task by determining the occurrence of an incidental conjunction of color and shape in the visual short-term memory. Although the existence of incidental conjunction is well documented in the literature (Elsley and Parmentier 2015; Maybery et al. 2009; Olson and Marshuetz 2005), there are still some situations where it does not occur. For example, Stefurak and Boynton (1986) show no evidence of incidental conjunction of shapes and colors in a recognition task. Ecker et al. (2013) conversely, found evidence of incidental conjunction of color and shape, but only in conditions in which the color was intrinsic to the memorized figure. Therefore, the objective of this first experiment was to verify the incidental conjunction between color and shape, and a possible asymmetry in a recognition task for colors and shapes.

Experiment 1

Method

Participants

A total of 13 undergraduate students (eight women), aged between 20 and 29 years (M = 23.5, SD = 2.9), with normal vision or vision corrected to normal (20/20) participated voluntarily in this experiment.

Materials and stimuli

The visual stimuli were colorful geometric shapes presented on a white background in the center of a 23″ monitor screen with a resolution of 1360 × 768 pixels. The experiment was assembled and run on the E-Prime 2.0 utility (Schneider et al. 2002).

Figure 1 shows the eight geometrical shapes used (circle, triangle, flag, arrow, cross, diamond, star, and bow), each with an area of approximately 160 × 160 pixels. The eight colors defined by RGB values were gray (128, 128, 128); green (0, 255, 0), red (255, 0, 0), blue (0, 0, 255), black (0, 0, 0), cyan (0, 255, 255), purple (153, 51, 153), or yellow (255, 255, 0). The stimuli were presented in an imaginary circle with a radius of 4.5 cm from the screen center, and a minimum distance of 60° between stimuli. The shapes and colors were based on Allen et al. (2006).

Fig. 1

Shapes and colors used in experiment 1

Procedure

The participants performed a recognition task based on the shape and color of geometric shapes. Each task began with a set of three colored shapes simultaneously presented for 1000 ms. The screen remained blank during the retention interval (900 ms) between the presentation of the stimuli to be memorized and the test stimulus. A tone (546 Hz) was presented for 100 ms at the end of the retention interval signaling the presentation of the test stimulus. The participants’ task was to respond as to whether the test stimulus was equal or not to one of the stimuli presented in the memorized set. The response was emitted on the numerical computer keypad. If the test stimulus was present at the memorized set (positive trials), the participant should press the number 1 key, otherwise (negative trials) he should press the number two key. Half of the trials were positive and half, negative. Participants were instructed to respond as quickly and accurately as possible.

Participants performed the recognition task in two different experimental blocks. In one, they were instructed to memorize the colors of geometric shapes and ignore their shapes; in the other, they were instructed to memorize shapes and ignore colors. The order of block presentations was counterbalanced among participants. In the block of relevant color, three types of positive trials (intact, recombined, and new irrelevant) and two types of negative trials (recombined and new irrelevant) were performed. In all positive trials, the presented test stimulus was one of the three colors presented for memorization. In the intact positive trials, the test stimulus was one of the memorized colors presented in the same shape that it had been memorized. In the recombined positive trials, the test stimulus was one of the memorized colors presented with one of the other shapes presented for memorization. In the new irrelevant trials, the test stimulus was one of the memorized colors presented in a different shape from that in the three memorized forms. Equivalently in the recombined negative trials, one color outside of the memorized set was presented with one of the shapes in the set; in the negative trials with the new irrelevant dimension, one color outside of the set was presented in a shape also outside of the set presented for memorization.

In the block in which shape was relevant, three types of positive trials (intact, recombined, and new irrelevant) and two types of negative trials (recombined and new irrelevant) were conducted. In the intact positive trials, the test stimulus was one of the shapes presented for memorization in the same color as that of the memorized stimulus. In the recombined trials, the shape was presented in one of the other two colors presented during memorization. In trials with the new irrelevant dimension, the shape was presented in a color different from those three presented for memorization. In the recombined negative trials, one shape outside of the memorized set was presented together with one of the colors in the set. In the negative trials with the new irrelevant dimension, one shape outside of the set was presented in a color also outside the memorized set.

A total of 144 valid trials were performed, equally divided between the two blocks, using relevant shape and color. The positive trials were equally divided in the intact, recombined, and new irrelevant dimension. In the negative trials, 18 with the repeated irrelevant dimension and 18 with a new irrelevant dimension were performed in each block.

Results

The percentage of correct answers was submitted to an analysis of variance (ANOVA) for repeated measures (2 × 5) considering the treatments resulting from the combination of relevant dimensions (shape and color) and types of trials (intact, recombined, and new irrelevant in the positive trials; recombined and new irrelevant in the negative trials). This analysis did not show a significant difference between relevant dimensions (color and shape) and between types of trials. However, a significant interaction between relevant dimensions and types of trials was observed (F(4.48) = 2,64; p < 0.05; η² _P = 0.18). To better understand this interaction, an analysis evaluating only the most critical conditions for incidental conjunction was performed. The rate of correct responses in the trials where the relevant dimension was shape and color with intact and recombined positive responses was considered separately in this analysis.

No significant difference between intact and recombined trials was found in the analysis of responses when color was the relevant dimension (F <1). However, when shape was the relevant dimension, the performance was better in intact trials (M = 95 %; EPM = 2 %) than in recombined trials (M = 85 %; EPM = 3 %) (F(1, 12) = 6.18; p < 0.05; η² _P = 0.34) (Fig. 2).

Fig. 2

Accuracy and reaction time as a function of the relevant dimension in the recognition task (Color, Shape) for Intact and Recombined stimuli in Experiment 1; (*) P < 0.05

The response time was subjected to the same analysis considering the type of positive trials (intact, recombined) separately for each of the relevant dimensions (shape and size). No significant difference was observed between trials when color was the relevant dimension (F <1). An advantage was observed in the intact trials in the analysis when shape was the relevant dimension; participants were faster in these trials (M = 922 ms; EPM = 64 ms) than those in recombined trials (M = 1127 ms; EPM = 86 ms) (F (1, 12) = 8.94; p <0.05; η² _P = 0.42).

Discussion

The objective of this experiment was to verify the incidental conjunction of color and shape in the visual store. Our results show that when shape was relevant, the participant’s performance was related to the object’s color, once in the intact condition, in which color and shape were the same, the rate of correct responses was greater (95 %) than in recombined trials (85 %), when shape was presented in the color of another item. Conversely, when color was the relevant dimension for memorization, no difference between the intact and recombined conditions was observed. This response pattern demonstrates the occurrence of incidental color codification when shapes should be memorized, and their asymmetrical relationship. According to these results, we can assume that the memorized information for shape incidentally integrates color while the memorization of color does not integrate shape. This pattern is also found in tasks with incidental relationships between sound and location (Maybery et al. 2009) and color and location (Jiang et al. 2000).

In previous studies, such as Stefurak and Boynton (1986), evidence of incidental conjunction of shape and color was not found when participants were instructed to memorize only one characteristic (color or shape of animals). In this study, the test stimulus could be identical to the memorized stimulus, or it could have a different irrelevant dimension from that presented during memorization. Stefurak and Boynton (1986) do not find a difference in performance in trials in which the irrelevant dimension was modified using a procedure very similar to ours. It is possible that these differences between results can be attributed to minor differences in procedure, for example, the use of animal figures and longer retention intervals, which may have allowed a contribution from the long-term memory to override the effect of incidental color conjunction.

Our study presented a pattern similar to that reported by Ecker et al. (2013). The results obtained by these authors about intrinsic color conjunction and our results demonstrate that the manipulation of color affects the recognition of shape and that color recognition is not affected by shape manipulation. The uniformity of responses in the testing conditions suggests that people can make the judgment for colors without processing the shapes presented.

This first experiment confirmed our hypothesis that there is an incidental conjunction of shape and color, and that, as in other involuntary conjunctions, this type of conjunction is asymmetric, once the recognition of only shape is affected by the irrelevant change in color.

Experiment 2

Having established because we detected the incidental conjunction between color and shape as asymmetric, we investigated the assumption that this incidental conjunction follows the same principles of the perceptual load theory. According to this theory (Lavie 2005), the increase in perceptual load during codification leads the participant to prioritize the most relevant information to respond to the main task, which consequently could reduce the incidental codification of color. Our assumption was that the increase in perceptual load would lead to a more careful selection of shape. Therefore, we added irrelevant colored stimuli to the scene to be memorized that would not be taken into account in the memory task. We believed that the presence of these stimuli would lead to a more stringent selection of the relevant characteristic, and thus, a more efficient selection of shape, reducing the effect of the incidental conjunction of color.

Method

Participants

A total of 11 undergraduate students (five women), aged between 19 and 31 years (M = 24.1, SD = 4.1), with normal or corrected vision, took part voluntarily in this experiment.

Materials

All materials and stimuli were the same ones used in the first experiment with the following exceptions: the size of objects was reset to 50 × 50 pixels and the circle was replaced by a diamond to avoid confusion with the irrelevant colors presented.

Three gradient colored circles without defined edges were added to the scene to increase the perceptual load. The GIMP 2.8.10 image manipulation program was used to prepare these stimuli. Colorful circles (50 × 50 pixels) were processed in a way that color intensity decreased gradually from the center (full color) towards the edge (in white), creating a blur with no defined edges.

Procedure

Only the shape recognition task was performed in this experiment. Three colored shapes were presented for memorization in trials with a low perceptual load. In the trials with high perceptual load, the three shapes were presented for memorization interleaved by three gradient colored circles with different colors from those presented in the colored shapes to be memorized.

Participants were instructed to memorize only the geometric shapes with well-defined edges ignore colors in which they were presented, and also ignore irrelevant stimuli represented by circles in a color gradient.

The stimuli were randomly presented in six positions around the fixation point on an imaginary circle with a radius of 3 cm, separated from one another by a distance of 60 degrees.

An articulatory suppression task was also added in order to avoid the vocal repetition of presented shapes; the “1, 2, 3” and “2, 3, 4” numerical sequences were used for this task.

A numerical sequence was presented at the beginning of each trial to start the articulatory suppression for 1000 ms; three colored shapes were subsequently and simultaneously presented for 900 ms followed by a 900 ms retention interval. At the end of the retention interval, a tone of 100 ms indicated the presentation of the test stimulus, which remained on the screen until the participant’s response. The response was given through the numerical computer keyboard; the participant should press the number 1 key if the test stimulus was equal to one of the items presented for memorization, or press the number two key if the test stimulus was different from those presented for memorization.

Results

The percentage of correct responses was analyzed using an ANOVA and considering load types (low and high) and five types of trials (intact, recombined, and new irrelevant in the positive trials, and recombined and new irrelevant in the negative trials). The overall analysis showed no difference in performance as a function of the type of perceptual load used. However, it showed differences in the type of trials (F(4, 40) = 12.55; p <0.001; η² _P = 0.55). The post hoc Tukey’s test showed that the performance in the recombinant trials was inferior to all other types of responses. There was no interaction between the type of load and trials.

A second analysis was performed considering the rates of correct responses in the positive trials (intact and recombined) and perceptual loads (high and low). No difference was found between perceptual loads (F < 1). Significant differences were observed in the analysis of intact and recombined trials (F(1,10) = 15.41; p < 0.001; η² _P = 0.60). The participants had a better performance (M = 83 %; EPM = 4 %) in intact trials, in which the color of the shape in the test stimulus was the same as that presented during memorization than in the recombined trials (M = 66 %; EPM = 3 %) (Fig. 3).

Fig. 3

Accuracy and reaction time for the recognition of Intact and Recombined stimuli at Low and High Perceptual Load in Experiment 2; (*) P <0.05

The response time was subjected to the same analysis considering the types of perceptual loads (low and high) and types of positive trials (intact and recombined). The performance did not vary according to the perceptual load (F < 1). Participants were faster (M = 880 ms; EPM = 46 ms) in the intact trials than in the recombined trials (M = 995 ms; EPM = 49 ms). (F(1, 10) = 16.59; p < 0.05; η² _P = 0.62).

Discussion

In this experiment, we analyzed the influence of perceptual load in the incidental conjunction of color during the memorization of shape. The difference in performance between intact (83 %) and recombined trials (66 %) confirmed the incidental conjunction of color when the shape is the relevant dimension. Our results also show that the increase in perceptual load obtained by introducing irrelevant items during the presentation of colored shapes has no effect on the incidental conjunction of color.

A high reduction or even elimination of the neural and behavioral signatures of distractor images in brain regions specialized in the codification of these images is common in studies with high perceptual load. In tasks with high perceptual load, responses to visually striking images, such as high-contrast stimuli, are significantly reduced. Lavie (2005; 2010) suggests that the increase in perceptual load affects the participant’s performance by decreasing the influence of distractor stimuli in attentive tasks. However, in our experiment, we could not verify the effect of perceptual load possibly because the load can have different effects on memory in relation to the effects on attentive tasks. The attended stimuli possibly did not affect the prioritization of stimuli in the high perceptual load condition. It is also possible that the presentation of distractors in a gradient has facilitated the prioritization of relevant stimuli and did not overload the perception as expected, maintaining the low perceptual load in both conditions, and therefore, the incidental color conjunction.

General discussion

The main objectives of this study were to investigate the incidental coding within the same memory store using shapes and colors and to determine if this incidental codification is affected by the perceptual load. The incidental codification was evaluated through the difference in performance between intact and recombined conditions while the perceptual load was manipulated, making the task of identifying a relevant object more demanding through the insertion of irrelevant objects in the memorized scene (de Fockert et al. 2001; Forster and Lavie 2008).

A task modified from Maybery et al. (2009) was used in the first experiment to identify the incidental conjunction. In this paradigm, the author used spatial and auditory items working with different stores (visual-spatial sketchpad and phonological loop) whereas, in our study, we used visual stimuli (color and shape) to work only with the visuospatial sketchpad. Our findings show that the incidental coding was identified in the shape recognition given that changes in the irrelevant color dimension affected the performance in the shape recognition task, suggesting that visual information may suffer interference from the incidental conjunction within the same store. Conversely, changes in shape did not interfere with performance when color was the relevant dimension in the recognition task. Our results show asymmetry in relation to the incidental codification of color, similar to the asymmetry in the incidental conjunction of location in the work of Olson and Marshuetz (2005 This asymmetry indicates the existence of a hierarchy in maintaining relevant objects in our memory that are also related to memorized objects and not just between stores (Ecker et al. 2013).

The incidental color conjunction was more evident in the second experiment. One possible explanation for this result is that the presence of articulatory suppression made it more difficult to use the phonological loop, and thereby, the memorizing task became more dependent on the visual memory. Despite the incidental codification of color, the overall rate of correct responses was, in general, lower in the first than in the second experiment; this change is also probably due to the articulatory suppression. In this case, the articulatory suppression hampered the vocal rehearsal of information visually presented, causing the rate of correct responses to decline without the phonological loop aid.

The objective of the second experiment was to identify the effect of the perceptual load on incidental conjunction based the load theory (Lavie 2005) wherein the increased selection demand would reduce the effect of distractors. Therefore, our hypothesis was that the increase in perceptual load would decrease the incidental link between shape and color. However, the results of the second experiment showed no difference in the incidental color conjunction when the perceptual load was high or low, which indicates that the perceptual load does not interfere in incidental conjunction. This insensitivity to the perceptual load suggests that the incidental conjunction is an automatic process, independent of attention. It is also possible that the bound representation of shape and color forms a unified structure that is stored as a unit of information in the memory and, therefore, cannot be divided or suppressed (Morey 2009).

Conclusion

In conclusion, our results suggest the occurrence of an incidental conjunction of color and shape. This conjunction is asymmetric, and the coding of shape occurs together with that of color despite the fact that this information is not required by the task. Conversely, the coding of color is not dependent on the object’s shape. This asymmetry in the conjunction of shape and color deserves further investigation.