NEW WORKING MEMORY PARADIGM FOR NEUROIMAGING TESTING OF VISUAL AND VERBAL MODALITY UNDER DIFFERENT ATTENTIONAL INVOLVEMENT Текст научной статьи по специальности «Медицинские технологии»

Psychology. Journal of the Higher School of Economics. 2024. Vol. 21. N 3. P. 456-471. Психология. Журнал Высшей школы экономики. 2024. Т. 21. № 3. С. 456-471. DOI: 10.17323/1813-8918-2024-3-456-471

NEW WORKING MEMORY PARADIGM FOR NEUROIMAGING TESTING OF VISUAL AND VERBAL MODALITY UNDER DIFFERENT ATTENTIONAL INVOLVEMENT

N.S. OTSTAVNOVa, E.A. VOEVODINAa, S.M. MENINGa, R.R. URAZAEVAb, T. FEDELEa

a HSE University, 20 Myasnitskaya Str, Moscow, 101000, Russian Federation

b Moscow Institute of Physics and Technology, 9, Institutsky Lane, Moscow Region, Dolgoprudny, 141700, Russian Federation

Новая парадигма рабочей памяти для нейровизуализационного исследования зрительной и вербальной модальностей с различным уровнем вовлечения внимания

Н.С. Отставнов", Е.А. Воеводина", С.М. Менинг", Р.Р. Уразаеваь, Т. Феделе"

'Национальный исследовательский университет «Высшая школа экономики», 101000, Россия, Москва, ул. Мясницкая, д. 20

' Московский физико-технический институт, 141700, Россия, Московская область, г. Долгопрудный, Институтский переулок, д. 9

Abstract

Working memory (WM) is a cognitive function essential for short-term maintenance of information in a highly accessible state to support goal-directed behavior. The classical behavioral model of WM includes a visuospatial sketchpad, a phonological loop and the central executive. Neuroimaging studies selectively targeted the activity associated with maintenance and processing of

Резюме

Рабочая память (РП) — это когнитивная функция, необходимая для кратковременного хранения информации в легкодоступном виде для осуществления целенаправленного поведения. Классическая модель РП включает в себя визуально-пространственный блокнот, фонологическую петлю и центральный процессор. Нейровизуа-лизационные исследования обнаружили разную активность мозга для каждого отдельного компонента. Однако на текущий момент нет экспери-

This work/article is an output of a research project implemented as part of the Basic Research Program at the National Research University Higher School of Economics (HSE University).

Исследование осуществлено в рамках Программы фундаментальных исследований НИУ ВШЭ.

modality specific information. However, an experimental design is still missing that would enable the assessment of all components of WM drawing a holistic neuroimaging model. In this study, we propose a modified paradigm based on the classical retro-cue task, which allows disentangling the activity of all WM components, and in particular of the central executive. This paradigm consists of five conditions: passive perception, simple verbal storage, simple visual storage, alphabetical reordering (complex verbal) and mental rotation (complex visual). Testing on a cohort of 35 healthy adults, we obtained a similar workload for simple storage conditions with a low engagement of the central executive. A different workload was verified between the simple and complex conditions in both verbal and visual modalities. This experimental design provides the framework to assess the neural activity associated with the central executive components in different modalities and to address the question of a unitary or modality-specific central executive nature. Therefore, the paradigm is suitable for utilization in neu-roimaging to potentially advance our comprehension of the WM organization.

Keywords: working memory; modified retro-cue task; central executive; information processing.

Nikita S. Otstavnov — Junior Research Fellow, International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, MSc. Research Area: psychophysiology, working memory.

E-mail: [email protected]

Ekaterina A. Voevodina — Research Assistant, Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, MSc. Research Area: neurophysiology, aging, memory.

E-mail: [email protected]

ментальной парадигмы, позволяющей целостно оценить все компоненты РП с учетом их взаимодействия. В данном исследовании мы разработали модифицированную парадигму, основанную на классической задаче с ретро-подсказкой, которая позволяет разграничить активность каждого компонента РП, включая центральный процессор. Парадигма состоит из пяти условий: пассивное восприятие, простое хранение вербальной информации, простое хранение визуальной информации, упорядочение последовательности букв по алфавиту (сложное вербальное условие) и мысленный переворот матрицы на 90 градусов (сложное визуальное условие). Исследование, проведенное на 35 здоровых взрослых, показало, что парадигма позволяет получить сопоставимую точность ответов для простых условий с низкой вовлеченностью центрального процессора. Были обнаружены значимые отличия в решениях между простыми и сложными условиями для каждой модальности. Таким образом, экспериментальная парадигма позволяет разграничить нейрональную активность, связанную с компонентами модально-специфического хранения и центральным процессором, и, в частности, исследовать вопрос о едином центральном процессоре или двух различных для каждой модальности. Данная парадигма может быть использована для создания целостного понимания взаимосвязанной работы компонентов РП в исследованиях с применением технологий нейровизуализации.

Ключевые слова: рабочая память; модифицированная задача с ретро-подсказкой; центральный процессор; обработка информации.

Отставнов Никита Сергеевич — младший научный сотрудник, Международная лаборатория социальной нейробиологии, Институт когнитивных нейронаук, НИУ ВШЭ. Сфера научных интересов: психофизиология, рабочая память.

Контакты: [email protected]

Воеводина Екатерина Алексеевна — стажер-исследователь, Центр нейроэкономики и когнитивных исследований, Институт когнитивных нейронаук, НИУ ВШЭ.

Сфера научных интересов: нейрофизиология, старение, память.

Контакты: [email protected]

Semyon M. Mening — Assistant, Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience; Research Assistant, Laboratory for Cognitive Research, Faculty of Social Sciences, HSE University, BS. Research Area: psychophysiology, working memory.

E-mail: [email protected]

Renata R. Urazaeva — Student, Moscow Institute of Physics and Technology. Research Area: medical physics. E-mail: [email protected]

Fedele Tommaso — Lead Research Fellow, Institute for Cognitive Neuroscience, HSE University, PhD in Life Sciences. Research Area: psychophysiology, signal processing, memory. E-mail: [email protected]

Менинг Семен Михайлович — лаборант, Центр нейроэкономики и когнитивных исследований, Институт когнитивных нейронаук; стажер-исследователь, научно-учебная лаборатория когнитивных исследований, факультет социальных наук, НИУ ВШЭ. Сфера научных интересов: психофизиология, рабочая память. Контакты: [email protected]

Уразаева Рената Рустемовна — студент, Московский физико-технический институт. Сфера научных интересов: медицинская физика. Контакты: [email protected]

Феделе Томмасо — ведущий научный сотрудник, Институт когнитивных нейронаук, НИУ ВШЭ, PhD.

Сфера научных интересов: психофизиология, обработка сигналов, память. Контакт: [email protected]

Working memory (WM) is a cognitive function providing the maintenance of visual and verbal information in a highly accessible state, supporting goal-directed behavior (Brady et al., 2024) spanning the domain of language, mathematical skills (Raghubar et al., 2010), imagery (Baddeley, 1988), spatial navigation (Meneghetti et al., 2021), and decision-making (Hinson et al., 2003). WM model includes three main components: a phonological loop, a visuospatial sketchpad and the central executive with an episodic buffer (Baddeley, 2000, 2010). The Central Executive (CE) serves as an attentional control system that coordinates the activity of the short-term storage and allocates the attentional resource for processing tasks (Collette & Van Der Linden, 2002; Morris & Jones, 1990). Therefore, working memory is an extension of the concept of short-term memory (STM) by including the central executive as an attentional controller to storage of verbal and visual information (Cowan, 2008). This model has defined the agenda of neuroimaging studies targeting neural correlates of each single component and of their interaction.

Following this schema, several paradigms have been implemented to explore the impact of different behavioral parameters (Braver et al., 2008; Rottschy et al., 2012). A consistent body of literature focused on memory storage via Sternberg-like paradigms, delayed-matching-to-sample tasks and to a lesser extent the retro-cue task (Gazzaley & Nobre, 2012). These tasks allow disentangling storage from encoding and retrieval and thus found the application in neuroimaging studies of storage components (Sternberg, 1966; Vogel & Machizawa, 2004). Moreover, many studies utilized complex span tasks (reading span, operation span, etc.) to target the CE activation. These tasks require simultaneous storage and processing of both relevant and irrelevant information, thus utilizing the dual-task approach (Osaka et al., 2007; Wager & Smith, 2003). While eliciting the activation of CE, these

tasks were limited by the overlap of CE and storage in the same temporal window (Collette & Van Der Linden, 2002). The diversity of experimental approaches deepens our understanding of each separate component but hinders the integration of different findings into a holistic understanding of WM.

While characterizing different phases of WM processing, the above-mentioned experimental designs cannot purely disentangle the activity of the central executive component (CE). WM studies typically probe one domain, either verbal (sometimes sequential) or visual (object and spatial) (Emch et al., 2019; Luck & Vogel, 2013; Pavlov & Kotchoubey, 2022; Rottschy et al., 2012; Wager & Smith, 2003). Several studies investigating multimodal WM brain networks (Boran et al., 2021; Daume et al., 2017; Li et al., 2014; Perfetti et al., 2014; Xie et al., 2021) assume independent functioning of the phonological loop and the visuospatial sketchpad components (except (Li et al., 2014; Perfetti et al., 2014)). However, in Baddeley's model, all WM components work interactively and simultaneously, coding one complex stimulus into visual and verbal modality, and then integrating the multi-modal information into one complex representation (Logie et al., 2020). Thus, a considerable amount of literature has been dedicated to cross-modal interactions in WM (Allen et al., 2009; Izmalkova et al., 2022; Zhang et al., 2014). However, most of the approaches allow neither analyzing the complex components interplay nor the allocation of the specific neural oscillatory activity to each component. Without researching the neural basis of the WM components and their complex interplay, the architecture of CE remains ambiguous. For example, it is still unclear whether the CE is unified for all storage components (unitary WM) or whether each component has its own executive mechanism (two modality-specific WMs) (Stuss & Knight, 2013).

Thus, for a holistic understanding of the WM neural basis, we would need a cross-paradigm providing different levels of CE involvement for complex stimuli in both verbal and visual domains at once during storage. This paradigm would provide a clear distinction between short-term memory and goal-directed WM in order to isolate modality-specific components and distinguish between the neural activity of storage and CE, verifying the existence of unitary or separate executive mechanisms.

Thus, in this study we propose an experimental design to capture and compare different aspects of WM, comparing sensory modalities and recruitment of CE. The paradigm features two retention intervals (before and after the retro-cue): while the first retention corresponds with the storage of the entire item representing the interplay of WM components, the second retention aims to isolate the CE involvement during processing of the information. The originality of the study is premised on the application of the complex design with both verbal and visual WM modalities and different levels of attentional involvement in the same cohort of participants.

We selected a low demanding task and a high demanding operation: filtering (simple maintenance) and manipulation (complex processing) for spatial and verbal domains (representing visual and verbal WM modalities, respectively). In particular, in the low demanding task we asked participants to filter either spatial or

verbal information from a complex stimulus representation. In the high demanding task, we asked participants to manipulate spatial and verbal information by mental rotation or alphabetical reordering (for more details about theoretical assumptions under the paradigm see Supplementary materials: https://psy-journal.hse.ru/ data/2024/09/26/1882463452/Supplementary%20materials.pdf). We hypothesize that in the new comprehensive WM paradigm similar attentional involvement is required for simple maintenance of verbal and spatial information. Meanwhile, different attentional involvement reflecting different involvement of the CE characterizes unimodal conditions for simplex maintenance and complex processing. Finally, we hypothesize that the attentional involvement in the complex processing condition might be similar across modalities.

Methods

Participants

Thirty-two adult subjects participated in the behavioral study (mean age = 23.93, SD = 5.41, 20 female/12 male). Inclusion criteria were: age of 18-35 years old (Ferguson et al., 2021), absence of neurological or psychiatric disorders, and normal or corrected-to-normal vision. We calculated that the sample size of 25 participants would be sufficient to obtain 80% power of statistical analysis for the repeated measures analysis of variance (ANOVA) with the 0.25 estimated effect size and the .05 alpha level (G*power 3.1.9.7) (Faul et al., 2007).

The study was conducted in accordance with the Institutional and Ethical Review Board (IRB) guidelines of the Higher School of Economics. All participants signed a consent form.

Paradigm

The paradigm is based on a traditional retro-cue task, which allows disentangling different stages of informational processing in working memory: encoding, storage, manipulation and recall, which is a huge advantage for neuroimaging studies (Sternberg, 1966; Vogel & Machizawa, 2004). We modified the retro-cue task in order to involve both WM storage and the central executive.

In our paradigm, participants had to memorize a stimulus — a 3 by 3 matrix that was filled in with letters: 4 target letters and 5 noise letters. The cells, which contained target letters, were highlighted with a bright grey color (Figure 1). Hence, the matrix contained both the target (highlighted cells as a foreground) and noise (the grid as a background) visual information. In order to balance the noise-target perception between modalities, we added the letter "X" as verbal noise. The structure of the single trial is depicted in Figure 1. First, participants were presented with a fixation cross (0.5s). Next, a pro-cue appeared, signaling what participants should do with the stimulus. If the pro-cue was an "eye", participants should simply observe the matrix without a goal to memorize. This condition had the lowest demand on attention and thus it referred to a simple perception. Since there was no

behavioral outcome in this condition, it was not analyzed in this study; however, neural oscillation during perception condition can serve as a baseline for WM conditions in neuroimaging studies.

If the pro-cue was a "brain", participants should memorize both verbal and spatial information in the matrix for further processing (both letters and their locations). This condition engaged WM, because participants had a goal to memorize the stimulus for future manipulation. Participants were trained to read the matrix from left to right, from top to bottom. The pro-cue lasted 0.5s.

Next, the stimulus (matrix) was presented for 2s (Boran et al., 2019). Following the encoding phase, a first storage phase is characterized by a blank screen that was presented for 3s (Proskovec et al., 2019). Then a retro-cue was presented for 0.5s, indicating which type of processing should be performed with the stimulus. The paradigm is composed of four conditions: spatial or verbal information with filtering or manipulation.

The retro-cue "A" indicated that only verbal information should be retained (and spatial information should be filtered out). During retrieval phase following

this condition, the probe was a sequence of three dashes with a letter (- R--)

placed either in the correct place or not. The participant had to specify whether the letter in the probe was placed in the correct position as in the initial matrix. The retro-cue "A ^ H" indicated alphabetical reordering of the sequence (high CE involvement). During retrieval phase following this condition, the probe was a

sequence of three dashes with a letter (- R--) placed either in the correct place

or not regarding the alphabetical order.

The Scheme of the Experimental Paradigm

Figure 1

The retro-cue "map pin" indicated that spatial information should be retained (and verbal information should be filtered out). During retrieval phase following this condition, the probe was a matrix with one cell highlighted. The participant had to specify whether this cell was highlighted in the initial matrix. The retro-cue "arrow" indicated the matrix should be mentally rotated by 90 degrees clockwise (high CE involvement). During retrieval phase following this condition, the probe was a matrix with one cell highlighted. The participant had to specify whether this cell would be highlighted in the mentally rotated matrix.

Thus, the first and second conditions aimed to assess verbal information with a low or high CE involvement. The third and fourth conditions aimed to assess spatial information with a low or high CE involvement. The first and third conditions are referred to as simple conditions, the second and fourth - as complex conditions. The participant had 3s to filter or manipulate the information after the retro-cue. Then the probe appeared for 2s for simple conditions and 3s for a complex condition. Participants were required to answer within these time intervals, otherwise the trial was considered a miss. No feedback was presented after the response.

Participants answered "correct" or "incorrect" by pressing a key on the keyboard: Q (marked as blue) and P (marked as red). The answers were balanced across the participants with half of them answering "correct" by pressing the blue key (15), and half - by pressing the red key (17).

Both pro- and retro-cues were presented for 0.5s (Schneider et al., 2016). We aimed to have easily interpretable cues for the task. Thus, we employed visual cues (pictograms) to indicate the condition types. Detailed information about the cue selection can be found in the Supplementary materials.

For the verbal domain, we selected Cyrillic letters of the equal or approximately equal visual and acoustic complexity, which did not have similarity with other symbols. The final sample contained 10 letters (THKM^nrafflO). Detailed information about the letter selection can be found in the Supplementary materials.

In total the participants had five conditions with 60 trials each. All conditions were randomly mixed in such a way that there could not be more than three trials of the same condition, and not more than five trials of the same domain in a row. The duration of the perception condition was 6s, the durations of the WM conditions were 11.5s for the simple conditions and 12.5s for the complex conditions. In total, the experiment lasted 60 minutes. We divided the experiment into five blocks each separated by two-minute breaks.

Procedure

Participants performed the experiment in a soundproof room. They were seated at approximately 50 cm from a 27-inches electron monitor (1920 X 1080 resolution, 144 Hz refreshing rate). The experimental paradigm was run on Psychopy (v2022.2.5) in OS Windows.

Before the experiment, participants performed a training session. The first five practice trials were conducted without a time limit with verbal prompts from the experimenter. In the next five practice trials participants performed on their own

with a slow pace. Then participants had 20 practice trials on their own at a real experiment pace.

Statistical methods

Before data analysis we applied two criteria to consider guesses and the chance level performance. Firstly, we deleted answers faster than 400ms (Llorens et al., 2023). Secondly, we aimed to keep at least 30 clear trials for the further neuroimag-ing task (Chaumon et al., 2021; Cohen, 2014). Considering the average amount of rejected trials to be 25% (Otstavnov et al., 2024), we calculated that the 70% threshold is enough to obtain such a number.

Our primary goal was to investigate differences in accuracy and reaction time among experimental conditions; therefore, we used a within-group design. To mitigate the non-normal distribution of the data (see Supplementary material for more details), we applied a Box-Cox transformation (Osborne, 2010). We tested differences in the transformed accuracy and median response time by the repeated measures ANOVA with the factors of Modality (visual and verbal) and Load (simple and complex). Post hoc pairwise comparisons were conducted by the two-sided t-test with a Sidak correction to adjust for multiple comparisons (Howell, 2013).

The data analysis was conducted with custom scripts in Python with the pingouin package (v0.5.4). Visualization of the results was performed by matplotlib (v3.7.0) and seaborn packages (v0.12.2).

Data availability

All data used to design a new comprehensive WM paradigm is available at https://github.com/meggrouphse/working-memory-project including psychopy file, python scripts for stimulus creation, stimulus descriptive statistics and data analysis pipeline.

Results

Accuracy

Repeated measures ANOVA (N = 32, RT > 400ms) revealed significant differences in accuracy across Modalities (F(1, 31) = 8.87, pGGcorrected < .01) and Load (F(1, 31) = 57.22, pGGcorrected < .01). The interaction of factors was also significant (F(1, 31) = 4.65, pGGcorrected = .04). Post-hoc comparison revealed no differences between simple visual and simple verbal conditions (T(31) = —0.72, pcorr = .98, common language effect size (CLES) = .47). Significant differences were observed between simple verbal and complex verbal condition (T(31) = —7.50, pcorr < 0.01, CLES = .14); simple and complex visual conditions (T(31) = —4.60, pcorr < 0.01, CLES = .25); complex visual and complex verbal conditions (T(31) = —3.606, p = 0.02, CLES = .32). The results are visually represented in Figure 2. The accuracy means and standard deviations for each condition are presented in Table 1.

Figure 2

Accuracy across Conditions Expressed as Percentage of Correct Responses

Table 1

Means and Standard Deviations of Accuracy (in Percentage) for Each Condition for General,

Male and Female Samples

Measure Simple Visual Simple Verbal

General Male Female General Male Female

Accuracy Mean 92.86 91.53 93.67 92.14 92.36 92.00

Accuracy Standard deviation 5.19 5.53 4.94 5.43 5.88 5.29

Complex Visual Complex Verbal

Accuracy Mean 86.40 87.78 85.58 81.25 78.61 82.83

Accuracy Standard deviation 7.82 7.86 7.88 7.90 8.61 7.20

Median response time

Repeated measures ANOVA (N = 32, RT > 400ms) revealed significant differences in median response time across Modalities (F(1, 31) = 41.38, PGGcorrected < .01) and Load (F(1, 31) = 143.13, pGGcorrected < .01). The interaction of Modality and Load was significant (F(1, 31) = 5.58,PGGcorrected = .03). Post-hoc comparison revealed differences between simple visual and simple verbal (T(31) = 4.67,p < .01, CLES = .65), simple verbal and complex verbal (T(31) = 10.63, p < .01, CLES = .82), simple visual and complex visual (T(31) = 7.09, p < 0.01, CLES = .72) and complex visual and complex verbal conditions (T(31) = 5.67, p < .01, CLES = .75). The results are

visually represented in Figure 3. The response time median and standard deviation for each condition are presented in Table 2.

Discussion

Our findings show that the proposed experimental paradigm is suitable for further neuroimaging studies. There are no significant differences between simple conditions of different modalities (condition balance), which indicates relatively similar attentional involvement for filtering operation and supports the first hypothesis. Importantly, significant differences were found between conditions of

Figure 3

Median Response Time Expressed in Seconds across Conditions

Complex verbal

Table 2

Means and Standard Deviations of response Time (in Seconds) for Each Condition for General,

Male and Female Samples

Measure Simple Visual Simple Verbal

General Male Female General Male Female

Response time median 0.88 0.84 0.89 0.97 0.97 0.97

Response time Standard deviation 0.15 0.16 0.15 0.17 0.24 0.12

Complex Visual Complex Verbal

Response time median 1.01 0.98 1.03 1.22 1.14 1.23

Response time Standard deviation 0.27 0.38 0.18 0.24 0.35 0.15

«L 1.6

a>

E

Response time across conditions

.V

Simple visual

p < 0.01.

• V

•

Simple verbal Complex visual

Experimental conditions

the same modality mirroring different levels of involvement of the CE (between filtering and manipulation) along with the second hypothesis.

The paradigm provides the opportunity to verify, in a single experiment, the neuropsychological difference between WM storage and involvement of the CE for each sensory modality and the presence of a unified or modality specific CE (Stuss & Knight, 2013). The retention during the first storage reveals the complex interplay of WM components, as the stimulus is retained in its entirety. The cue between first and second storage guides participants into prioritizing the information, splitting it into task-relevant and task-irrelevant (De Vries et al., 2020). The filtering and manipulation occurring during the second storage in the simple and complex conditions, respectively, involve CE at different levels (Ecker et al., 2014). Therefore, the distinct contribution of the CE in a specific sensory modality can be obtained by contrasting simple vs complex conditions in the second storage (Figure 4). The further comparison of the obtained activity between modalities may allow answering the question whether CE is unified or modality-specific (Stuss & Knight, 2013). Therefore, we believe that implementation of this paradigm in the EEG/MEG settings might elucidate critical aspects of the WM architecture.

However, it is important to address difference between complex conditions, at least through the qualitative comparison of neurophysiological patterns. Firstly, we assumed that CE is engaged in the same way for complex processing of the verbal

Figure 4

Paradigm Contrasts in the Neuroimaging Studies

Note. Grey boxes indicate the activity during the first retention interval. Black boxes indicate the activity during the second retention interval. Black lines indicate contrasts of neural activity with the expected outcomes.

and spatial domain. However, alphabetical reordering of the verbal information might require the additional involvement of an episodic buffer as the alphabet is stored in the long-term memory (Artuso & Palladino, 2016). Thus, while the complex visual condition might induce the activation of modality-specific storage and CE, the complex verbal condition additionally engages the episodic buffer (Baddeley, 2003; Nobre et al., 2013). Therefore, the manipulation of the verbal (alphabetization) and spatial (mental rotation) information cannot be directly compared in terms of attentional involvement. Neuroimaging studies point to the involvement of different brain areas across modalities and confirm this assumption. The FRMI targeting stimulus alphabetization showed an increased activity in the dorsolateral prefrontal cortex (Postle et al., 1999), the right anterior prefrontal cortex and the left superior frontal area (Collette et al., 1999), while the mental rotation task revealed a higher activity in the left premotor and left primary motor cortex (Wraga et al., 2005). Moreover, alphabetization is associated with the alpha power suppression and midline theta increase (Pavlov & Kotchoubey, 2017), while mental rotation elicited the enhancement of alpha and low beta oscillatory power over the parietal area, with subsequent spreading to the frontal area (Riecansky & Katina, 2010). Thus, the neural activity patterns for verbal manipulation possibly involve additional WM components, like the episodic buffer.

Secondly, this type of experimental design has been previously implemented to compare the storage of sequential and spatial information in WM, which are conceptually close to the retention of the letter order and location in a particular study. Such domains induce specific brain activation, modulate oscillatory power in theta, alpha and beta frequency ranges (Roberts et al., 2013), and rely on different network profiles (Otstavnov et al., 2024). The theoretical background of this domain specificity of WM was proposed by E. Abrahamse, S. Majerus and W. Fias (2014) who formulated the mental whiteboard hypothesis implying that the retention of sequential information requires constant translation into a spatial code, which is an additional step for WM (Abrahamse et al., 2014). Thus, storage and processing of letter sequence require additional activity because of information recoding and result in a higher response time for both simple and complex conditions. This effect becomes obvious in the complex verbal condition. We believe that these scenarios might be validated by further neuroimaging studies providing the comparison of neurophysiological activity under different conditions.

This study also presents some limitations. Firstly, the sample size of the behavioral study was limited by 32 participants, which met our power analysis criteria, but does not allow a deep stratification of the population (i.e. male vs female, see Supplementary material for more details). Secondly, we observed an unbalance between the complex visual and verbal conditions in the analyzed cohort. While this requires further investigation in the potential sources of variability in the population, a refined tuning of the threshold might mitigate this difference, allowing for the direct comparison of CE involvement across modalities. Thirdly, the assessment of visual modality was limited by a spatial information, which is only one aspect of the visuo-spatial sketchpad (Jonides et al., 1993). Further studies can address object aspects of the visuo-spatial sketchpad by changing letter stimuli to

shapes or objects. Finally, the rearrangement of the verbal sequence might be contributed by spatial WM, weakening the contrast between complex conditions. However, alphabetization seems to rely mostly on the retention mediated by inner speech (phonological loop) (Marvel & Desmond, 2012). This should be further addressed in neuroimaging studies.

In conclusion, we developed an experimental paradigm for WM components investigation, which can be implemented in future neuroimaging research. The paradigm supports the comparison between different levels of CE engagement within a single domain and across modalities, potentially offering an opportunity to investigate the unitary or modality-specific nature of CE at the neurophysiolog-ical level. Thus, the paradigm validated in this study at a behavioral level is already structured to expand our knowledge on the neural organization of the multicom-ponent nature of working memory in future neuroimaging studies.

References

Abrahamse, E., Van Dijck, J.-P., Majerus, S., & Fias, W. (2014). Finding the answer in space: The mental whiteboard hypothesis on serial order in working memory. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00932 Allen, R. J., Hitch, G. J., & Baddeley, A. D. (2009). Cross-modal binding and working memory. Visual

Cognition, 17(1-2), 83-102. https://doi.org/10.1080/13506280802281386 Artuso, C., & Palladino, P. (2016). Modulation of working memory updating: Does long-term memory lexical

association matter? Cognitive Processing, 17(1), 49-57. https://doi.org/10.1007/s10339-015-0735-4 Baddeley, A. D. (1988). Imagery and working memory. In M. Denis, J. Engelkamp, & J. T. E. Richardson (Eds.), Cognitive and neuropsychological approaches to mental imagery (pp. 169-180). Springer Netherlands. https://doi.org/10.1007/978-94-009-1391-2_15 Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive

Sciences, 4(11), 417-423. https://doi.org/10.1016/S1364-6613(00)01538-2 Baddeley, A. (2003). Working memory and language: An overview. Journal of Communication

Disorders, 36(3), 189-208. https://doi.org/10.1016/S0021-9924(03)00019-4 Baddeley, A. (2010). Working memory. Current Biology, 20(4), R136-R140.

https://doi.org/10.1016/jxub.2009.12.014 Boran, E., Fedele, T., Klaver, P., Hilfiker, P., Stieglitz, L., Grunwald, T., & Sarnthein, J. (2019). Persistent hippocampal neural firing and hippocampal-cortical coupling predict verbal working memory load. Science Advances, 5(3). https://doi.org/10.1126/sciadv.aav3687 Boran, E., Stieglitz, L., & Sarnthein, J. (2021). Epileptic high-frequency oscillations in intracranial EEG are not confounded by cognitive tasks. Frontiers in Human Neuroscience, 15, Article 613125. https://doi.org/10.3389/fnhum.2021.613125 Brady, T. F., Robinson, M. M., & Williams, J. R. (2024). Noisy and hierarchical visual memory across timescales. Nature Reviews Psychology, 3, 147-163. https://doi.org/10.1038/s44159-024-00276-2 Braver, T. S., Gray, J. R., & Burgess, G. C. (2008). Explaining the many varieties of working memory variation: Dual mechanisms of cognitive control. In A. Conway, C. Jarrold, M. Kane, A. Miyake, & J. Towse (Eds.), Variation in working memory (pp. 76-106). New York, NY: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195168648.003.0004

Chaumon, M., Puce, A., & George, N. (2021). Statistical power: Implications for planning MEG studies. NeuroImage, 233, Article 117894. https://doi.org/10.1016/j.neuroimage.2021.117894 Cohen, M. X. (2014). Analyzing neural time series data: Theory and practice. The MIT Press.

https://doi.org/10.7551/mitpress/9609.001.0001 Collette, F., Salmon, E., Van Der Linden, M., Chicherio, C., Belleville, S., Degueldre, C., Delfiore, G., & Franck, G. (1999). Regional brain activity during tasks devoted to the central executive of working memory. Cognitive Brain Research, 7(3), 411-417. https://doi.org/10.1016/S0926-6410(98)00045-7 Collette, F., & Van Der Linden, M. (2002). Brain imaging of the central executive component of working memory. Neuroscience & Biobehavioral Reviews, 26(2), 105-125. https://doi.org/10.1016/S0149-7634(01)00063-X

Cowan, N. (2008). What are the differences between long-term, short-term, and working memory? In Progress in Brain Research (Vol. 169, pp. 323-338). Elsevier. https://doi.org/10.1016/S0079-6123(07)00020-9

Daume, J., Graetz, S., Gruber, T., Engel, A. K., & Friese, U. (2017). Cognitive control during audiovisual working memory engages frontotemporal theta-band interactions. Scientific Reports, 7(1), Article 12585. https://doi.org/10.1038/s41598-017-12511-3 De Vries, I. E. J., Slagter, H. A., & Olivers, C. N. L. (2020). Oscillatory control over representational states in working memory. Trends in Cognitive Sciences, 24(2), 150-162. https://doi.org/10.1016Zj.tics.2019.11.006 Ecker, U. K. H., Lewandowsky, S., & Oberauer, K. (2014). Removal of information from working memory: A specific updating process. Journal of Memory and Language, 74, 77-90. https://doi.org/10.1016/j.jml.2013.09.003 Emch, M., von Bastian, C. C., & Koch, K. (2019). Neural correlates of verbal working memory: An fMRI metaanalysis. Frontiers in Human Neuroscience, 13, Article 180. https://doi.org/10.3389/fnhum.2019.00180 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146 Ferguson, H. J., Brunsdon, V. E. A., & Bradford, E. E. F. (2021). The developmental trajectories of executive function from adolescence to old age. Scientific Reports, 11(1), Article 1382. https://doi.org/10.1038/s41598-020-80866-1 Gazzaley, A., & Nobre, A. C. (2012). Top-down modulation: Bridging selective attention and working memory. Trends in Cognitive Sciences, 16(2), 129-135. https://doi.org/10.1016/j.tics.2011.11.014 Hinson, J. M., Jameson, T. L., & Whitney, P. (2003). Impulsive decision making and working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(2), 298-306. https://doi.org/10.1037/0278-7393.29.2.298 Howell, D. C. (2013). Statistical methods for psychology (8th ed.). Belmont, CA: Wadsworth Cengage Learning.

Izmalkova, A., Barmin, A., Velichkovsky, B. B., Prutko, G., & Chistyakov, I. (2022). Cognitive resources in working memory: domain-Specific or general? Behavioral Sciences, 12(11), Article 459. https://doi.org/10.3390/bs12110459 Jonides, J., Smith, E. E., Koeppe, R. A., Awh, E., Minoshima, S., & Mintun, M. A. (1993). Spatial working memory in humans as revealed by PET. Nature, 363(6430), 623-625. https://doi.org/10.1038/363623a0 Li, D., Christ, S. E., & Cowan, N. (2014). Domain-general and domain-specific functional networks in working memory. NeuroImage, 102, 646-656. https://doi.org/10.1016/j.neuroimage.2014.08.028 Llorens, A., Bellier, L., Blenkmann, A. O., Ivanovic, J., Larsson, P. G., Lin, J. J., Endestad, T., Solbakk, A.-K., & Knight, R. T. (2023). Decision and response monitoring during working memory are

sequentially represented in the human insula. iScience, 26(10), Article 107653. https://doi.org/10.1016/jisci.2023.107653 Logie, R., Camos, V., & Cowan, N. (Eds.). (2020). Working memory: The state of the science. Oxford

University Press. https://doi.org/10.1093/oso/9780198842286.001.0001 Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17(8), 391-400. https://doi.org/10.1016Xj.tics.2013.06.006 Marvel, C. L., & Desmond, J. E. (2012). From storage to manipulation: How the neural correlates of verbal working memory reflect varying demands on inner speech. Brain and Language, 120(1), 42-51. https://doi.org/10.1016Zj.bandl.2011.08.005 Meneghetti, C., Labate, E., Toffalini, E., & Pazzaglia, F. (2021). Successful navigation: The influence of task goals and working memory. Psychological Research, 85(2), 634-648. https://doi.org/10.1007/s00426-019-01270-7 Morris, N., & Jones, D. M. (1990). Memory updating in working memory: The role of the central executive.

BritishJournal of Psychology, 81(2), 111-121. https://doi.org/10.1111/j.2044-8295.1990.tb02349.x Nobre, A. D. P., Rodrigues, J. D. C., Sbicigo, J. B., Piccolo, L. D. R., Zortea, M., Junior, S. D., & De Salles, J. F. (2013). Tasks for assessment of the episodic buffer: A systematic review. Psychology & Neuroscience, 6(3), 331-343. https://doi.org/10.3922/j.psns.2013.3.10 Osaka, N., Logie, R. H., & D'Esposito, M. (Eds.). (2007). The cognitive neuroscience of working memory.

Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198570394.001.0001 Osborne, J. (2010). Improving your data transformations: Applying the Box-Cox transformation.

Practical Assessment, Research, and Evaluation, 15(1). https://doi.org/10.7275/QBPC-GK17 Otstavnov, N., Riaz, A., Moiseeva, V., & Fedele, T. (2024). Temporal and spatial information elicit different power and connectivity profiles during working memory maintenance. Journal of Cognitive Neuroscience, 36(2), 290-302. https://doi.org/10.1162/jocn_a_02089 Pavlov, Y. G., & Kotchoubey, B. (2017). EEG correlates of working memory performance in females.

BMC Neuroscience, 18(1), Article 26. https://doi.org/10.1186/s12868-017-0344-5 Pavlov, Y. G., & Kotchoubey, B. (2022). Oscillatory brain activity and maintenance of verbal and visual working memory: A systematic review. Psychophysiology, 59(5), Article e13735. https://doi.org/10.1111/psyp.13735 Perfetti, B., Varanese, S., Mancino, E., Mercuri, P., Tesse, M., Franciotti, R., Bonanni, L., Thomas, A., & Onofrj, M. (2014). Electrophysiological indices of interference resolution covary with individual fluid intelligence: Investigating reactive control processes in a 3-back working memory task. Neurolmage, 93, 146-153. https://doi.org/10.1016/j.neuroimage.2014.02.020 Postle, B. R., Berger, J. S., & D'Esposito, M. (1999). Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. PNAS, 96(22), 12959-12964. https://doi.org/10.1073/pnas.96.22.12959 Proskovec, A. L., Heinrichs-Graham, E., & Wilson, T. W. (2019). Load modulates the alpha and beta oscillatory dynamics serving verbal working memory. Neurolmage, 184, 256-265. https://doi.org/10.1016/j.neuroimage.2018.09.022 Raghubar, K. P., Barnes, M. A., & Hecht, S. A. (2010). Working memory and mathematics: A review of developmental, individual difference, and cognitive approaches. Learning and Individual Differences, 20(2), 110-122. https://doi.org/10.1016/j.lindif.2009.10.005

Riecansky, I., & Katina, S. (2010). Induced EEG alpha oscillations are related to mental rotation ability: The evidence for neural efficiency and serial processing. Neuroscience Letters, 482(2), 133-136. https://doi.org/10.1016/j.neulet.2010.07.017 Roberts, B. M., Hsieh, L.-T., & Ranganath, C. (2013). Oscillatory activity during maintenance of spatial and temporal information in working memory. Neuropsychologia, 51(2), 349-357. https://doi.org/10.1016Zj.neuropsychologia.2012.10.009 Rottschy, C., Langner, R., Dogan, I., Reetz, K., Laird, A. R., Schulz, J. B., Fox, P. T., & Eickhoff, S. B. (2012). Modelling neural correlates of working memory: A coordinate-based meta-analysis. NeuroImage, 60(1), 830-846. https://doi.org/10.1016/j.neuroimage.2011.11.050 Schneider, D., Mertes, C., & Wascher, E. (2016). The time course of visuo-spatial working memory updating revealed by a retro-cuing paradigm. Scientific Reports, 6(1), Article 21442. https://doi.org/10.1038/srep21442 Sternberg, S. (1966). High-speed scanning in human memory. Science, 153(3736), 652-654.

https://doi.org/10.1126/science.153.3736.652 Stuss, D. T., & Knight, R. T. (Eds.). (2013). Principles of frontal lobe function. Oxford University Press.

https://doi.org/10.1093/med/9780199837755.001.0001 Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in visual

working memory capacity. Nature, 428(6984), 748-751. https://doi.org/10.1038/nature02447 Wager, T. D., & Smith, E. E. (2003). Neuroimaging studies of working memory: Cognitive, Affective,

& Behavioral Neuroscience, 3(4), 255-274. https://doi.org/10.3758/CABN.3.4.255 Wraga, M., Shephard, J. M., Church, J. A., Inati, S., & Kosslyn, S. M. (2005). Imagined rotations of self versus objects: An fMRI study. Neuropsychologia, 43(9), 1351-1361. https://doi.org/10.1016/j.neuropsychologia.2004.11.028 Xie, Y., Li, Y., Duan, H., Xu, X., Zhang, W., & Fang, P. (2021). Theta oscillations and source connectivity during complex audiovisual object encoding in working memory. Frontiers in Human Neuroscience, 15, Article 614950. https://doi.org/10.3389/fnhum.2021.614950 Zhang, Y., Hu, Y., Guan, S., Hong, X., Wang, Z., & Li, X. (2014). Neural substrate of initiation of cross-modal working memory retrieval. PLoS ONE, 9(8), Article e103991. https://doi.org/10.1371/journal.pone.0103991