Pipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics... Show morePipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics Processing Units (GPUs). However, with the sustaining increase in Deep Learning (DL) model sizes, pipeline parallelism provides only a partial solution to the memory bottleneck in large-scale DNN training. Careful partitioning of the DL model over the available GPUs based on memory usage is required to further alleviate the memory bottleneck and train larger DNNs. mCAP is such a memory-oriented partitioning approach for pipeline parallel systems, but it does not scale to models with many layers and very large hardware setups, as it requires extensive profiling and fails to efficiently navigate the partitioning space to find the most memory-friendly partitioning. In this work, we propose CAPSlog, a scalable memory-centric partitioning approach that can recommend model partitionings for larger and more heterogeneous DL models and for larger hardware setups than existing approaches. CAPSlog introduces a new profiling method and a new, much more scalable algorithm for recommending memory-efficient partitionings. CAPSlog reduces the profiling time by 67% compared to existing approaches, searches the partitioning space for the optimal solution orders of magnitude faster and can train significantly larger models. Show less
What we collectively call “stress” is how we experience our body’s reaction to a stressor. This response is aimed to deal with the current stressor and to prepare for recurrences in the future. The... Show moreWhat we collectively call “stress” is how we experience our body’s reaction to a stressor. This response is aimed to deal with the current stressor and to prepare for recurrences in the future. The stress response is for an important part dependent on glucocorticoid hormones. By and large, the acute response to glucocorticoids is beneficial, but chronic exposure often becomes maladaptive. To improve prevention and treatment of disorders we can develop due to stress, it is important to better understand the effects and working mechanisms of glucocorticoids. While we already possess extensive knowledge regarding glucocorticoids and glucocorticoid receptor signaling, we introduced and studied five “aspects of context”, which we felt address important current misconceptions or gaps of knowledge. Corticosterone was at the center of all the studies we performed, yet the eventual outcome of glucocorticoid receptor activation differed extensively in all experiments. Thus, the context in which corticosterone exerts its effects matters, and it is to researcher to be aware of this when designing new studies and interpreting available data. Whilst our research merely addressed some specific processes, the lessons learned from these experiments can be applied much broader to the biology of glucocorticoid signaling and other nuclear family members. Show less
Dreuning, H.; Bal, H.E.; Nieuwpoort, R.V. van 2023
Deep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models... Show moreDeep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models contain billions of trainable parameters. Training such massive neural networks incurs significant memory requirements and financial cost. Hybrid-parallel training approaches have emerged that combine pipelining with data and tensor parallelism to facilitate the training of large DL models on distributed hardware setups. However, existing approaches to design a hybrid-parallel partitioning and parallelization plan for DL models focus on achieving high throughput and not on minimizing memory usage and financial cost. We introduce CAPTURE, a partitioning and parallelization approach for hybrid parallelism that minimizes peak memory usage. CAPTURE combines a profiling-based approach with statistical modeling to recommend a partitioning and parallelization plan that minimizes the peak memory usage across all the Graphics Processing Units (GPUs) in the hardware setup. Our results show a reduction in memory usage of up to 43.9% compared to partitioners in state-of-the-art hybridparallel training systems. The reduced memory footprint enables the training of larger DL models on the same hardware resources and training with larger batch sizes. CAPTURE can also train a given model on a smaller hardware setup than other approaches, reducing the financial cost of training massive DL models. Show less
To comprehend texts readers build mental representations. To establish coherence and protect these representations against inaccuracies readers routinely monitor and validate textual information... Show moreTo comprehend texts readers build mental representations. To establish coherence and protect these representations against inaccuracies readers routinely monitor and validate textual information against two main informational sources –what they just read (the text itself) and what they know (their background knowledge). This dissertation focuses on validation processes in the context of reading comprehension. Texts today vary in accuracy and trustworthiness. To better understand how readers validate (written) materials against various sources of information, the experimental studies in this dissertation employed different research methods to examine the (neuro)cognitive architecture of the processes involved in validating against prior text (i.e., text-based validation) and validating against background knowledge (i.e., knowledge-based validation) and how these processes affect readers;’ memory for text information.Results illustrate that readers validate incoming information against these two sources in dissociable, (partially) interactive, text-based and knowledge-based validation processes. Moreover, these processes seem to protect readers’ memory against inaccuracies or incongruencies. These observations deepen our understanding of validation processes, provide starting points for investigations of people’s susceptibility to false information and how inaccurate knowledge can be revised and provide insight into the complex interplay between recently acquired knowledge from the text itself and background knowledge in constructing meaning from language. Show less
Gela, A.; Murphy, M.; Rodo, M.; Hadley, K.; Hanekom, W.A.; Boom, W.H.; ... ; Delayed BCG Study Team 2022
Background: Non-protein antigen classes can be presented to T cells by near-monomorphic antigen-presenting molecules such as CDI, MRI, and butyrophilin 3AI. Such T cells, referred to as donor... Show moreBackground: Non-protein antigen classes can be presented to T cells by near-monomorphic antigen-presenting molecules such as CDI, MRI, and butyrophilin 3AI. Such T cells, referred to as donor unrestricted T (DURT) cells, typically express stereotypic T cell receptors. The near-unrestricted nature of DURT cell antigen recognition is of particular interest for vaccine development, and we sought to define the roles of DURT cells, including MRI-restricted MAIT cells, CDth-restricted glucose monomycolate (GMM)-specific T cells, CDth-restricted NKT cells, and gamma delta T cells, in vaccination against Mycobacterium tuberculosis. Methods: We compared and characterized DURT cells following primary bacille Calmette-Guerin (BCG) vaccination in a cohort of vaccinated and unvaccinated infants, as well as before and after BCG-revaccination in adults. Findings: BCG (re)vaccination did not modulate peripheral blood frequencies, T cell activation or memory profiles of MAIT cells, CDth-restricted GMM-specific and germline-encoded mycolyl-reactive (GEM) cells or CDId-restricted NKT cells. By contrast, primary BCG vaccination was associated with increased frequencies of gamma delta T cells as well as a novel subset of CD26(+)CDI6I(+)TRAVI-2(-) IFN-gamma-expressing CD4(+) T cells in infants. Interpretation: Our findings, that most DURT cell populations were not modulated by BCG, do not preclude a role of BCG in modulating other qualitative aspects of DURT cells. More studies are required to understand the full potential of DURT cells in new TB vaccine strategies. Copyright (C) 2022 The Authors. Published by Elsevier B.V. Show less
Ancient rhetoricians such as Cicero and Quintilian advised orators to apply memoria strategies to memorize their speech. Modern public-speaking textbooks recommend various rhetorical techniques to... Show moreAncient rhetoricians such as Cicero and Quintilian advised orators to apply memoria strategies to memorize their speech. Modern public-speaking textbooks recommend various rhetorical techniques to make messages memorable for audiences. However, the effect of these techniques on information retention has been sparsely studied.This dissertation investigates how rhetorical techniques in speeches can enhance information retention by the audience. Using a three-way approach, it connects rhetorical theory, advice and practice to insights from memory psychology.First, an analysis of influential English-language and Dutch-language public-speaking textbooks showed that frequently advised retention techniques are related to organization, elaboration and visualization—three encoding principles that contribute to information storage. Authors prefer the conclusion of a speech for influencing retention.Secondly, the study describes how scholars, politicians and TED speakers use organization and elaboration techniques (e.g. summaries and anecdotes) in public-speaking practice. Results indicate discrepancies with textbook advice and context-dependent technique variants.Thirdly, two experiments were performed that focused on three organization techniques applied in conclusions of informative presentations. Results show that announcing the conclusion and providing an informative summary can enhance information retention by the audience. We are one step closer to knowing what makes messages memorable. Show less
This dissertation examines how the Serbs as a nation deal with the past through the prism of the 1915 Serbian Army retreat across Montenegro and Albania in the wake of the invasion of Serbia in... Show moreThis dissertation examines how the Serbs as a nation deal with the past through the prism of the 1915 Serbian Army retreat across Montenegro and Albania in the wake of the invasion of Serbia in October 1915. We investigate the remembrance of this iconic event as a symptom of Serbian mythologized self-perception. The Retreat today is a canonised memory symbolising Serbian heroism and sacrifice. The First World War and the Retreat are experienced as specific and personal. Any attempted introduction of alternative views of the Serbian role in the First World War are rejected as hostile revisionism. The Serbian government has used the centenary commemorations to distract the Serbian public from the more recent unexamined past — Serbia’s role in the wars of the Yugoslav succession of the 1990s. This commemorative opportunism has been termed in the thesis as historical frame switching —framing current political events within arguably irrelevant historical contexts mostly in order to present an image of biased and unjust treatment of Serbs by the West. Further, politically loaded commemorations are analysed alongside grassroot events and ceremonies that memorialise the War and the Retreat. The latter are recognised as fictive kinships of remembrance, as previously characterised by Jay Winter. Show less
Background: The cholinergic system and M-1 receptor remain an important target for symptomatic treatment of cognitive dysfunction. The selective M-1 receptor partial agonist HTL0018318 is under... Show moreBackground: The cholinergic system and M-1 receptor remain an important target for symptomatic treatment of cognitive dysfunction. The selective M-1 receptor partial agonist HTL0018318 is under development for the symptomatic treatment of Dementia's including Alzheimer's disease (AD) and dementia with Lewy bodies (DLB). We investigated the safety, tolerability, pharmacokinetics and exploratory pharmacodynamics of multiple doses of HTL0018318 in healthy younger adults and elderly subjects.Methods: This randomised, double blind, placebo-controlled study was performed, investigating oral doses of 15-35 mg/day HTL0018318 or placebo in 7 cohorts of healthy younger adult (n = 36; 3 cohorts) and elderly (n = 50; 4 cohorts) subjects. Safety, tolerability and pharmacokinetic measurements were performed. Pharmacodynamics were assessed using a battery of neurocognitive tasks and electrophysiological biomarkers of synaptic and cognitive functions.Results: HTL0018318 was generally well-tolerated in multiple doses up to 35 mg/day and were associated with mild or moderate cholinergic adverse events. There were modest increases in blood pressure and pulse rate when compared to placebo-treated subjects, with tendency for the blood pressure increase to attenuate with repeated dosing. There were no clinically significant observations or changes in blood and urine laboratory measures of safety or abnormalities in the ECGs and 24-h Holter assessments. HTL0018318 plasma exposure was dose-proportional over the range 15-35 mg. Maximum plasma concentrations were achieved after 1-2 h. The apparent terminal half-life of HTL0018318 was 16.1 h (+/- 4.61) in younger adult subjects and 14.3 h (+/- 2.78) in elderly subjects at steady state. HTL0018318 over the 10 days of treatment had significant effects on tests of short-term (working) memory (n-back) and learning (Milner maze) with moderate to large effect sizes.Conclusion: Multiple doses of HTL0018138 showed well-characterised pharmacokinetics and were safe and generally well-tolerated in the dose range studied. Pro-cognitive effects on short-term memory and learning were demonstrated across the dose range. These data provide encouraging data in support of the development of HTL0018138 for cognitive dysfunction in AD and DLB. Show less
Context-dependency effects on memory exist, whereby people’s context influences their ability to accurately recall items from memory. This effect was not previously studied when considering virtual... Show moreContext-dependency effects on memory exist, whereby people’s context influences their ability to accurately recall items from memory. This effect was not previously studied when considering virtual reality as an environmental context. We show that adverse effects on recall of memorized items exist when changing between virtual and real environments. The effect was not present when memorizing and recall were both done in VR; it appears to be caused by the change of environmental context. This previously unknown effect may impact how we use VR for memorization tasks, particularly when accurate recall of memorized information in a real environment is important. In a memory-recall experiment (n = 51) participants that underwent a context change involving VR after memorizing performed significantly worse on 24-h later item recall than those who did not change context (17% lower accuracy, p < 0.001). In particular memorizing in VR as opposed to a real environment lowers accuracy of recall in a real environment (24% lower, p = 0.001). Show less
This dissertation investigates memorisation strategies that were employed in the fields of painting and calligraphy in imperial China, with a focus on the Ming (1368-1644) and Qing (1644-1911)... Show moreThis dissertation investigates memorisation strategies that were employed in the fields of painting and calligraphy in imperial China, with a focus on the Ming (1368-1644) and Qing (1644-1911) dynasties. Its core questions are: How do memory aids reflect the society that produced them? What role did they play in the transmission and codification of practical knowledge in the field of arts?With the expansion of the printing industry during the Ming dynasty, knowledge of artistic practices came to be valued not only by craftsmen, but also by editors, publishers and highly educated authors. By analyzing both publication context and practices of remembering recorded in works from the Ming and Qing, this study provides insight into the dynamic changes of social values attributed to crafts. It takes a socio-historical approach to analyse memory aids in textual and visual formats recorded in manuals, including formulae (jue 訣) and charts. It provides six case studies to discuss under which circumstances memory aids were composed and how they were received over time, laying a foundation for understanding how practical skills were taught and how new canons of artistic knowledge were constantly being negotiated. Show less
This multidisciplinary volume brings together scholars and writers who try to come to terms with the histories and legacies of European slavery in the Indian Ocean. The volume discusses a variety... Show moreThis multidisciplinary volume brings together scholars and writers who try to come to terms with the histories and legacies of European slavery in the Indian Ocean. The volume discusses a variety of qualitative data on the experience of being a slave in order to recover ordinary lives and, crucially, to place this experience in its Asian local context. Building on the rich scholarship on the slave trade, this volume offers a unique perspective that embraces the origin and afterlife of enslavement as well as the imaginaries and representations of slaves rather than the trade in slaves itself. Show less
Rostamian, S.; Haan, S. de; Grond, J. van der; Buchem, M.A. van; Ford, I.; Jukema, J.W.; Sabayan, B. 2019
Exploring novel environments enhances learning in animals. Due to differing traditions, research into the effects of spatial novelty on learning in humans is scarce. Recent developments of... Show moreExploring novel environments enhances learning in animals. Due to differing traditions, research into the effects of spatial novelty on learning in humans is scarce. Recent developments of affordable and fMRI-compatible virtual reality (VR) and mobile EEG systems can help bridge the gap between the two literatures. One promising study showed that spatial novelty also promotes learning in humans. It still remains largely unknown, however, which aspect of novelty underlies the beneficial effect on memory, as novelty, expectations, and volition are often confounded in animal studies. In humans, these factors can be experimentally manipulated, but such studies are currently lacking. Future studies in humans could combine pharmacological interventions, neuroimaging and VR or use mobile EEG to help elucidate whether the plasticity enhancing mechanisms observed in animals, also exist in humans. When the aspects of exploring a novel environment underlying beneficial memory effects have been identified, effective novelty-exposure interventions could be designed to improve learning and counteract age-related memory decline. Show less
Although useful, digital screens have not yet replaced physical books. The printed book offers three concurrent psychological advantages that explain its puzzling resilience: (1) Better learning... Show moreAlthough useful, digital screens have not yet replaced physical books. The printed book offers three concurrent psychological advantages that explain its puzzling resilience: (1) Better learning and remembering are associated with the physical act of moving through the space of printed pages, which stimulates the visuo-spatial sketchpad component of working memory; (2) since reading is based on temporal, sequential processing, the fixed frame of reference of printed pages and their fixed shape are great aids to memory; (3) the properties of paper and the standard design of a printed book guarantee that it will always function properly: one day, a year, or many centuries later. One reads the same words on screen or in print, but the context in which the symbols are read is very different. The digital book appears and vanishes. The physical book – where kinesthetic experience is more vivid – remains as an actual, remembered space. Show less
Stijntjes, M.; Craen, A.J.M. de; Grond, J. van der; Meskers, C.G.M.; Slagboom, P.E.; Maier, A.B. 2016
