Pipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics... Show morePipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics Processing Units (GPUs). However, with the sustaining increase in Deep Learning (DL) model sizes, pipeline parallelism provides only a partial solution to the memory bottleneck in large-scale DNN training. Careful partitioning of the DL model over the available GPUs based on memory usage is required to further alleviate the memory bottleneck and train larger DNNs. mCAP is such a memory-oriented partitioning approach for pipeline parallel systems, but it does not scale to models with many layers and very large hardware setups, as it requires extensive profiling and fails to efficiently navigate the partitioning space to find the most memory-friendly partitioning. In this work, we propose CAPSlog, a scalable memory-centric partitioning approach that can recommend model partitionings for larger and more heterogeneous DL models and for larger hardware setups than existing approaches. CAPSlog introduces a new profiling method and a new, much more scalable algorithm for recommending memory-efficient partitionings. CAPSlog reduces the profiling time by 67% compared to existing approaches, searches the partitioning space for the optimal solution orders of magnitude faster and can train significantly larger models. Show less
Indonesia, as the world's most populous Muslim nation, finds itself at a pivotal juncture in the realm of technological advancement. Recognizing the profound impact of emerging technologies,... Show moreIndonesia, as the world's most populous Muslim nation, finds itself at a pivotal juncture in the realm of technological advancement. Recognizing the profound impact of emerging technologies, especially Artificial Intelligence (AI), on its trajectory is of utmost importance. While AI and Islamic beliefs may initially seem distinct, they share a common theme: an orientation toward envisioning future possibilities. Employing a blend of multimodal and mixed research methods, our objective is to scrutinize and draw comparisons between narratives and visual representations of AI's influence on religious prospects, particularly within the higher education landscape of Indonesia. We propose to investigate the curriculum, teaching practices and careers of young AI-professionals at Indonesia's oldest secular state university, namely Institut Teknologi Bandung (ITB), as well as Indonesia's first Islamic state university, namely Universitas Islamic Negeri Syarif Hidayatullah Jakarta (UIN-JKT). Notably, both institutions have recently introduced AI programs, making them rare in this regard. In this STEM-focused context, shedding light on the societal implications of AI education within Islam takes on heightened significance, with a focus on challenging Western-centric perspectives and contributing to a decolonization-centered research narrative. As this project is still in its early stages, this short paper will discuss related work and propose future directions to study Indonesia's AI-Islamic-educational future(s). Show less
The rapid advancement of Artificial Intelligence (AI) has opened up new frontiers of technological possibilities, yet the traditional education system in Indonesia has largely remained entrenched... Show moreThe rapid advancement of Artificial Intelligence (AI) has opened up new frontiers of technological possibilities, yet the traditional education system in Indonesia has largely remained entrenched in con- ventional practices. Despite Indonesia’s recognition for its vibrant AI innovation scene and its diverse student population, there is a signifi- cant gap in exploring the multifaceted implications of AI in, for, and by education within the country. This paper aims to address this gap by delving into three key areas: AI tutors, governance, and (virtue-based) ethical considerations. Drawing insights from both global and Islamic literature, we first examine the discourse surrounding AI tutors within Indonesia’s education system. Next, we discuss the potential applications of AI in governance, including the role of the government and the emer- gence of AI-related education in Indonesia. Thirdly, we contemplate an ethical framework encompassing issues of inequality, public policy, and Islamic-based principles. Throughout, this paper emphasizes the critical importance of examining these three facets of AI’s impact in education. Ultimately, this research raises the intriguing question of how education and AI will mutually shape each other in the future, urging further ex- ploration of this dynamic relationship. Show less
Onderzoek naar werkwoordspelling laat zien dat fouten in de vervoeging veroorzaakt worden door o.a. tijdsdruk en homofone werkwoordsvormen, zoals 'verhuist'-'verhuisd' (cf. Sandra, Frisson &... Show moreOnderzoek naar werkwoordspelling laat zien dat fouten in de vervoeging veroorzaakt worden door o.a. tijdsdruk en homofone werkwoordsvormen, zoals 'verhuist'-'verhuisd' (cf. Sandra, Frisson & Daems, 1999; Chamalaun, 2023). Wij onderzochten een ander struikelblok voor veel leerlingen: de spelling van leenwerkwoorden. Dergelijke werkwoorden dienen volgens de officiële spellingregels van de Nederlandse taal te worden vervoegd (Taalunie, 2011, p. 36), maar veelgebruikte ezelsbruggetjes, zoals die om de stam te vinden (wij-vorm minus 'en'), leveren hier het incorrecte 'gam' in plaats van 'game' op. Daarnaast wordt de spelling van leenwerkwoorden, zo blijkt uit onze data, vaak 'verengelst' - in plaats van 'genetflixt' wordt 'genetflixed' geschreven. In deze bijdrage beantwoorden wij de vraag in welke mate de leenwoordstatus van een werkwoord een rol speelt bij het maken van spelfouten. Daarnaast laten we zien in hoeverre er een relatie bestaat tot typen leenwerkwoorden uitgesplitst naar stameinde, zoals -e en -x in bovenstaande voorbeelden. Daartoe vergeleken we in een grootschalige analyse van recente data afkomstig van website Gespeld (Reuneker, z.d.) de door leerlingen gemaakte fouten in de vervoegingen van leenwerkwoorden en niet-leenwerkwoorden. We laten zien dat een beperkt aantal stamuitgangen het grootste deel van de fouten veroorzaakt en we doen suggesties voor het spellingonderwijs. Show less
Wang, Z.; Zhao, K.; He, Y.; Chen, Z.; Ren, P.; Rijke, M. de; Ren, Z. 2023
This paper proposes a new conceptual framework for the creation of an interoperable metaverse seamlessly incorporating AI and blockchain technologies. To achieve this, current issues of... Show moreThis paper proposes a new conceptual framework for the creation of an interoperable metaverse seamlessly incorporating AI and blockchain technologies. To achieve this, current issues of interoperability within the discipline’s state-of-the-art are first identified. Next, a new virtual ontology of the interoperable metaverse is proposed, inspired by recent developments in assemblage theory and in specific concepts by Gilles Deleuze, Manuel DeLanda, and Paulo de Assis. In the third and fourth parts of this paper, a new technical solution is presented: a taxonomy of Periodic Spacetime Sequences, devised for practical implementation of a new distributed version control system. This solution could offer an innovative framework for the interoperable metaverse, which could be built using the proposed taxonomy. Show less
Dreuning, H.; Bal, H.E.; Nieuwpoort, R.V. van 2023
Deep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models... Show moreDeep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models contain billions of trainable parameters. Training such massive neural networks incurs significant memory requirements and financial cost. Hybrid-parallel training approaches have emerged that combine pipelining with data and tensor parallelism to facilitate the training of large DL models on distributed hardware setups. However, existing approaches to design a hybrid-parallel partitioning and parallelization plan for DL models focus on achieving high throughput and not on minimizing memory usage and financial cost. We introduce CAPTURE, a partitioning and parallelization approach for hybrid parallelism that minimizes peak memory usage. CAPTURE combines a profiling-based approach with statistical modeling to recommend a partitioning and parallelization plan that minimizes the peak memory usage across all the Graphics Processing Units (GPUs) in the hardware setup. Our results show a reduction in memory usage of up to 43.9% compared to partitioners in state-of-the-art hybridparallel training systems. The reduced memory footprint enables the training of larger DL models on the same hardware resources and training with larger batch sizes. CAPTURE can also train a given model on a smaller hardware setup than other approaches, reducing the financial cost of training massive DL models. Show less
In this work, we propose a novel joint frailty model assuming bivariate discretely- distributed non-parametric frailties, with an unknown finite number of mass points. This ap- proach allows to... Show moreIn this work, we propose a novel joint frailty model assuming bivariate discretely- distributed non-parametric frailties, with an unknown finite number of mass points. This ap- proach allows to detect a latent structure among subjects, clustering them in sub-populations where individuals are characterized by a common frailty value. Our method can be interpreted as an unsupervised classification tool and motivates further investigation into the reasons for similarities within the clustered subjects and dissimilarities across the clusters. This work is motivated by a study of patients with Heart Failure (HF) undergoing ACE inhibitors treatment in the Lombardia region of Italy. Recurrent events of interest are hos- pitalizations due to HF and terminal event is death for any cause. Show less
A destructive force across entire landscapes, mining at times threatens the preservation of priceless archaeological sites across southern Africa. The UNESCO World Heritage site of the Mapungubwe... Show moreA destructive force across entire landscapes, mining at times threatens the preservation of priceless archaeological sites across southern Africa. The UNESCO World Heritage site of the Mapungubwe Cultural Landscape is one of the recent legal battlegrounds pitting the interests of the natural world and archaeological heritage against mining. Mineral resource extraction carries within its destruction creative potential too. In southern Africa mining has been instrumental in illuminating human history and in the creation of archaeological landmarks.The relationship between archaeology and mining is a complex one. The gold industry and associated lime-mining has specifically been instrumental in the development of palaeoanthropology in South Africa (Bonner 2007). With the discovery of gold, lime was required for the desulfurization process. This was obtained locally and many lime works were exploited during the 1890s (Esterhuysen 2019). The miners drew attention to fossils found within the deposits leading to the discovery of hominins at Taung, Makapan and Sterkfontein in the early 20th century. Mining may thus be seen as crucial to the genesis of the Cradle of Humankind UNESCO World Heritage site in South Africa. Similarly guano, zinc and vanadium mining have yielded important palaeoanthropological finds in southern Africa.In the 19th and early 20th century, scientific focus was on discovering unknown objects. In this context, the efficient removal of great quantities of overburden was a great asset. With increasing scientific rigour in the 20th century, archaeological focus shifts from objects to their interrelationships and their context. The documentation of finds and find contexts requires time that is not available in the context of industrial resource extraction. Fruitful collaboration now becomes more difficult. As the early lime works closed down in the first half of the 20th century, archaeologists were free to continue work at a comparative snail’s pace but yielding far greater insight in the human past.In active mining operations, collaboration is difficult. While mining still exposes valuable sites and materials that would otherwise be inaccessible, this is of limited value in current archaeological practice. Current ethical standards emphasise in situ preservation of archaeological remains over excavation. When finds are documented, the time-pressure from mining companies makes these situations ambiguous in value. Examples are the discovery of Pleistocene shell middens after their partial destruction, or the recovery of lithic materials from conveyor belts in diamond plants.In the late 20th century and 21st century, the discipline of archaeology has been included in legal frameworks on environmental planning. Archaeological contractors conduct Heritage impact assessments and advise on mitigation measures for large-scale environmental projects such as mining operation and the building of large dams of infrastructure works.Highlighting human history was an unintended consequence for large mining operations. Yet both on a grand level, the activity of mining, as well as on a personal level, the activities of miners and foremen have played a crucial role in uncovering ancient humans and their world in southern Africa. I argue that the insights produced as a result of early resource extraction are one (albeit often minor) consideration in the network of values, stakeholders and resources that embed mining operations in society. Show less
