In this thesis, topics relating to the optimization of high-throughput pipelines used for imaging are discussed. In particular, different levels of implementation, i.e., conceptual, software, and... Show moreIn this thesis, topics relating to the optimization of high-throughput pipelines used for imaging are discussed. In particular, different levels of implementation, i.e., conceptual, software, and hardware, are discussed and the thesis outlines how advances on each level need to be made to make gains in computationally demanding imaging applications. Chapter 2 concerns the implementation of real-time segmentation of X-ray computed tomography (CT). Chapter 3 concerns the end-to-end optimization of various CT workflows by using auto-differentiation frameworks. Chapter 4 concerns a novel pruning method for neural network to significantly increase the speed of convolutional neural networks (CNNs). Chapter 5 comprises a benchmark study of optimization algorithms for tuning GPU kernels, and introduces a novel graph-based approach to quantify search space difficulty. Chapter 6 introduces a novel model to improve the energy efficiency of GPUs. Show less
Dreuning, H.; Bal, H.E.; Nieuwpoort, R.V. van 2023
Deep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models... Show moreDeep Learning (DL) model sizes are increasing at a rapid pace, as larger models typically offer better statistical performance. Modern Large Language Models (LLMs) and image processing models contain billions of trainable parameters. Training such massive neural networks incurs significant memory requirements and financial cost. Hybrid-parallel training approaches have emerged that combine pipelining with data and tensor parallelism to facilitate the training of large DL models on distributed hardware setups. However, existing approaches to design a hybrid-parallel partitioning and parallelization plan for DL models focus on achieving high throughput and not on minimizing memory usage and financial cost. We introduce CAPTURE, a partitioning and parallelization approach for hybrid parallelism that minimizes peak memory usage. CAPTURE combines a profiling-based approach with statistical modeling to recommend a partitioning and parallelization plan that minimizes the peak memory usage across all the Graphics Processing Units (GPUs) in the hardware setup. Our results show a reduction in memory usage of up to 43.9% compared to partitioners in state-of-the-art hybridparallel training systems. The reduced memory footprint enables the training of larger DL models on the same hardware resources and training with larger batch sizes. CAPTURE can also train a given model on a smaller hardware setup than other approaches, reducing the financial cost of training massive DL models. Show less