This thesis mainly focuses on multimodal understanding and Visual Question Answering (VQA) via deep learning methods. For technical contributions, this thesis first focuses on improving multimodal... Show moreThis thesis mainly focuses on multimodal understanding and Visual Question Answering (VQA) via deep learning methods. For technical contributions, this thesis first focuses on improving multimodal fusion schemes via multi-stage vision-language interactions. Then, the thesis seeks to overcome the language bias challenges to build robust VQA models, and also extend the bias problem into the more complex audio-visual-textual question answering tasks. Furthermore, this thesis explores the open-world applicability of VQA algorithms from the aspects of lifelong learning and federated learning, thereby expanding the continuous and distributed training ability. The efficacy of the proposed methods in this thesis is verified by extensive experiments. This thesis also gives an overview of challenges, benchmarks and strategies for robust VQA algorithms. Show less
Sbrollini, A.; Barocci, M.; Mancinelli, M.; Paris, M.; Raffaelli, S.; Marcantoni, I.; ... ; Burattini, L. 2023
Heart failure (HF) diagnosis, typically visually performed by serial electrocardiography, may be supported by machine-learning approaches. Repeated structuring & learning procedure (RS & LP... Show moreHeart failure (HF) diagnosis, typically visually performed by serial electrocardiography, may be supported by machine-learning approaches. Repeated structuring & learning procedure (RS & LP) is a constructive algorithm able to automatically create artificial neural networks (ANN); it relies on three parameters, namely maximal number of hidden layers (MNL), initializations (MNI) and confirmations (MNC), arbitrarily set by the user. The aim of this study is to evaluate RS & LP robustness to varying values of parameters and to identify an optimized combination of parameter values for HF diagnosis. To this aim, the Leiden University Medical Center HF data-base was used. The database is constituted by 129 serial ECG pairs acquired in patients who experienced myocardial infarction; 48 patients developed HF at follow-up (cases), while 81 remained clinically stable (controls). Overall, 15 ANNs were created by considering 13 serial ECG features as inputs (extracted from each serial ECG pair), 2 classes as outputs (cases/controls), and varying values of MNL (1, 2, 3, 4 and 10), MNI (50, 250, 500, 1000 and 1500) and MNC (2, 5, 10, 20 and 50). The area under the curve (AUC) of the receiver operating characteristic did not significantly vary with varying parameter values (P >= 0.09). The optimized combination of parameter values, identified as the one showing the highest AUC, was obtained for MNL = 3, MNI = 500 and MNC = 50 (AUC = 86 %; ANN structure: 3 hidden layers of 14, 14 and 13 neurons, respectively). Thus, RS & LP is robust, and the optimized ANN represents a potentially useful clinical tool for a reliable auto-matic HF diagnosis. Show less
The archaeology domain produces large amounts of texts, too much to effectively read or manually search through for research. To alleviate this problem, we created a search system (called AGNES),... Show moreThe archaeology domain produces large amounts of texts, too much to effectively read or manually search through for research. To alleviate this problem, we created a search system (called AGNES), which combines full text search with entity and geographical search. We first created a manually labelled data set to train a Named Entity Recognition model, which is used to extract entities from text. We also did a user requirement study, and usability evaluation on the system, to make sure it is suitable for archaeological research. In a case study on Early Medieval cremations, we show that using AGNES leads to a knowledge increase when compared to the knowledge of experts, gathered using previously available search engines. This shows that this kind of intelligent search system can help with literature research, find more relevant data, and lead to a better understanding of the past. Show less
The manual analysis of remotely-sensed data is a widespread practice in local and regional scale archaeological research, as well as heritage management. However, the amount of available high... Show moreThe manual analysis of remotely-sensed data is a widespread practice in local and regional scale archaeological research, as well as heritage management. However, the amount of available high-quality, remotely-sensed data is continuously growing at a staggering rate, which creates new challenges to effectively and efficiently analyze these data and find and document the seemingly overwhelming number of potential archaeological objects. Therefore, computer-aided methods for the automated detection of archaeological objects are needed. In this thesis, the development and application of automated detection methods, based on Deep Convolutional Neural Networks, for the detection of multiple classes of archaeological objects in LiDAR data is investigated. Furthermore, the implementation of these methods into archaeological practice and the opportunities of knowledge discovery—on both a quantitative and qualitative level—for landscape or spatial archaeology are explored. Show less
