Purpose: Upper airway segmentation on MR images is a prerequisite step for quantitatively studying the anatomical structure and function of the upper airway and surrounding tissues. However, the... Show morePurpose: Upper airway segmentation on MR images is a prerequisite step for quantitatively studying the anatomical structure and function of the upper airway and surrounding tissues. However, the complex variability of intensity and shape of anatomical structures and different modes of image acquisition commonly used in this application makes automatic upper airway segmentation challenging. In this paper, we develop and test a comprehensive deep learning-based segmentation system for use on MR images to address this problem. Materials and Methods: In our study, both static and dynamic MRI data sets are utilized, including 58 axial static 3D MRI studies, 22 mid-retropalatal dynamic 2D MRI studies, 21 mid-retroglossal dynamic 2D MRI studies, 36 mid-sagittal dynamic 2D MRI studies, and 23 isotropic dynamic 3D MRI studies, involving a total of 160 subjects and over 20 000 MRI slices. Samples of static and 2D dynamic MRI data sets were randomly divided into training, validation, and test sets by an approximate ratio of 5:2:3. Considering that the variability of annotation data among 3D dynamic MRIs was greater than for other MRI data sets, we increased the ratio of training data for these data to improve the robustness of the model. We designed a unified framework consisting of the following procedures. For static MRI, a generalized region-of-interest (GROI) strategy is applied to localize the partitions of nasal cavity and other portions of upper airway in axial data sets as two separate subobjects. Subsequently, the two subobjects are segmented by two separate 2D U-Nets. The two segmentation results are combined as the whole upper airway structure. The GROI strategy is also applied to other MRI modes. To minimize false-positive and false-negative rates in the segmentation results, we employed a novel loss function based explicitly on these rates to train the segmentation networks. An inter-reader study is conducted to test the performance of our system in comparison to human variability in ground truth (GT) segmentation of these challenging structures. Results: The proposed approach yielded mean Dice coefficients of 0.84 +/- 0.03, 0.89 +/- 0.13, 0.84 +/- 0.07, and 0.86 +/- 0.05 for static 3D MRI, mid-retropalatal/mid-retroglossal 2D dynamic MRI, mid-sagittal 2D dynamic MRI, and isotropic dynamic 3D MRI, respectively. The quantitative results show excellent agreement with manual delineation results. The inter-reader study results demonstrate that the segmentation performance of our approach is statistically indistinguishable from manual segmentations considering the inter-reader variability in GT. Conclusions: The proposed method can be utilized for routine upper airway segmentation from static and dynamic MR images with high accuracy and efficiency. The proposed approach has the potential to be employed in other dynamic MRI-related applications, such as lung or heart segmentation. Show less
In this paper we propose a supervised method to predict registration misalignment using convolutional neural networks (CNNs). This task is casted to a classification problem with multiple classes... Show moreIn this paper we propose a supervised method to predict registration misalignment using convolutional neural networks (CNNs). This task is casted to a classification problem with multiple classes of misalignment: "correct" 0-3 mm, "poor" 3-6 mm and "wrong" over 6 mm. Rather than a direct prediction, we propose a hierarchical approach, where the prediction is gradually refined from coarse to fine. Our solution is based on a convolutional Long Short-Term Memory (LSTM), using hierarchical misalignment predictions on three resolutions of the image pair, leveraging the intrinsic strengths of an LSTM for this problem. The convolutional LSTM is trained on a set of artificially generated image pairs obtained from artificial displacement vector fields (DVFs). Results on chest CT scans show that incorporating multi-resolution information, and the hierarchical use via an LSTM for this, leads to overall better F1 scores, with fewer misclassifications in a well-tuned registration setup. The final system yields an accuracy of 87.1%, and an average F1 score of 66.4% aggregated in two independent chest CT scan studies. Show less
Lens, F.P.; Liang, C.; Guo, Y.; Tang, X.; Jahanbanifard, M.; Soares Correa da Silva, F.; ... ; Verbeek, F.J. 2020
Wood anatomy is one of the most important methods for timber identification. However, training wood anatomy experts is time-consuming, while at the same time the number of senior wood anatomists... Show moreWood anatomy is one of the most important methods for timber identification. However, training wood anatomy experts is time-consuming, while at the same time the number of senior wood anatomists with broad taxonomic expertise is de- clining. Therefore, we want to explore how a more automated, computer-assisted approach can support accurate wood identification based on microscopic wood anatomy. For our exploratory research, we used an available image dataset that has been applied in several computer vision studies, consisting of 112 — mainly neotropical — tree species representing 20 images of transverse sections for each species. Our study aims to review existing computer vision methods and compare the success of species identification based on (1) several image classifiers based on manually adjusted texture features, and (2) a state-of-the-art approach for image classification based on deep learning, more specifically Convolutional Neural Networks (CNNs). In support of previous studies, a considerable increase of the correct identification is accomplished using deep learning, leading to an accuracy rate up to 95.6%. This remarkably high success rate highlights the fundamental potential of wood anatomy in species identification and motivates us to expand the existing database to an extensive, worldwide reference database with transverse and tangential microscopic images from the most traded timber species and their look-a-likes. This global reference database could serve as a valuable future tool for stakeholders involved in combatting illegal logging and would boost the societal value of wood anatomy along with its collections and experts. Show less
Higher dimensional data such as video and 3D are the leading edge of multimedia retrieval and computer vision research. In this survey, we give a comprehensive overview and key insights into the... Show moreHigher dimensional data such as video and 3D are the leading edge of multimedia retrieval and computer vision research. In this survey, we give a comprehensive overview and key insights into the state of the art of higher dimensional features from deep learning and also traditional approaches. Current approaches are frequently using 3D information from the sensor or are using 3D in modeling and understanding the 3D world. With the growth of prevalent application areas such as 3D games, self-driving automobiles, health monitoring and sports activity training, a wide variety of new sensors have allowed researchers to develop feature description models beyond 2D. Although higher dimensional data enhance the performance of methods on numerous tasks, they can also introduce new challenges and problems. The higher dimensionality of the data often leads to more complicated structures which present additional problems in both extracting meaningful content and in adapting it for current machine learning algorithms. Due to the major importance of the evaluation process, we also present an overview of the current datasets and benchmarks. Moreover, based on more than 330 papers from this study, we present the major challenges and future directions. Show less
