Weakly supervised phrase grounding aims to map the phrases in an image caption to the objects appearing in the image under the supervision of image-caption correspondence. We observe that the... Show moreWeakly supervised phrase grounding aims to map the phrases in an image caption to the objects appearing in the image under the supervision of image-caption correspondence. We observe that the current studies are insufficient to model the complicated interactions between the visual components (i.e., the visual regions) and between the visual and textual components (i.e., the phrases). Therefore, this paper presents a novel weakly supervised learning approach to phrase grounding in which we systematically model the visual contextualized representation with three modules: (1) object proposals pooling (OPP), (2) visual self-attention (VSA) and (3) visual-textual cross-modal attention (VTCA). OPP alleviates the suppression of the object proposals and benefits the visual representation in terms of trading off the richness of the visual components and the computational efficiency. VSA aims to capture the correlation among the object proposals and generate a representation of each proposal by incorporating the visual information of the others. To measure the cross-modal compatibility in terms of topics, we introduce the VTCA module to represent the visual topic corresponding to each textual component in a cross-modal common vector space. In the training process, we build a mixed contrastive loss function by considering both the cross-modal compatibility and the differences in the visual representations in the VSA module. Compared with the state-of-the-art methods, the proposed approach improves the performance by 3.88% points and 1.24% points on R@1, and by 2.23% points and 0.26% points on Pt_Acc, when trained on the MS COCO and Flickr30K Entities training sets, respectively. We have made our code available for follow-up research. Show less
Wang, X.; Du, Y.; Verberne, S.; Verbeek, F.J. 2022
This paper addresses the task of fine-grained label learning in object detection with the weak supervision of auxiliary information attached to images. Most of the recent work focused on the label... Show moreThis paper addresses the task of fine-grained label learning in object detection with the weak supervision of auxiliary information attached to images. Most of the recent work focused on the label prediction for objects in the same category space as in training data under the fully-supervised learning framework and cannot be expanded to the learning of more fine-grained categories that have not been defined in training sets. In this paper, we propose a new weakly-supervised learning approach, called label inference curriculum network (LICN), to detecting objects and learning their fine-grained category labels based on supervision of captions via curriculum learning. First, we build a semantic mapping based on embedding techniques and a knowledge base to measure the correspondence between coarse labels and fine-grained label proposals; second, we introduce a label inference curriculum network, which ranks the order of training samples by the complexity of samples. We construct two datasets, namely FG-COCO and FGs-COCO, consisting of both coarse and fine-grained labels based on MS COCO and Visual Genome to train and test our approach. Experimental results demonstrate the effectiveness of our proposed LICN model, and LICN-E2C achieves an improvement of 1.7% mAP with 0.5:0.05:0.95 IoU compared with the LICN-C2E on the FG-sCOCO test dataset. Show less
Xie, M.; Liu, H.; Wan, S.; Lu, X.; Hong, D.; Du, Y.; ... ; Tian, Y. 2022
.Sensitive detection of weak acoustic signals at nanometer scale is challenging. Here, the authors present an acoustic detection system based on a single molecule as a probe, where frequency and... Show more.Sensitive detection of weak acoustic signals at nanometer scale is challenging. Here, the authors present an acoustic detection system based on a single molecule as a probe, where frequency and amplitude of acoustic vibrations can be extracted from its minute variations in distance to the surface of a plasmonic gold nanorod.Sensitive detection of local acoustic vibrations at the nanometer scale has promising potential applications involving miniaturized devices in many areas, such as geological exploration, military reconnaissance, and ultrasound imaging. However, sensitive detection of weak acoustic signals with high spatial resolution at room temperature has become a major challenge. Here, we report a nanometer-scale system for acoustic detection with a single molecule as a probe based on minute variations of its distance to the surface of a plasmonic gold nanorod. This system can extract the frequency and amplitude of acoustic vibrations with experimental and theoretical sensitivities of 10 pm Hz(-1/2) and 10 fm Hz(-1/2), respectively. This approach provides a strategy for the optical detection of acoustic waves based on molecular spectroscopy without electromagnetic interference. Moreover, such a small nano-acoustic detector with 40-nm size can be employed to monitor acoustic vibrations or read out the quantum states of nanomechanical devices. Show less
Xiao, Y.; Ma, J.; Xian, J.; Peijnenburg, W.J.G.M.; Du, Y.; Tian, D.; ... ; Lihua, T 2022
Copper (Cu) pollution is common in the soil. Due to the widespread application of TiO2 NPs, there is a high propensity for the co-occurrence of TiO2 nanoparticles (NPs) and Cu in agricultural soils... Show moreCopper (Cu) pollution is common in the soil. Due to the widespread application of TiO2 NPs, there is a high propensity for the co-occurrence of TiO2 nanoparticles (NPs) and Cu in agricultural soils. It is therefore imperative to evaluate the joint effects of TiO2 NPs and Cu on crops. In this study, the mutual effects of TiO2 NPs and Cu on their toxicity and accumulation in soybean seedlings and on their fates in a hydroponic system were determined. When Cu was at levels of 1 and 2 mg/L, the co-occurring TiO2 NPs at a non-toxic concentration (10 mg/L) significantly enhanced the toxicity and accumulation of Cu and Ti in soybeans, and inhibited the translocation of Cu from soybean roots to shoots. However, when the Cu concentration for co-exposure was ≥ 5 mg/L, such mutual effects disappeared. The amount of Cu ions adsorbed onto TiO2 NPs after 48 h of co-exposure gradually increased from 31 to 118 mg/g when the Cu concentration was increased from 1 to 20 mg/L. The aggregation and sedimentation of TiO2 NPs were significantly increased after 48 h of co-exposure with the Cu at a concentration higher than 5 mg/L, as compared to the single TiO2 NPs exposure. The increasing aggregation and sedimentation might reduce the bioavailability of TiO2 NPs associated with the adsorbed Cu to soybeans, and consequently alleviate or even neutralize the enhanced toxicity and accumulation of Cu in soybeans exerted by the co-existing TiO2 NPs. Our results thus suggest that consideration of the impact of TiO2 NPs on the phytotoxicity of heavy metals, and specifically Cu, needs to be interpreted with care, and highlight the importance of integrating the interaction and fates of TiO2 NPs and metals into their risk assessment. Show less
Subunit-selective proteasome inhibitors are valuable tools to assess the biological and medicinal relevance of individual proteasome active sites. Whereas the inhibitors for the β1c, β1i, β5c, and... Show more Subunit-selective proteasome inhibitors are valuable tools to assess the biological and medicinal relevance of individual proteasome active sites. Whereas the inhibitors for the β1c, β1i, β5c, and β5i subunits exploit the differences in the substrate-binding channels identified by X-ray crystallography, compounds selectively targeting β2c or β2i could not yet be rationally designed because of the high structural similarity of these two subunits. Here, we report the development, chemical synthesis, and biological screening of a compound library that led to the identification of the β2c- and β2i-selective compounds LU-002c (4; IC50 β2c: 8 nM, IC50 β2i/β2c: 40-fold) and LU-002i (5; IC50 β2i: 220 nM, IC50 β2c/β2i: 45-fold), respectively. Co-crystal structures with β2 humanized yeast proteasomes visualize protein–ligand interactions crucial for subunit specificity. Altogether, organic syntheses, activity-based protein profiling, yeast mutagenesis, and structural biology allowed us to decipher significant differences of β2 substrate-binding channels and to complete the set of subunit-selective proteasome inhibitors. Show less
Xin, B.T.; Schimmack, G.; Du, Y.; Florea, B.I.; Marel, G.A. van der; Driessen, C.; ... ; Overkleeft, H.S. 2016