Håkan Lane has joined the Data Mining group and will be working in the Cluster for Atherothrombosis and Individualized Medicine (curATime), a future cluster funded by the BMBF, where powerful players from science and industry have joined forces to develop tailored treatment and prevention concepts for cardiovascular diseases and their clinical application.

Our paper "Identifying Aircraft Motions and Patterns from Magnetometry Data Using a Knowledge-Based Multi-Fusion Approach", which is a joint work of Julian Vexler and Stefan Kramer, has been accepted at the international conference on information fusion (https://fusion2023.org/). The paper is a result of an on-going project about the air-side integration of magnetometers for object detection.

Abstract

In aviation there are many safety-critical domains where reliable safety systems are essential to prevent any kind of hazard. This paper focuses on airport aprons, where currently used holding point protection systems have shown to be not faultless, sometimes leading to avoidable accidents. One way to avoid such accidents is by means of innovative sensor technology, in our case, magnetometers, i.e. sensors measuring the distortion of the earth’s magnetic field by metallic objects. The main goal is to use the magnetometry data to detect passing aircraft and to capture their geometrical pattern as well as to estimate their motion vector. Therefore, we present a spatio-temporal cluster fusion and an event fusion algorithm. The cluster fusion can be applied as a post-processing step to any spatio-temporal clustering method and is able to more accurately represent aircraft patterns by integrating expert knowledge into the fusion process. In this context, we present a spatio-temporal cluster tree representation for a fast and accurate estimation of the motion vector. Finally, the data-driven event fusion is able to separate detected aircraft crossings into separate events by employing domain-knowledge. In future work, we aim to come up with a framework making use of the cluster results and estimated motion vector to classify and infer the position of an aircraft, before this is deployed as a real-time application.

Our paper "Four-dimensional trapped ion mobility spectrometry lipidomics for high throughput clinical profiling of human blood samples", which is a joint work of Raissa Lerner, Dhanwin Baker, Claudia Schwitter, Sarah Neuhaus, Tony Hauptmann, Julia M. Post, Stefan Kramer & Laura Bindila, has been accepted at Nature Communications.

Abstract

Lipidomics encompassing automated lipid extraction, a four-dimensional (4D) feature selection strategy for confident lipid annotation as well as reproducible and cross-validated quantification can expedite clinical profiling. Here, we determine 4D descriptors (mass to charge, retention time, collision cross section, and fragmentation spectra) of 200 lipid standards and 493 lipids from reference plasma via trapped ion mobility mass spectrometry to enable the implementation of stringent criteria for lipid annotation. We use 4D lipidomics to confidently annotate 370 lipids in reference plasma samples and 364 lipids in serum samples, and reproducibly quantify 359 lipids using level-3 internal standards. We show the utility of our 4D lipidomics workflow for high-throughput applications by reliable profiling of intra-individual lipidome phenotypes in plasma, serum, whole blood, venous and finger-prick dried blood spots.

Our paper "A fair experimental comparison of neural network architectures for latent representations of multi-omics for drug response prediction", which is a joint work of Tony Hauptmann and Stefan Kramer, has been accepted at BMC Bioinformatics.

 

Abstract:

Background

Recent years have seen a surge of novel neural network architectures for the integration of multi-omics data for prediction. Most of the architectures include either encoders alone or encoders and decoders, i.e., autoencoders of various sorts, to transform multi-omics data into latent representations. One important parameter is the depth of integration: the point at which the latent representations are computed or merged, which can be either early, intermediate, or late. The literature on integration methods is growing steadily, however, close to nothing is known about the relative performance of these methods under fair experimental conditions and under consideration of different use cases.

 

Results

We developed a comparison framework that trains and optimizes multi-omics integration methods under equal conditions. We incorporated early integration, PCA and four recently published deep learning methods: MOLI, Super.FELT, OmiEmbed, and MOMA. Further, we devised a novel method, Omics Stacking, that combines the advantages of intermediate and late integration. Experiments were conducted on a public drug response data set with multiple omics data (somatic point mutations, somatic copy number profiles and gene expression profiles) that was obtained from cell lines, patient-derived xenografts, and patient samples. Our experiments confirmed that early integration has the lowest predictive performance. Overall, architectures that integrate triplet loss achieved the best results. Statistical differences can, overall, rarely be observed, however, in terms of the average ranks of methods, Super.FELT is consistently performing best in a cross-validation setting and Omics Stacking best in an external test set setting.

 

Conclusions

We recommend researchers to follow fair comparison protocols, as suggested in the paper. When faced with a new data set, Super.FELT is a good option in the cross-validation setting as well as Omics Stacking in the external test set setting. Statistical significances are hardly observable, despite trends in the algorithms’ rankings. Future work on refined methods for transfer learning tailored for this domain may improve the situation for external test sets. The source code of all experiments is available at Github.

Our paper "Invariant Representations with Stochastically Quantized Neural Networks", which is a joint work of Mattia Cerrato, Marius Köppel, Roberto Esposito and Stefan Kramer has been accepted at AAAI 2023.

 

Abstract:

Representation learning algorithms offer the opportunity to learn invariant representations of the input data with regard to nuisance factors. Many authors have leveraged such strategies to learn fair representations, i.e., vectors where information about sensitive attributes is removed. These methods are attractive as they may be interpreted as minimizing the mutual information between a neural layer's activations and a sensitive attribute. However, the theoretical grounding of such methods relies either on the computation of infinitely accurate adversaries or on minimizing a variational upper bound of a mutual information estimate. In this paper, we propose a methodology for direct computation of the mutual information between a neural layer and a sensitive attribute. We employ stochastically-activated binary neural networks, which lets us treat neurons as random variables. We are then able to compute (not bound) the mutual information between a layer and a sensitive attribute and use this information as a regularization factor during gradient descent. We show that this method compares favorably with the state of the art in fair representation learning and that the learned representations display a higher level of invariance compared to full-precision neural networks.

Kirsten Köbschall and Kiara Stempel have joined the Data Mining group and will be working on the project Trading off Non-Functional Properties of Machine Learning (TOPML).

The TOPML project ("Trading off Non-Functional Properties of Machine Learning") funded by the Carl-Zeiss-Stiftung, started on the 1st of July. An online kick-off meeting has taken place on the 6th of July, hosted by the spokesperson Prof. Stefan Kramer.

Prof. Stefan Kramer gave an informal presentation which touched upon the various themes of TOP-ML (fairness, interpretability, resource-efficiency, privacy, law and ethics) and described opportunities for collaboration. In particular, a series of lunch meetings is currently being planned. As the project members have different backgrounds, information sharing and regular informal meetings will help foster collaboration and multi-disciplinarity.

New PhD Students and researchers are currently being hired, with the first ones starting in the month of July. More hirings are currently planned for early fall.

The findings from the various themes are to be used within JGU itself, but also transferred to industrial practice. An AI Lab is to be located at Mainz University of Applied Sciences due to its proximity to regional and national industry.  The developed methods in the planned research project will be implemented as software applications in the lab. They will serve to test and validate the developed methods. In addition, the implementation of the results as executable software applications ensures a successful transfer to science and industry

The University Medical Center Mainz also participated in the meeting to explore ways of joining the TOPML project.

 

Our preprint "Fair Interpretable Representation Learning with Correction Vectors" (by Cerrato, Coronel, Koppel, Segner, Esposito and Kramer) has been featured by the Montreal AI Ethics Institute. You may read the research summary here.

Our paper "Learning to Rank Higgs Boson Candidates", which is a joint work of Marius Köppel, Alexander Segner, Martin Wagener, Lukas Pensel, Andreas Karwath, Christian Schmitt and Stefan Kramer has been accepted at Nature Scientific Reports.

 

Abstract:

In the extensive search for new physics, the precise measurement of the Higgs boson continues to play an important role. To this end, machine learning techniques have been recently applied to processes like the Higgs production via vector-boson fusion. In this paper, we propose to use algorithms for learning to rank, i.e., to rank events into a sorting order, first signal, then background, instead of algorithms for the classification into two classes, for this task. The fact that training is then performed on pairwise comparisons of signal and background events can effectively increase the amount of training data due to the quadratic number of possible combinations. This makes it robust to unbalanced data set scenarios and can improve the overall performance compared to pointwise models like the state-of-the-art boosted decision tree approach. In this work we compare our pairwise neural network algorithm, which is a combination of a convolutional neural network and the DirectRanker, with convolutional neural networks, multilayer perceptrons or boosted decision trees, which are commonly used algorithms in multiple Higgs production channels. Furthermore, we use so-called transfer learning techniques to improve overall performance on different data types.

Our short paper "Ranking Creative Language Characteristics in Small Data Scenarios", which is a joint work of Julia Siekiera, Marius Köppel, Edwin Simpson, Kevin Stowe, Iryna Gurevych, Stefan Kramer has been accepted at ICCC'22.

 

Abstract:

The ability to rank creative natural language provides an important general tool for downstream language understanding and generation. However, current deep ranking models require substantial amounts of labeled data that are difficult and expensive to obtain for new domains, languages and creative characteristics. A recent neural approach, DirectRanker, reduces the amount of training data needed but has not previously been used to rank creative text. We therefore adapt DirectRanker to provide a new deep model for ranking creative language with small numbers of training instances, and compare it with a Bayesian approach, Gaussian process preference learning (GPPL), which was previously shown to work well with sparse data. Our experiments with short creative language texts show the effectiveness of DirectRanker even with small training datasets. Combining DirectRanker with GPPL outperforms the previous state of the art on humor and metaphor novelty tasks, increasing Spearman's ρ by 25% and 29% on average. Furthermore, we provide a possible application to validate jokes in the process of creativity generation.