Evidence Transfer: A versatile deep representation learning method for information fusion

Abstract

In the recent years, the collection of more and more data instances, has increasingly storing, managing and collecting of large-scale or diverse data, the research interest of the scientific community has shifted into the extraction of meaningful information from such collections. Deep learning lends itself particularly well to the process of extracting valuable information. Deep learning methods thrive with large-scale datasets. Due to their ability to learn alternative representations from raw observations, the abundance of data instances allows for generalised representations. In turn, generalised representations allow for effective learning of complex tasks. Despite valuable efforts in extraction of information from single data sources or data types, dealing with multiple diverse data sources remains an open question in the scientific community. Representation learning enables combination and juxtaposition of multiple diverse data sources in a meaningful, common and lower-dimensional space. However, typical learning frameworks for joint representation learning, face a plethora of challenges. Initially, architectural decisions of the involved neural networks is often a product of manual work or application specific decisions that rarely generalise to multiple domains or tasks. At the same time, directly tying data sources in the input layers of the neural network introduces an expectation of constant availability. However, expecting all data sources to be constantly available, is not realistic in real world applications. In addition, the involvment of redudant or non-complementary data sources may lead to detoriating performance. However, dealing with such sources requires manual effort. Such effort, is put into creating explicit assumptions or rules that will ensure stability, or to understand the intricate relations between data sources, in order to avoid non-complementary ones. This thesis includes the formulation and investigation of the hypothesis that external data evidence improves deep representation learning. The above investigation results in the proposal of a deep representation learning method, called Evidence Transfer (EviTraN). EviTraN is a versatile and automated fusion scheme based on deep representation learning, transfer learning and hybrid modelling between genervii ative and discriminative views. In addition, it leads to the proposal of a set of evaluation criteria for deep representation learning for the purposes of information fusion. Furthermore, this thesis includes a theoretical interpretation of the above method, based on comparison with the well-received Information Bottleneck method, that acts as a stepping stone towards explainable modelling and open science. The evaluation process of EviTraN also includes a realistic scenario of detecting severe weather in an unsupervised manner. Thus, demonstrating its impact and potential use in additional real-world applications. Experimental evaluation with artificially generated, as well as, realistic evidence sources suggest that EviTraN is a robust and effective method. In addition, it is versatile, as it allows the introduction of a variety of relations, including non-complementary ones. Furthermore, due to its learning process based on the transfer learning setting, it is a modular fusion scheme that does not require all data sources to be present during inference (only the primary data instances).