Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learning

1 University of Verona, Str. le Grazie, 15, Verona, 37134, Italy 2 HUMATICS - SYS-DAT Group, Str. le Grazie, 15, Verona, 37134, Italy 3 University of Milano-Bicocca, Piazza dell'Ateneo Nuovo, 1, Milan, 20126, Italy
Detaux teaser
Detaux involves two steps. 1) First, we use weakly supervised disentanglement to isolate the structural features specific to the principal task in one subspace (red rectangle at the top of the image). 2) Next, we identify the subspace with the most disentangled factor of variation related to the principal task, and through a clustering module, we obtain new labels (blue rectangle in the bottom left part of the image). These can be used to create a new classification task that can be combined with the principal task in any MTL model (bottom right part of the image).


In deep learning, auxiliary objectives are often used to facilitate learning in situations where data is scarce or the principal task is extremely complex. This idea is primarily inspired by the improved generalization capability induced by solving multiple tasks simultaneously, which leads to a more robust shared representation. Nevertheless, finding optimal auxiliary tasks that give rise to the desired improvement is a crucial problem that often requires hand-crafted solutions or expensive meta-learning approaches. In this paper, we propose a novel framework, dubbed Detaux, whereby a weakly supervised disentanglement procedure is used to discover new unrelated classification tasks and the associated labels that can be exploited with the principal task in any Multi-Task Learning (MTL) model. The disentanglement procedure works at a representation level, isolating a subspace related to the principal task, plus an arbitrary number of orthogonal subspaces. In the most disentangled subspaces, we generate the additional classification tasks through a clustering procedure, and the associated labels become their representatives. Subsequently, the original data, the labels associated with the principal task, and the newly discovered ones can be fed into any MTL framework. Extensive validation on both synthetic and real data, along with various ablation studies, demonstrate promising results, revealing the potential in what has been, so far, an unexplored connection between learning disentangled representations and MTL.


    title={Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learning},
    author={Skenderi, Geri and Capogrosso, Luigi and Toaiari, Andrea and Denitto, Matteo and Fummi, Franco and Melzi, Simone and Cristani, Marco},
    journal={arXiv preprint arXiv:2310.09278},