Wykaz obszarów badawczych związanych z tagiem Representation-learning:
| # | Obszar badawczy | Dziedzina naukowa |
|---|---|---|
| 1 |
computer vision (SLAM, visual search), machine learning (deep learning, generative models, continual learning), representation learning (binary descriptors).
|
|
| 2 |
Self-supervised learning (SSL) methods have shown significant potential in improving the sample efficiency of deep learning and in providing a starting point for the learning of downstream tasks. Contrastive SSL methods have become a standard pre-training approach for a range of domains such as NLP and vision. Initial results suggest that non-contrastive SSL has been able to narrow the gap to the performance levels of contrastive methods, while obviating the need for explicit construction of negative samples. The aim of this research project is two-fold: 1) investigate the extent to which contrastive and non-contrastive methods can be used in novel SSL architectures, 2) examine whether the alignment of representations in SSL can be achieved by alternative methods, such as enforcing the ability to predict an input representation from the representations of similar inputs.
|
|
| 3 |
A prevalent theme present in the contemporary representation learning approaches is to pre-train large foundation models on huge datasets. Such approaches utilize static datasets constructed at a particular point in time, which contrasts with the constantly changing and expanding nature of data available on the internet. The proposed research will explore a new paradigm where the training dataset is constructed on the fly by querying the internet, enabling efficient adaptation of representation learning models to selected target tasks. The aims of this research project include 1) design methods to query relevant training data and use it to adapt the representation learning model in a continuous manner, 2) make progress towards building self-supervised methods that given a description of a task, autonomously formulate their learning curricula, query the internet for relevant training data, and use it to iteratively optimize the model.
|