This is more or less the occlusion version of Deep InfoMax with an intra-image contrastive loss rather than an inter-image one (negative sample patches are drawn from different locations of the same image rather than patches from other images) + fine tuning. 
-
-
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Do you think a two-dimensional position embedding is better? And looking forward to experiments on much larger datasets.
-
By two-dimensional position embedding, I mean use 2D positions like (2, 1) for row 2 and column 1 instead of a single index 4. The usage might be: add position embeddings for row 2 and column 1 together as the final position embedding.
Kraj razgovora
Novi razgovor -
-
Tweet je nedostupan.
-
-
Do you think this way of pretraining is better than pretraining in a way similar to the unsupervised sequential modeling of Image Transformer or Sparse Transformer?
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
This is awesome!
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.