Conversation

8:42 PM · Feb 10, 2022Twitter for iPad

· Feb 10

the word2vec manifold must be folded in on itself in weird ways by words with multiple meanings, i was i guess naively expecting it to be a lot "flatter"

Show this thread

Likes

Replying to

afaict the notion of semantic distance word2vec is trying to capture must satisfy the triangle inequality. so it looks to me like there are words that are “artificially too close” because of these paths through ambiguous words. does that sound right?

looks like the keyword i want here is “word sense embeddings” or something like that, nice jair.org/index.php/jair

nice, here it is In The Literature: “semantically unrelated words that are similar to different senses of a word are pulled towards each other in the semantic space”

this circle of ideas around exploring the structure of "wordspace" is interesting to me among other things b/c everyone already has their own intuitive sense of "wordspace" and of semantic distance between words, etc. that's why you can play games like french toast w/ anyone

so any kind of technical attempt to machine-learn something about the structure of wordspace, like word2vec, can be compared and contrasted to people's existing conceptions of wordspace. that's good shit right there

Replying to

I think the problem here is that the two "bus" are really different words, and so we should distinguish "semantic distance" from "syntactical distance"

Lúcás Meier

@cronokirby

So then you have an interplay between two metrics

Replying to

I would think that words with distinct semantic meanings should be treated as effectively distinct words, for such a metric. Anything else is conflating syntactic and semantic levels.

Mike English

@englishm_

Understanding word vectors: A tutorial for "Reading and Writing Electronic Text," a class I teach...

I found a good explanation from

@aparrish

of how word vectors make the simplifying assumption that similarity of distribution implies similarity of meaning. She's done a bunch of really cool work in this space, in general.

gist.github.com

Understanding word vectors: A tutorial for "Reading and Writing Electronic Text," a class I teach at ITP. (Python 2.7) Code examples released under CC0 https://creativecommons.org...

Warty (110/200 IQ points)

@warty_dog

Replying to