Conversation

QC (in Berlin 7/29-???)

@QiaochuYuan

·

Feb 10

given that there exist words with multiple distinct meanings, should “semantic distance” between words even satisfy the triangle inequality? e.g. intuitively “car” is close to “bus” (noun) and “bus” is close to “wait” (verb), but “car” isn’t close to “wait”

Quote Tweet

QC (in Berlin 7/29-???)

@QiaochuYuan

· Feb 10

the word2vec manifold must be folded in on itself in weird ways by words with multiple meanings, i was i guess naively expecting it to be a lot "flatter"

Show this thread

5

17

QC (in Berlin 7/29-???)

@QiaochuYuan

afaict the notion of semantic distance word2vec is trying to capture must satisfy the triangle inequality. so it looks to me like there are words that are “artificially too close” because of these paths through ambiguous words. does that sound right?

8:42 PM · Feb 10, 2022·Twitter for iPad

6

Likes

QC (in Berlin 7/29-???)

@QiaochuYuan

·

Feb 10

Replying to

@QiaochuYuan

looks like the keyword i want here is “word sense embeddings” or something like that, nice https://jair.org/index.php/jair/article/view/11259…

1

4

QC (in Berlin 7/29-???)

@QiaochuYuan

·

Feb 10

nice, here it is In The Literature: “semantically unrelated words that are similar to different senses of a word are pulled towards each other in the semantic space”

2

1

3

QC (in Berlin 7/29-???)

@QiaochuYuan

·

Feb 10

this circle of ideas around exploring the structure of "wordspace" is interesting to me among other things b/c everyone already has their own intuitive sense of "wordspace" and of semantic distance between words, etc. that's why you can play games like french toast w/ anyone

2

QC (in Berlin 7/29-???)

@QiaochuYuan

·

Feb 10

so any kind of technical attempt to machine-learn something about the structure of wordspace, like word2vec, can be compared and contrasted to people's existing conceptions of wordspace. that's good shit right there