-
-
Replying to @o_guest @usethespacebar
sklearn.metrics.pairwise.pairwise_distances has n_jobs parameter, which does precisely what you need, imho. (just back from vacation, slow)
1 reply 0 retweets 1 like -
Replying to @dimpase @usethespacebar
no worries, I think it can't be what I need because it runs out of memory?
1 reply 0 retweets 0 likes -
Replying to @o_guest @usethespacebar
do you mean it does this with any value of n_jobs? or only some?
1 reply 0 retweets 1 like -
Replying to @dimpase @usethespacebar
Lemme try a bunch of n_jobs values and get back to you
1 reply 0 retweets 0 likes -
I think the problem with this function if I recall correctly is it tried to load the distance matrix in memory and I have 50mil data points!
2 replies 0 retweets 0 likes -
Replying to @o_guest @usethespacebar
aren't you trying to build that matrix (~20GB of data, assuming 8 bytes per entry) anyway? What info do you need to get from it?
1 reply 0 retweets 1 like -
Replying to @dimpase @usethespacebar
I need the pairwise distances ideally with a customisable distance metric.
1 reply 0 retweets 0 likes -
Do you want to switch to another medium for conversation that allows more characters?
1 reply 0 retweets 0 likes -
Replying to @o_guest @usethespacebar
we can surely discuss in your github repo issues
1 reply 0 retweets 1 like
Thank you so much!
-
-
Replying to @o_guest @usethespacebar0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.