oh I see OK lemme try again
-
-
sorry I don't get how to use combinations to get what I want into a single array — and then the function has to accept the whole array?
2 replies 0 retweets 0 likes -
Replying to @o_guest @seaandsailor
nop. In your case you actually need something like: results = mp.starmap(calculate_pairwise_distance, combinations(X,X)).
2 replies 0 retweets 0 likes -
or why not use n_jobs param of http://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.pairwise_distances.html … ?
1 reply 0 retweets 0 likes -
Replying to @draxus @seaandsailor
because I only started working using these libraries literally yesterday so I'm a n00b OK I'll try that too feels a bit anti-educational to
1 reply 0 retweets 0 likes -
be just trying random stuff without learning what I am trying exactly but I'll give that a go...
2 replies 0 retweets 0 likes -
hang on it's just another pairwise distance calculator that will certainly run out of memory... I need around 50 million data points!!
1 reply 0 retweets 0 likes -
yep memory error
1 reply 0 retweets 0 likes -
Replying to @o_guest @seaandsailor
doh! That's a lot of points
You'll need a different approach then. Not sure if I can help with that, but definitely interested #bigdata2 replies 0 retweets 1 like -
Replying to @draxus @seaandsailor
I will certainly let you know. I think it's only an edge case that has so many points but regardless the average points are high and I NEED
1 reply 0 retweets 0 likes
a function that is low memory. So time complexity can be high-ish but space complexity HAS to be low.
I really appreciate your help tho! 
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.