For scientists with "normal" resources (i.e., without access to industrial-scale computing): what is the largest network they have computed PageRank for? Any references?
-
Here's a way of parallelizing it on a Hadoop cluster if you want to go beyond a single machine: http://michaelnielsen.org/blog/using-mapreduce-to-compute-pagerank/ …
Looking at that Pagerank implementation, I'd be surprised if a little care in a C implementation didn't make it work for billions of pages. The approach I take using Python is almost hilariously inefficient; I might as well be doing it using a souped-up abacus.
-
-
So a big question for me is how to save the network itself, when it gets dense...
-
Look at sebastiano Vigna’s stuff. As long as you can store 16-24 bytes per node and stream edges you can compute it.
- 1 more reply
New conversation -
-
-
Sure it's more efficient than paying undergrads to click links at random and count how often they hit each page, but can any algorithm really match the true subjective experience of PageRank?
-
#neo4j has a pagerank implementation & is easily deployable on AWS. If you need to do ongoing queries, this might be worth looking at.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.