Dan Luu

e.g., http://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/14 …, http://www.cs.rutgers.edu/~ricardob/papers/asplos15-fm.pdf …, etc.

I'd be really interested to know what the bottleneck was

@b0rk 50GB fits in RAM. Yes, I'd claim it should be possible to do better. (Don't know anything about Lucene, though)

sounds terrible, wonder how clucene and Xapian would compare

hmmm, I could have sworn I got sub 1ms per query on 10GbE w/ http://PyParallel.org  (see Parallelopedia code example on site).

possible by reducing term dictionary size. I bet the term dict for Wikipedia is large

but that's suspiciously slow. How are you querying?

and you need to reserve a good bit of the systems ram for FS caching

(i.e. Don't give it all to the JVM)