I want to write a blog post called "Hadoop is slow" but I still don't really understand if/why Hadoop is slow. halp.https://gist.github.com/jvns/9381521489a99e888c0e …
-
-
@b0rk ... -> reduce fn -> write to HDFS (local to node + replicated). Then repeat for any number of further mapreduce steps in the job. -
@avibryant I will ask you about this in more than 140 characters! -
@b0rk also keep in mind that you are basically redeploying your application (copying the jars out, starting new JVMs) every time you do this - 1 more reply
New conversation -
-
-
@avibryant@b0rk shuffle data is written to disk on reducer only if reqd. - i.e. spill occurs only iff under memory pressureThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.