@cwensel the @cascading 2.5 release notes claim to support YARN, but all the artifacts are for hadoop2-mr1. Is there any MR2 support?
@cwensel "MR1 on YARN"? I don't think that combination is possible. I thought YARN+MR2, together, replaced MR1. Am I missing something?
-
-
@avibryant twitter team has been running this for a couple months now. YARN is just a resource manager. abstracts away MR1/2/Tez/MPI/etc -
@cwensel ok, I'd love to hear more about this setup.@squarecog any pointers? -
@avibryant@cwensel I have no idea on mr1 vs mr2, but we are running cascading/scalding on MR in Yarn in production. -
@squarecog interesting. I *think* that's quite an unusual setup, and would love to hear more if you can point me to the right person. -
@avibryant we are running MR2 on yarn. I guess hadop2-mr1 is actually that? -
@squarecog MR2 on YARN makes sense. But Cascading won't run on MR2. So I'm still confused. -
@avibryant@squarecog I seems to me you're failing to take into account the possibly of dark magick playing a role here. -
@sorenmacbeth He writes stuff in Scala all the time. Of course he's factored in dark magic.@avibryant - 2 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.