@avibryant you ever run into parquet-cascading failing to import because it's unable to find the cascading.scheme.Scheme class?
-
-
Replying to @sorenmacbeth
@sorenmacbeth no, but I have run into strange classpath problems because CDH includes an old version of parquet-mr and I want a later build.2 replies 0 retweets 0 likes -
Replying to @avibryant
@sorenmacbeth we always have to do HADOOP_USER_CLASSPATH_FIRST=true to get around that.1 reply 0 retweets 0 likes -
Replying to @avibryant
@avibryant yep, that fixed it. thanks again. hail satan, etc etc.1 reply 0 retweets 2 likes -
Replying to @sorenmacbeth
@sorenmacbeth@avibryant I have no idea why that is not set to true by default. I almost always have to set it.2 replies 0 retweets 0 likes -
Replying to @amcclosky
@amcclosky@avibryant took me a minute to figure out I also had to set HADOOP_CLASSPATH to my uberjar as well to get it work.3 replies 0 retweets 0 likes -
Replying to @sorenmacbeth
@sorenmacbeth@amcclosky yeah, I think the fact that you always have to set those things is someone's sick idea of job security.2 replies 0 retweets 2 likes -
Replying to @avibryant
@sorenmacbeth@amcclosky speaking of which: ever since expanding our cluster, datanodes now occasionally freak out with constant OOME (1/2)1 reply 0 retweets 0 likes -
Replying to @avibryant
@sorenmacbeth@amcclosky specifically, OutOfMemoryError: Direct buffer memory. Restarting fixes them for a while. Ever seen this? Thoughts?4 replies 0 retweets 0 likes -
Replying to @avibryant
@avibryant@sorenmacbeth@amcclosky does your HDFS client process use DBB otherwise? Try increasing -XX:MaxDBB for the process.1 reply 0 retweets 0 likes
@xefyr @sorenmacbeth @amcclosky yeah, we've tried that, though my understanding is that it defaults to be the same as -Xmx anyway.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.