@HenryR hey @danielhfrank and I have a question about Impala performance on unbalanced partitions. where's the best place to ask?
@b0rk @HenryR @danielhfrank my own mental model of impala is: no, partitions can never make things worse.
-
-
@avibryant@HenryR@danielhfrank we're trying to figure out if it's worth it to break the data up into more evenly-sized partitions -
@avibryant@HenryR@danielhfrank or if it's okay to leave it being unbalanced -
@b0rk@avibryant@danielhfrank depends - how selective are your queries wrt partition key? do your queries mostly hit the large partition? -
@b0rk@danielhfrank as@avibryant says, for reasonable numbers of partitions, it's not harmful to partition your table. [cont] -
@b0rk@danielhfrank@avibryant but you may not realise all the benefits if you usually select in most of the data.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.