-
-
yes for new jobs on
@googlecloud but we still have hundreds of scalding jobs that'll run on@hadoop with Google Cloud Dataproc -
Did you guys do any cost analysis of running the same job using Scalding on Dataproc vs Dataflow?
-
No but probably hard since Dataflow has auto-scaling and we offload a lot to BigQuery
-
we use BQ for simple transforms & columnar ops (cheaper/faster) and Scio for custom logic
-
Got it. scio looks pretty cool btw.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.