Lazy Twitter: is there a Hadoop S3 OutputFormat (not emulation using s3://)? Want to write list of output keys into a SUCCESS key.
-
-
@avibryant@0x138 you’re just trying to put me in an infinite loop, right?Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
@avibryant@posco Is there not a consistency issue doing it there? Unless you expand at airflow to every input file + size/no globs? -
-
@avibryant At some point@posco, are you building metacat/DAL, but just keeping ground truth self-contained on S3 but loosing search/gui?
End of conversation
New conversation -
-
-
@avibryant@posco nice prop of airflow would be support for pig/other processing engines would be easier since the IF's could be dumb/diffThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.