@avibryant Can you just make a fuse filesystem for the hdfs, then redo can treat it like normal files+timestamps?
-
-
-
@apenwarr might work ok for outputs+timestamps, but poorly, I suspect, for the SQLite dependency data. -
@avibryant You could just leave the sqlite database on a local filesystem. Central dependency management server is fastest anyway. -
.
@apenwarr fair enough. Project root w/main redo files/scripts is local FS on central server, mount HDFS as FUSE below that, cron rebuilds.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.