@posco @peterseibel you know what would be cool? For typed Scalding jobs, graphviz output of the "type flow" from inputs to outputs.
@daashu @posco @peterseibel any thoughts about implementation? Seems like type erasure makes this painful. Cache all the manifests?
-
-
@avibryant@daashu@peterseibel compiler plugin or macro. The latter probably. -
@posco@avibryant@daashu@peterseibel Yeah it would be quite easy to write a graph { ... } macro if the job was literally defined inside. -
@d6@avibryant@daashu@peterseibel I think that is a find constraint for this kind of debug out. -
@posco@d6@daashu@peterseibel hm, really? Means it wouldn't work when your job used, say, the DISCO stuff, or other similar libraries. -
@posco@d6@daashu@peterseibel ... I feel like we could get a lot out of storing the Manifest chain in FlatMapFn, + equivalent in Grouped -
@avibryant@d6@daashu@peterseibel as@0x138 reminds me: Manifests are invariant, which will complicate the variance of this.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.