cc @slightlylate @__apf__ and other web superheroes :) It's getting used at big JS teams like Uber , for a lot of our GPU work in @graphistry , and even some banks. These APIs should help open up the tent to the wider browser+node community :)
-
Show this thread
-
Replying to @lmeyerov @ApacheArrow and
We've often discussed an easier/better way to load data (particularly CSV) into the browser. Does arrow assume pre-parsed data?
1 reply 0 retweets 2 likes -
Replying to @slightlylate @ApacheArrow and
Yes and no. The core is the in-memory format. E.g., how to represent a table of nested json. Surrounding IO ecosystem has helpers for stuff like csv, e.g., in node, we use a GPU-accelerated csv parser and push around results via Arrow. Think protobuf all grown up.
1 reply 0 retweets 0 likes -
Replying to @lmeyerov @slightlylate and
*GPU accelerated csv->arrow parser. And others have optimized csv->arrow CPU parsers (guessing dremio + modin are doing coolest there.)
1 reply 0 retweets 0 likes -
Replying to @lmeyerov @slightlylate and
So it gives a standard target for performance nuts, and for the browser itself, ways to do immutable data + parallel + streaming data. We do it for streaming server->webgl, for example, and others have done w/ workers.
1 reply 0 retweets 1 like -
Replying to @lmeyerov @slightlylate and
IMO fine to not worry about browser support for now. But if an app is doing something like loading in a data table for a big grid w/ long-scroll, or showing a big chart with lots of rich dots, those components may be better as arrow, both for speed + later interop (similar to TS)
2 replies 0 retweets 0 likes -
-
Replying to @BrendanEich @slightlylate and
Yes, the thing I'm watching is webgpu acceptance of spir-v, and how much of it. Exposing primitives like memory fences via spir-v is what takes gpu-on-the-web into the early 2010s. Definitely frustrating as we're in 2020 gpu tech now, and graphics is a tiny piece of GPUs.
1 reply 0 retweets 1 like -
Replying to @lmeyerov @slightlylate and
I know :-|. Talk of GL_compute_shader into WebGL from many years ago went nowhere.
2 replies 0 retweets 3 likes -
Replying to @BrendanEich @lmeyerov and
I’ve been surprised how far I’ve gotten just abusing the rasterization pipeline and/or transform feedback to do compute things. e.g. my on-GPU cloth simulation is all abusing the blending hardware. I’ve been thinking maybe someone should polyfill compute shader using TF.
2 replies 1 retweet 5 likes
(Yes, I know transform feedback doesn’t cover everything compute shader can do—but it’s surely better than twiddling our thumbs waiting for the outcome of the squabbling over SPIR-V.)
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.