Im kind of a bit afraid of going too deep into building my backend before I can figure out how to incrementalise stuff... 😰
Conversation
Here is what I’ve collected so far - not sure if you have any more ideas?
2
9
some keywords: "incremental computation" (eg. github.com/Adapton/fungi-), "view maintenance" (eg. arxiv.org/pdf/1403.6968.), "differential dataflow" (eg. blog.acolyer.org/2015/06/17/dif), "build systems" (eg. microsoft.com/en-us/research), "reference attribute grammars" (lucris.lub.lu.se/ws/files/33120)
3
22
Wait. Has any of those techniques (beyond build systems) been demonstrated in compilers?
1
We incremental computation researchers cannot say it to get published, but the basic assumption for readers seems that these techniques aren’t really black boxes and writing any essentially new program with them is a new paper, unless your program is already well *parallelized*!
3
By that do you mean that it’s hard to transfer the techniques between programs? Or hard to get reliable performance improvements?
1
See other replies: it’s hard to get performance improvements. You can write highly parallel reusable primitives (such as folds that require associativity and whose computation tree is balanced, or many query languages ops), then use them for your program.
1
1
Let me admit I did a PhD on this and realized the issue halfway, so there might be a “disgruntled failed employee” vibe to this (and I haven’t studied the type systems to guide authors of incremental programs).
3
Not to worry! I value the perspective!
2
1
Still pretty opaque from an outsider perspective though. Hard to know what direction to take to get me from a sequential scan over my ast, to something that can work as a language server/incremental compiler ☹️ - hard to know what to invest time in experimenting/learning.
2
I feel like I have a slightly better grasp on the dependent types tbh (but I might be fooling myself there, haha)
‘Grasp’ as in I know what I don’t know right now, and have an idea of what order to learn it