What I do in Kitten is have a lexing pass that generates tokens annotated with source spans, followed by a bracketing pass that desugars them into explicitly delimited blocks using those spans—the concept of an “indent/dedent token” seems obvious but doesn’t work well in practice
Conversation
(Because you’re attempting to deduce the bracketing as if it were already present, in addition to the basic mistakes of mixing concerns)
1
2
Hmm. I mentioned the other things just to avoid being overly specific, but my overly specific problem was with the "newlines terminate statements (but are also allowed elsewhere)" thing -- does your advice translate to that case?
1
Yeah—the specifics depend on what you want to do in particular, but for example I allow newlines within parentheses/square braces, or outside a layout context, and they don’t cause statement terminators to be inserted there
1
2
Here’s the latest code for my implementation of this technique, if you’re curious: github.com/evincarofautum
1
1
Yeah, I believe that this is what Gluon and Haskell do too.
1
I've been doing some work in fsharp recently and .. whoo boy I did not expect to have so many feelings against whitespace sensitivity boil up all of a sudden. But for real: it does not go well in expression languages where you're doing a lot of nesting & continuation lines.
1
6
I’m almost considering not doing whitespace sensitivity in Pikelet. 🤔
3
1
It's weird because it did not bother me in haskell (nor python) but it's like fingernails on blackboard in fsharp. Might just be the years I've spent doing ocaml which is not. But also I think there's much more rightward drift with pervasive keyword args and module decl nesting.
3
2
(it wouldn't be an ocaml variant if it didn't offer multiple syntax skins of course: there's a "verbose" mode in which it accepts the delimeter tokens as guidance / overrides the whitespace, though all the auto-formatting tools strip it out and generally nobody seems to use it..)
1
1
Yeah, you can also write the delimiters explicitly in Haskell as well. Not sure what auto formatters do with them though.
Perhaps if you were rigorous about making a fully-supported delimiter mode you could flip into when you wanted to do a bunch of whitespace-disruptive editing / moving stuff around, and then auto-reformat and flip back out of at the end of the moves, it .. might be ok?
1
1
I mean even if you fix the editing problem (eg. "adopt sexps") expression languages still have a legibility problem ("what existing expr just closed" / "what does the next arg attach to") that I think it's hard to fix overall. I'm no fan of names, but sometimes they're best.
2
Show replies