Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify illegal $ tokens in string interpolators #196

Merged
merged 3 commits into from
Feb 24, 2021

Conversation

nicolasstucki
Copy link
Contributor

Fixes #195

Scala 2 stops parsing after one of these failures. It does not matter how we highlight the rest.

Scala 3 continues parsing assuming the string has been closed. Highlighting the following code as if the string was closed helps to understand the following parsing errors.

Screenshot 2021-01-29 at 15 46 26

Fixes scala#195

Scala 2 stops parsing after one of these failures. It does not matter how we highlight the rest.

Scala 3 continues parsing assuming the string has been closed. Highlighting the following code as if the string was closed helps to understand the following parsing errors.
@nicolasstucki
Copy link
Contributor Author

The second commit provides a nicer user experience as the $ is not highlighted in red before the rest is typed.

Screenshot 2021-01-29 at 15 56 06

@MaximeKjaer
Copy link
Contributor

I see that Scala 3 considers the string closed when there's an illegal $ token to be able to parse the rest. However, by working on the Pygments highlighter in parallel to this highlighter, I've come to the following insight: parsing/highlighting in real-time (like this highlighter does), and parsing a complete piece of code (like Pygments or the Scala compiler do) are different tasks, and can therefore treat errors differently.

To me, when a highlighter highlights more than I expect as a string, it's a very clear indication that I haven't closed the string properly. I think that's actually the best user experience for a case like this: it indicates very clearly the mistake that I made: the string is unclosed, and that's why I'm seeing the rest highlighted as a string. As soon as I fix that, I'll see the other parsing errors, if there are any. The feedback loop is very short here.

When running the compiler, or building a webpage where Pygments is highlighting code, the feedback loop is longer, so it may make sense to put more information about errors at once, but this sometimes comes at the cost of the clarity of the errors.

In the example with s"$; val a = "", there's a mistake, but no indication from the highlighter about it; I don't think that's the best behavior. Instead, I would suggest that we highlight illegal $ tokens as follows:

  • The $ is treated as an interpolation marker
  • The character after $ is treated as an illegal character
  • If the string is unclosed, we keep highlighting the end of the line as a string. If it's possible to mark the \n as an error, that would be nice (does VS Code show something if we do that?)

This can be implemented as a very simple rule, something along the lines of (\\$)(.) as the final rule for string interpolation, and possibly another rule for \\n as an illegal token.

Also fix missing rules for `"""` and `raw`
@nicolasstucki
Copy link
Contributor Author

Now the highlighting is
Screenshot 2021-02-05 at 15 51 54

Copy link
Contributor

@MaximeKjaer MaximeKjaer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the slow reply on this. These new changes look very good, let's get them merged.

@MaximeKjaer MaximeKjaer merged commit fb98da1 into scala:master Feb 24, 2021
@nicolasstucki nicolasstucki deleted the fix-#195 branch February 25, 2021 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Illegal $ in string interpolator could be highlighted as an illegal token
2 participants