Update nim syntax to nim 2.0+

[5BNeumann]

This adds support for multi-line comments/strings, and fix some mistakes for the single line comments/string. (eg. "#" was considered a comment and not a string which contains a #)
I borrowed some of the logic from the C syntax.

[Andriamanitra]

I think it would be nice touch to add raw string literals too while we are at it.

[Andriamanitra]

This doesn't seem to be quite right.

The ending of the string literal is defined by the pattern """[^"], so this:
""""long string within quotes""""
Produces:
"long string within quotes"

https://nim-lang.org/docs/manual.html#lexical-analysis-triple-quoted-string-literals

[5BNeumann]

This is how the python3 syntax handles multi-line comments and it works pretty much how it should with the new changes. (""""long string within quotes"""" does give the expected result which was not the case prior to the newest commit.)

[Andriamanitra]

Python's triple quoted strings are different. In nim """"x"""" is a string containing "x", in Python it's a string containing "x followed by an unpaired double quotation mark which starts another string.

I think changing it to end: "\"\"\"+" should work. You also need to put the rule for """-strings before the "-strings, otherwise """ is seen as an empty string ("") and a start of new string.

[Andriamanitra]

According to the docs single quotes are used for char literals (max 1 byte), not strings.

https://nim-lang.org/docs/manual.html#lexical-analysis-character-literals

[Andriamanitra]

I'm not sold on making the strings end at $ when you forget the closing token. I think it's easier to notice that you forgot a closing quote if the string just continues to next line.

[Andriamanitra]

Python's triple quoted strings are different. In nim """"x"""" is a string containing "x", in Python it's a string containing "x followed by an unpaired double quotation mark which starts another string.

I think changing it to end: "\"\"\"+" should work. You also need to put the rule for """-strings before the "-strings, otherwise """ is seen as an empty string ("") and a start of new string.

[Andriamanitra]

According to the docs generalized raw string literals can have any identifier before the opening quotation mark (without white space in between), and they can also be triple quoted. They also don't interpret escape sequences, the skip rule should be skip: '""'.

[Andriamanitra]

\p and \u are not allowed in character literals because they may not fit in one byte.

[5BNeumann]

I'm not sold on making the strings end at $ when you forget the closing token. I think it's easier to notice that you forgot a closing quote if the string just continues to next line.

I took this from the C syntax, I'd rather not have this telltale that the closing quote was forgotten than having the whole file blinking in string color when typing an escaped quote somewhere in a string but that's more of a personal opinion.

[Andriamanitra]

The parts you've implemented seem to work fine now. The char highlighting has similar issue as the one we encountered in #4236 (comment) (if an escape sequence inside a char literal is just the right length it seemingly randomly changes color to constant.string instead of constant.specialChar), but that's a bug in the highlighter that should get fixed eventually.

let a = '\0123456'
let b = '\01234567'
let c = '\012345678'

---

However while testing the syntax more extensively with some bits of Nim code I found online I noticed that it still has some flaws that I think we should fix while we are at it:

• Pragmas are highlighted everywhere. I think we should move the highlighting for pragmas into a region delimited by {. and .}.
• Backticks can be used to make identifiers out of words that are keywords (eg. `var`) but we currently only highlight the backticks themselves as special. They should rather be another region rule (although I'm not sure if identifier, special, or default would be the most apt).
• , and ; are highlighted as special which doesn't match what we do with similar punctuation in other languages, for example in Python and C they are statement.

let `var`: string = "hello\nworld"
for idx, line in splitLines(`var`):
  echo(line); echo(line)

[5BNeumann]

I think backticks should use special, its not a very exploited color in this syntax and it pops out a lot more this way since all of their uses are pretty special cases (using a reserved keyword as identifier and declaring an operator).
I am currently working on the pragmas and I'm trying to get them closer to the official vscode syntax (as in, all identifiers in a pragma are highlighted, builtin or not), however I am struggling with the syntax engine to get it to end properly the statement boundary of the last pragma. I feel like my current results using this rule:

    - special:
        start: "\\{\\."
        end: "\\}"
        rules:
            - statement:
                start: "(, *|^\\w)"
                end: "[:,.]"

On this code:

proc foo(bar: cstring) {.importc: "foo", header: "buz.h", async.}

Are pretty decent but I'd like to avoid the ending `.` getting colored in statement instead of special. If you think its good enough I'll probably commit this version, otherwise I'll fallback to using a pre-defined list of identifiers in pragmas.

As a sidenote, I freshened the keywords list since quite a few of them weren't listed/got removed in nim 2.0+.

[Andriamanitra]

Regions inside regions tend to cause highlighting glitches so I think it's better to avoid them in the built-in syntaxes. I would go with the pre-defined list.