Adjustments to grammar for syntax highlighting (up to grammar v0.23.1)

[dhanak]

1. Fix grammar after the Julia tree-sitter grammar has been changed in Breaking: Refactor signatures and other stuff tree-sitter/tree-sitter-julia#135

2. Adjust syntax highlight of interpolations: use syntax information of interpolated code to highlight parts of it.

3. Highlight keyword argument names as symbols.

[dhanak]

Fixes #25.

[non-Jedi]

I know that we don't have tests yet, but any chance you could add a commit with one or two julia files (just pick from popular packages and try to pick files that exercise quite a few syntax features), and then run them through faceup-write-file and also commit the .faceup files?

This will at least give us a smoke test for the font-lock features. And something we can manually inspect to make sure the font-locking looks generally sane.

I'd like to get this merged so we can unbork julia-ts-mode for people installing it for the first time, but I haven't had time to sit down and learn how the treesit stuff works to give this a detailed review. If we had the .faceup files, I could probably merge with just a cursory review. Sorry for the delay, and thanks for your work here!

[dhanak]

I'll try. I'm not sure when I can get around to do it, though, so please be patient.

[atkirtland]

I tried the most recent version of dhanak/julia-ts-mode and tree-sitter-julia. While the "node type error at 177" issues were solved so I could see the light pink highlighting of variable names, I started getting "structure error at 559" issues and no other syntax highlighting showed. If tree sitter runs on the whole project at once, this could be because some files had somewhat unusual notation from Gen.jl. The first file I opened did not, however.

[dhanak]

I tried the most recent version of dhanak/julia-ts-mode and tree-sitter-julia. While the "node type error at 177" issues were solved so I could see the light pink highlighting of variable names, I started getting "structure error at 559" issues and no other syntax highlighting showed. If tree sitter runs on the whole project at once, this could be because some files had somewhat unusual notation from Gen.jl. The first file I opened did not, however.

Odd, I'm using it on a daily basis and haven't run into any issues recently. Have you tried upgrading the tree sitter language grammar? (E.g., type M-x treesit-install-language-grammar<RET>julia<RET>y.) If the issue still persists, could you include/link to a minimal expample that produces the issue?

Update: It turns out that my grammar was lagging behind. I was still using 0.23.0. When I upgraded to the newest version, I also started getting the mentioned structure errors. I'll look into it.

Update 2: There have been a bunch of breaking changes, again. I'll get around fixing it, but until then, install grammar version v0.23.0 M-x treesit-install-language-grammar<RET>julia<RET>y<RET>v0.23.0<RET><RET><RET><RET>

[dhanak]

@atkirtland, I adjusted the lisp code to fix the grammar version to 0.23.0, which it is compatible with. Please delete your grammar library by hand (/home/david/.emacs.d/tree-sitter/libtree-sitter-julia.so), and try restarting your emacs with a julia file. If you have treesit-auto, the correct grammar should install automatically. If not, calling M-x treesit-install-language-grammar should be enough (no further questions asked). Please let me know if this does the trick for you.

[atkirtland]

@dhanak Thanks! I'll try to look at this soon.

[atkirtland]

@dhanak When I run doom sync to try to use your fork now, I get the error

    > Building julia-ts-mode...
    x There was an unexpected runtime error
      Message: Symbol's value as variable is void
      Details: (treesit-language-source-alist)
      Backtrace:
        (member '(julia "https://github.com/tree-sitter/tree-sitter-...
        (if (member '(julia "https://github.com/tree-sitter/tree-sit...
        (load-with-code-conversion "/home/atkirtland/.config/emacs/....
        (load "/home/atkirtland/.config/emacs/.local/straight/build-...
        (progn (load autoloads-file nil 'nomessage))
        (if (file-exists-p autoloads-file) (progn (load autoloads-fi...
        (let ((autoloads-file (straight--autoloads-file package))) (...
        (straight--load-package-autoloads "julia-ts-mode")
        (if straight-cache-autoloads (progn (if (straight--checkhash...
        (let* ((--cl-rest-- recipe) (package (car (cdr (plist-member...
        (straight--activate-package-autoloads (:host github :repo "d...
        (if no-build nil (let ((tail (straight--get-dependencies pac...

this didn't happen last week.

[dhanak]

Hi Aaron! My guess is that it doesn't like the autoload on that variable. I'll remove it. Could you try it again, please?

…

On Wed, Dec 18, 2024 at 10:50 PM Aaron Kirtland ***@***.***> wrote: @dhanak <https://github.com/dhanak> When I try to use your fork, I get the error > Building julia-ts-mode... x There was an unexpected runtime error Message: Symbol's value as variable is void Details: (treesit-language-source-alist) Backtrace: (member '(julia "https://github.com/tree-sitter/tree-sitter-... (if (member '(julia "https://github.com/tree-sitter/tree-sit... (load-with-code-conversion "/home/atkirtland/.config/emacs/.... (load "/home/atkirtland/.config/emacs/.local/straight/build-... (progn (load autoloads-file nil 'nomessage)) (if (file-exists-p autoloads-file) (progn (load autoloads-fi... (let ((autoloads-file (straight--autoloads-file package))) (... (straight--load-package-autoloads "julia-ts-mode") (if straight-cache-autoloads (progn (if (straight--checkhash... (let* ((--cl-rest-- recipe) (package (car (cdr (plist-member... (straight--activate-package-autoloads (:host github :repo "d... (if no-build nil (let ((tail (straight--get-dependencies pac... — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANOGA3OC7TG44Q6YUVVZL2GHU2VAVCNFSM6AAAAABORYAB7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJSGMZTAMRSGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[atkirtland]

Hi dhanak, I think it's working for me now! Thanks!

[dhanak]

I finally got around to upgrading the font-lock queries to grammar version 0.23.1. Note that after an upgrade, you need to run M-x treesit-install-language-grammar RET julia RET manually in order to upgrade the grammar binary. If you know a way do do this automatically, let me know.

I also added a new source file to the faceup tests, directly from the tree-sitter-julia test corpus. This proved very useful when I patched up the queries.

[dhanak]

Obsoletes #22.