3 failing keyword tests

[philderbeast]

I've converted klister to use updo¹. I have it compiling with ghc-9.8.2. Before I did the conversion I found that 3 tests failed (both with stack test and with cabal test):

$ stack test
...

    keyword-used-incorrectly:                          FAIL (0.02s)
      Test output was different from 'examples/failing-examples/keyword-used-incorrectly.golden'. Output of ["diff","-u","examples/failing-examples/keyword-used-incorrectly.golden","/tmp/keyword-used-incorrectly126436-63.actual"]:
      --- examples/failing-examples/keyword-used-incorrectly.golden	2024-03-27 08:09:02.468849596 -0400
      +++ /tmp/keyword-used-incorrectly126436-63.actual	2024-03-27 09:56:54.292403439 -0400
      @@ -1,2 +1,4 @@
      -Syntax error from macro:
      -    #[keyword.kl:9.22-9.64]<(Keyword my-then cannot appear on its own)>
      +Module not found: keyword-test.kl
      +Searched in directory at /.../klister/examples/failing-examples
      +And on $KLISTERPATH:
      +
      
      Use -p '/keyword-used-incorrectly/' to rerun this test only.
    wrong-keyword:                                     FAIL (0.01s)
      Test output was different from 'examples/failing-examples/wrong-keyword.golden'. Output of ["diff","-u","examples/failing-examples/wrong-keyword.golden","/tmp/wrong-keyword126436-64.actual"]:
      --- examples/failing-examples/wrong-keyword.golden	2024-03-27 08:09:02.468849596 -0400
      +++ /tmp/wrong-keyword126436-64.actual	2024-03-27 09:56:54.308403730 -0400
      @@ -1,2 +1,4 @@
      -Syntax error from macro:
      -    #[keyword-test.kl:19.26-19.51]<(my-if: expected my-then)>
      +Module not found: keyword-test.kl
      +Searched in directory at /.../klister/examples/failing-examples
      +And on $KLISTERPATH:
      +
      
      Use -p '/wrong-keyword/' to rerun this test only.
    bound-vs-free:                                     FAIL (0.02s)
      Test output was different from 'examples/failing-examples/bound-vs-free.golden'. Output of ["diff","-u","examples/failing-examples/bound-vs-free.golden","/tmp/bound-vs-free126436-65.actual"]:
      --- examples/failing-examples/bound-vs-free.golden	2024-03-27 08:09:02.468849596 -0400
      +++ /tmp/bound-vs-free126436-65.actual	2024-03-27 09:56:54.328404096 -0400
      @@ -1,2 +1,4 @@
      -Syntax error from macro:
      -    #[bound-vs-free.kl:20.39-20.67]<"variables must be distinct">
      +Module not found: do.kl
      +Searched in directory at /.../klister/examples/failing-examples
      +And on $KLISTERPATH:
      +
      
      Use -p '/bound-vs-free/' to rerun this test only.
...

I also noticed some compiler warnings;

• -Wtype-equality-requires-operators
• -Wincomplete-uni-patterns

Footnotes

1. @gelisam Updo is the project I mentioned at your "Build Systems à la Carte" talk. ↩

[doyougnu]

This looks like an issue with mapping examples on the KLISTERPATH environment variable. Each error is about a missing .kl file. In the nix flake, I get around this here. Perhaps there must be some equivalent in up-do? I'm not familiar enough with the library to speak on it.

[gelisam]

In the nix flake, I get around this here.

And in CI, KLISTERPATH is set here.

Rather than set KLISTERPATH three times (in CI, nix, and updo), maybe the test itself should set this environment? variable 🤔

[philderbeast]

Thanks, this works for me (using up-do/klister);

$ KLISTERPATH=examples stack test
Warning: WARNING: This stack project is generated.
klister> test (suite: klister-tests)
...
All 132 tests passed (14.15s)
klister> Test suite klister-tests passed

$ KLISTERPATH=examples cabal test
Build profile: -w ghc-9.2.8 -O1
...
All 132 tests passed (13.51s)
Test suite klister-tests: PASS

[philderbeast]

Are relative imports possible? The unfound module is up one directory.

klister/examples/failing-examples/bound-vs-free.kl

Line 5 in 5840300

(import (shift "do.kl" 1))

$ tree -P 'do.kl' --prune
.
└── examples
    └── do.kl

$ git diff
diff --git a/tests/Golden.hs b/tests/Golden.hs
index f6b8df1..9f8c761 100644
--- a/tests/Golden.hs
+++ b/tests/Golden.hs
@@ -7,6 +7,7 @@
 
 module Golden where
 
+import Debug.Trace
 import Control.Lens hiding (argument)
 import Control.Monad.Catch (bracket)
 import Control.Monad
@@ -37,7 +38,7 @@ mkGoldenTests :: IO TestTree
 mkGoldenTests = do
   klisterFiles <- findByExtension [".kl"] "examples"
   return $ testGroup "Golden tests"
-    [ let actual = execWriterT $ runExamples file
+    [ let actual = execWriterT $ runExamples (traceShowId file)
       in goldenVsStringDiff testName diffCmd goldenFile (TE.encodeUtf8 <$> actual)
     | file <- klisterFiles
     , let testName = takeBaseName file

$ cabal run klister-tests -- -p '/bound-vs-free/'
Warning: this is a debug build of cabal-install with assertions enabled.
"examples/failing-examples/bound-vs-free.kl"
All tests
  Golden tests
    bound-vs-free: FAIL (0.03s)
      Test output was different from 'examples/failing-examples/bound-vs-free.golden'. Output of ["diff","-u","examples/failing-examples/bound-vs-free.golden","/tmp/bound-vs-free215146-0.actual"]:
      --- examples/failing-examples/bound-vs-free.golden	2024-03-26 09:25:41.912135442 -0400
      +++ /tmp/bound-vs-free215146-0.actual	2024-04-25 10:48:16.586209998 -0400
      @@ -1,2 +1,4 @@
      -Syntax error from macro:
      -    #[bound-vs-free.kl:20.39-20.67]<"variables must be distinct">
      +Module not found: do.kl
      +Searched in directory at /.../examples/failing-examples
      +And on $KLISTERPATH:
      +

1 out of 1 tests failed (0.03s)