Formatter introduces newlines in awkward places with comments

[xeger]

👓 What did you see?

When a scenario block has a comment block above it, the formatter add a newline between the comments and the block, instead of adding it above the comments.

As a result, the comments look like they're associated with the previous Gherkin statement.

Feature: comments
  # This is a leading comment of the scenario

  Scenario: leading comments

✅ What did you expect to see?

When the formatter adds newlines between Gherkin statements, it should be mindful of their associated comments blocks.

Feature: comments

  # This is a leading comment of the scenario
  Scenario: leading comments

There is a chance for semantic ambiguity, because comments could in theory be intended as "trailing" comments.

Feature: comments
# This is a trailing comment associated with the feature.

  # This is a leading comment of the scenario
  Scenario: leading comments

The only way we can tell that line 2 is a trailing feature comment is by examining the indentation of the # character.

There's a similar chance for ambiguity if steps within a container are commented out (which is a common occurrence during debugging).

Feature: comments

  Scenario: commented-out steps
    Given something
    # When I do something
    Then I see something

However, commented-out steps are probably not an issue because the formatter never inserts newlines between a step container and its sub-blocks.

📦 Which tool/library version are you using?

Version 8.0.5 of this package.

🔬 How could we reproduce it?

See the groupedComments branch for a failing test case.

📚 Any additional context?

I understand that the pretty() function in gherkin-utils is considered non-canonical and that we should be using formatters that operate on the JSON representation of gherkin ASTs.

Before putting too much work into fixing this bug, we should consider whether there is a better formatter available in the Cucumber Collective ecosystem, and/or whether to reimplement this formatter using the newer parsing utilities.