[GH-ISSUE #42] Issues with the full specification and pest code #15

New issue

Open

opened 2026-05-22 22:05:09 +01:00 by JakeStanger · 2 comments

JakeStanger commented

2026-05-22 22:05:09 +01:00

Owner

Originally created by @ghost on GitHub (Apr 16, 2025).
Original GitHub issue: https://github.com/corn-config/corn/issues/42

To start of with, I wanted to say I really like the idea of corn. I've long been frustrated with JSON config files. Recently I've been writing a corn parser with chumsky as a learning experience. As such I have been reading over the full spec a lot and have noticed some issues that I think should be addressed. I haven't experimented with the rust code directly but I have used the online pest editor to see how the pest code breaks down various examples.

Issues

The formal language grammar linked here https://cornlang.dev/spec/#grammar goes nowhere. The link needs to be updated (and should probably point to a stable tag/branch instead of the master branch).

Here the valid form for input names is described as:

Input names always start with a dollar $ when declaring and referencing them. The first character after the dollar must be either alphabetic (a-z A-Z) or an underscore _. This can be followed by any number of alphanumeric (a-z A-Z 0-9) characters or underscores.

This is slightly wrong, or at least, poorly worded. Here is the relevant line in the pest code:

input = ${ !"\\" ~ "$" ~ (ASCII_ALPHA | "_") ~ (ASCII_ALPHANUMERIC | "_")+ }

This requires an input name to be at least two characters long. The line '... This can be followed by any number of alphanumeric ...' in the spec should instead read '... This is followed by one or more alphanumeric ...' or the pest code should be changed to make the second character optional.

In the spec for strings:

You can also include any Unicode character using an \uXXXX escape code. XXXX must be replaced with exactly 4 hexadecimal digits.

This is incorrect. I'm no Unicode guru myself but eyeballing this with only four digits you can represent about 40% of Unicode characters. For example none of these 🌐 🌞 🌝 🌸 can be represented with only four digits. Ideally you would want to support 4-6 digits, but 4-5 would be enough for all but the Supplementary Private Use Area plane - B.

Originally created by @ghost on GitHub (Apr 16, 2025). Original GitHub issue: https://github.com/corn-config/corn/issues/42 To start of with, I wanted to say I really like the idea of `corn`. I've long been frustrated with JSON config files. Recently I've been writing a `corn` parser with `chumsky` as a learning experience. As such I have been reading over the [full spec](https://cornlang.dev/spec) a **lot** and have noticed some issues that I think should be addressed. I haven't experimented with the rust code directly but I have used the online [pest editor](https://pest.rs/#editor) to see how the pest code breaks down various examples. ## Issues The formal language grammar linked here https://cornlang.dev/spec/#grammar goes nowhere. The link needs to be updated (and should probably point to a stable tag/branch instead of the master branch). --- [Here](https://cornlang.dev/spec/#inputs) the valid form for input names is described as: > Input names always start with a dollar $ when declaring and referencing them. The first character after the dollar must be either alphabetic (a-z A-Z) or an underscore _. This can be followed by any number of alphanumeric (a-z A-Z 0-9) characters or underscores. This is slightly wrong, or at least, poorly worded. Here is the relevant line in the [pest code](https://github.com/corn-config/corn/blob/master/src/grammar.pest#L87): ```pest input = ${ !"\\" ~ "$" ~ (ASCII_ALPHA | "_") ~ (ASCII_ALPHANUMERIC | "_")+ } ``` This requires an input name to be at least two characters long. The line '... This can be followed by any number of alphanumeric ...' in the spec should instead read '... This **is** followed by **one or more** alphanumeric ...' or the pest code should be changed to make the second character optional. --- In the spec for [strings](https://cornlang.dev/spec/#string): > You can also include any Unicode character using an \uXXXX escape code. XXXX must be replaced with exactly 4 hexadecimal digits. This is incorrect. I'm no Unicode guru myself but eyeballing [this](https://unicodeplus.com/plane) with only four digits you can represent about 40% of Unicode characters. For example none of these 🌐 🌞 🌝 🌸 can be represented with only four digits. Ideally you would want to support 4-6 digits, but 4-5 would be enough for all but the [Supplementary Private Use Area plane - B](https://unicodeplus.com/plane/16).

JakeStanger added the

documentation

bug

labels

2026-05-22 22:05:09 +01:00

JakeStanger commented

2026-05-22 22:05:10 +01:00

Author

Owner

@JakeStanger commented on GitHub (Apr 16, 2025):

Thanks for raising these. I'll get the link fixed soon.

There are some other small changes coming to the spec for 0.11 to fix a few holes and really tighten it up, so we'll make sure these are taken into consideration too.

Just to provide some answers for now:

The grammar is at: https://github.com/corn-config/corn/blob/master/src/grammar.pest. There's a new parser in the works so this may change, but there will always be a permalinked grammar available somewhere.
Inputs should accept single-character names. I'd consider that a bug with the current parser.
Agreed the Unicode escape needs expanding to support the wider range. Open to feedback on that one.

@JakeStanger commented on GitHub (Apr 16, 2025): Thanks for raising these. I'll get the link fixed soon. There are some other small changes coming to the spec for 0.11 to fix a few holes and really tighten it up, so we'll make sure these are taken into consideration too. Just to provide some answers for now: - The grammar is at: https://github.com/corn-config/corn/blob/master/src/grammar.pest. There's a new parser in the works so this may change, but there will always be a permalinked grammar available *somewhere*. - Inputs should accept single-character names. I'd consider that a bug with the current parser. - Agreed the Unicode escape needs expanding to support the wider range. Open to feedback on that one.

JakeStanger commented

2026-05-22 22:05:10 +01:00

Author

Owner

@rdin777 commented on GitHub (Dec 17, 2025):

Subject: Identified discrepancies between Specification and Pest grammar

"I've compared the cornlang.dev specification with the current src/grammar.pest and found a few inconsistencies:

Integer underscores: The spec allows underscores in integers (e.g., 1_000), but the current decimal_integer rule in Pest seems to only accept pure digits.

Quoted keys in paths: The spec mentions support for keys like 'foo.bar'.baz = 6. Looking at the path_seg rule, it’s unclear if the current grammar correctly handles mixed quoted and regular segments in a chain.

Ambiguity in Object Values: The spec says no punctuation is needed, but the current object_value rule might be too restrictive regarding how pairs are separated if they are on the same line without commas.

I can provide test cases for these to verify the failures. Should I focus on fixing the decimal_integer rule first?"

@rdin777 commented on GitHub (Dec 17, 2025): Subject: Identified discrepancies between Specification and Pest grammar "I've compared the cornlang.dev specification with the current src/grammar.pest and found a few inconsistencies: Integer underscores: The spec allows underscores in integers (e.g., 1_000), but the current decimal_integer rule in Pest seems to only accept pure digits. Quoted keys in paths: The spec mentions support for keys like 'foo.bar'.baz = 6. Looking at the path_seg rule, it’s unclear if the current grammar correctly handles mixed quoted and regular segments in a chain. Ambiguity in Object Values: The spec says no punctuation is needed, but the current object_value rule might be too restrictive regarding how pairs are separated if they are on the same line without commas. I can provide test cases for these to verify the failures. Should I focus on fixing the decimal_integer rule first?"

JakeStanger referenced this issue

2026-05-22 22:05:14 +01:00

[PR #15] [MERGED] feat: spread operator #25