feat: SHOW/DESCRIBE/CREATE/DROP JSON STRUCTURE

[peterjumpnl]

Closes #74

Summary

• Adds full DDL support for JSON structures: SHOW JSON STRUCTURES, DESCRIBE JSON STRUCTURE, CREATE [OR REPLACE] JSON STRUCTURE, and DROP JSON STRUCTURE
• Automatically builds the element tree from a JSON snippet, matching Studio Pro's behavior (root objects and arrays, DateTime detection, reserved name handling, Wrapper elements for primitive
  arrays)
• Supports CUSTOM NAME MAP to override auto-generated ExposedNames (maps to Studio Pro's "Custom name" column)
• DESCRIBE outputs re-executable CREATE OR REPLACE MDL with the element tree as comments
• Adds json_structures catalog table for SQL querying

New MDL syntax

-- List all JSON structures
SHOW JSON STRUCTURES [IN Module];

-- Describe (re-executable output)
DESCRIBE JSON STRUCTURE Module.Name;

-- Create from JSON snippet (root object or array)
CREATE JSON STRUCTURE Module.Name SNIPPET '{"key": "value"}';

-- Multi-line with dollar quoting
CREATE JSON STRUCTURE Module.Name SNIPPET $${ "key": "value" }$$;

-- Idempotent create (preferred for AI agents)
CREATE OR REPLACE JSON STRUCTURE Module.Name SNIPPET '...';

-- Override auto-generated element names
CREATE JSON STRUCTURE Module.Name SNIPPET '...'
CUSTOM NAME MAP ('jsonKey' AS 'CustomName');

-- Delete
DROP JSON STRUCTURE Module.Name;

Pipeline coverage

Grammar → Parser (regenerated) → AST → Visitor → Executor → BSON reader/writer → Catalog → REPL autocomplete → LSP keywords → MDL Quick Reference

Verified against Studio Pro

Tested with a Mendix 11.6.3 project. Element trees generated by mxcli match Studio Pro output exactly for:
- Root object and root array snippets
- Reserved names (id → _id, type → _type)
- DateTime detection from ISO 8601 strings
- Primitive arrays (["a","b"] → Wrapper element with Value child)
- Null values → Unknown primitive type
- Custom name map detection on DESCRIBE roundtrip

[github-actions]

AI Code Review

Critical Issues

MDL Syntax Design Violation

The CUSTOM NAME MAP syntax uses AS keyword ('jsonKey' AS 'CustomName') instead of following the MDL property format guideline specified in CLAUDE.md: "uses ( Key: value, ... ) with colon separators, one per line". This violates the design skill consultation requirement and is inconsistent with existing MDL patterns like entity attributes (e.g., Name: String (length: 100)).

Fix Required: Change syntax to use colon format:

CUSTOM NAME MAP (jsonKey: 'CustomName');

or multi-line:

CUSTOM NAME MAP (
    jsonKey: 'CustomName'
);

Missing VS Code Extension Update

New LSP capabilities were added (completions for JSON STRUCTURES, STRUCTURE, CUSTOM NAME MAP) but vscode-mdl/package.json was not updated as required by the checklist: *"VS Code extension — if new LSP capabilities are added, update `vscode-mdl/package.json"**.

UTF-8 Bug in capitalizeFirstRune

The capitalizeFirstRune function in mdl/visitor/visitor_jsonstructure.go incorrectly handles multi-byte UTF-8 characters:

func capitalizeFirstRune(s string) string {
    if s == "" {
        return s
    }
    runes := []rune(s)
    runes[0] = rune(strings.ToUpper(string(runes[0]))[0]) // BUG:

---
*Automated review via OpenRouter (Nemotron Super 120B) — [workflow source](https://github.com/mendixlabs/mxcli/blob/main/.github/workflows/ai-review.yml)*

[github-actions]

AI Code Review

Okay, I need to review this pull request against the checklist provided. Let me analyze the PR systematically.

Overview of the PR

This PR adds full DDL support for JSON structures:

• SHOW JSON STRUCTURES [IN Module]
• DESCRIBE JSON STRUCTURE Module.Name
• CREATE [OR REPLACE] JSON STRUCTURE Module.Name SNIPPET '...'
• DROP JSON STRUCTURE Module.Name
• Support for CUSTOM NAME MAP to override auto-generated ExposedNames
• DESCRIBE outputs re-executable CREATE OR REPLACE MDL with element tree as comments
• Adds json_structures catalog table for SQL querying

The PR includes changes to multiple files across the project, following the pattern of other similar features.

Review Checklist Analysis

Overlap & duplication

• The PR closes issue Add support for JSON snippets #74, so it's addressing a specific requested feature
• I don't see any obvious duplication of existing functionality in the diff
• JSON Structures are a new feature, not a re-documentation of existing features
• The PR adds a new doctype test file (20-json-structure-examples.mdl) for coverage

MDL syntax design

Let me check if the new MDL syntax follows the guidelines:

1. Standard patterns: Uses CREATE/ALTER/DROP/SHOW/DESCRIBE - ✓ (CREATE [OR REPLACE], DESCRIBE, SHOW, DROP)

2. Qualified names: Uses Module.Element everywhere - ✓ (Module.Name format in all examples)

3. Property format: Uses ( Key: value, ... ) format - The examples show:

   • CREATE JSON STRUCTURE Module.Name SNIPPET '...' (no property list format here, but there is CUSTOM NAME MAP with parentheses)
   • CUSTOM NAME MAP ('jsonKey' AS 'CustomName') - This follows the property format with colons? Wait, it uses AS not colons...

   Let me check the grammar: In MDLParser.g4, I see:

   customNameMapping
       : STRING_LITERAL AS STRING_LITERAL   // 'jsonKey' AS 'CustomName'
   ;
   

   This uses "AS" not colons. Let me check if this is consistent with other MDL syntax...

   Looking at the MDL_QUICK_REFERENCE.md changes, I see:

   | Create with name map | `CREATE JSON STRUCTURE Module.Name SNIPPET '...' CUSTOM NAME MAP ('jsonKey' AS 'CustomName', ...);` | Override auto-generated ExposedNames |
   

   Other MDL statements use the (Key: value) format. For example, in the existing docs, I'd expect to see something like:

   CREATE JSON STRUCTURE Module.Name SNIPPET '...' (CUSTOM NAME MAP: ('jsonKey' AS 'CustomName'));
   

   But instead it's using:

   CREATE JSON STRUCTURE Module.Name SNIPPET '...' CUSTOM NAME MAP ('jsonKey' AS 'CustomName');
   

   This seems inconsistent with the guideline that says "Property format — uses ( Key: value, ... ) with colon separators, one per line".

   Let me check if there are precedents for this pattern in existing MDL...

   Looking at the grammar for other statements, let me check if there are similar constructs...

   Actually, looking at the visitor_jsonstructure.go, I see:

   if len(customMappings) > 0 {
       fmt.Fprintf(e.output, "\n  CUSTOM NAME MAP (\n")
       i := 0
       for jsonKey, customName := range customMappings {
           sep := ","
           if i == len(customMappings)-1 {
               sep = ""
           }
           fmt.Fprintf(e.output, "    '%s' AS '%s'%s\n", jsonKey, customName, sep)
           i++
       }
       fmt.Fprintf(e.output, "  )")
   }

   And in the grammar:

   createJsonStructureStatement
       : JSON STRUCTURE qualifiedName (COMMENT STRING_LITERAL)? SNIPPET (STRING_LITERAL | DOLLAR_STRING)
         (CUSTOM_NAME_MAP LPAREN customNameMapping (COMMA customNameMapping)* RPAREN)?
       ;
   
   customNameMapping
       : STRING_LITERAL AS STRING_LITERAL   // 'jsonKey' AS 'CustomName'
       ;
   

   This does NOT follow the (Key: value) format. It uses a custom syntax with AS.

   Let me check if this is an established pattern elsewhere in MDL...

   Looking at the existing MDL_QUICK_REFERENCE.md before this PR, I don't see many examples of property lists, but let me check if there are any...

   Actually, looking at the skill files referenced in CLAUDE.md, particularly .claude/skills/design-mdl-syntax.md which I should have read before designing syntax...

   Since I don't have access to that file, I need to infer from existing MDL syntax.

   Looking at the ALTER ENTITY syntax from the existing docs (not in the diff but mentioned in CLAUDE.md), I would expect to see patterns like:

   ALTER ENTITY Module.EntityName ADD ATTRIBUTE attrName (Type: String, Length: 50);
   

   Or similar.

   The CUSTOM NAME MAP syntax using AS instead of colons appears to be inconsistent with the guideline.

   However, let me double-check if there are any existing uses of AS in MDL for similar purposes...

   Looking through the diff, I see in MDLLexer.g4:

   CUSTOM_NAME_MAP: C U S T O M WS+ N A M E WS+ M A P;
   

   And in the grammar, it's used as:

   CUSTOM_NAME_MAP LPAREN customNameMapping (COMMA customNameMapping)* RPAREN
   

   With customNameMapping being:

   STRING_LITERAL AS STRING_LITERAL
   

   This seems to be a deliberate design choice, but I need to check if it follows the guidelines.

   Let me look at other parts of the PR to see if there are other syntax elements...

   The DESCRIBE output shows:

   CUSTOM NAME MAP (
       'jsonKey' AS 'CustomName'
   )
   

   This is readable as English, uses qualified names, etc.

   But the property format guideline specifically says: "uses ( Key: value, ... ) with colon separators, one per line"

   This syntax does not follow that - it uses AS instead of colons.

   However, let me consider if this might be acceptable because:

   1. It's a mapping (key AS value) which naturally reads as English
   2. The guideline might be more flexible for mapping-like constructs
   3. It might be consistent with how other mapping operations work in MDL

   Let me check if there are other places in MDL that use AS for mappings...

   I don't see any in the diff, but let me think about common MDL patterns...

   Actually, looking at the IMPORT statement mentioned in CLAUDE.md:

   IMPORT FROM <alias> QUERY '...' INTO Module.Entity MAP (...)
   

   And in the sql/import.go file mentioned, there might be mapping syntax...

   But I don't have that in the diff.

   Let me check the existing doctype test files for examples of mapping syntax...

   From the list of existing doctype test files, I see things like:

   • 05-database-connection-examples.mdl
   • 06-rest-client-examples.mdl

   But I don't have the content.

   Given that I need to make a judgment, and the guideline is quite specific about "Property format — uses ( Key: value, ... ) with colon separators, one per line", I believe this is a violation.

   However, let me also check if the CUSTOM NAME MAP is considered a "property list" in the sense of the guideline.

   The guideline says: "Property format — uses ( Key: value, ... ) with colon separators, one per line"

   In the CREATE JSON STRUCTURE statement, we have:

   • SNIPPET '...' (this is a value, not in parentheses)
   • Optionally COMMENT '...' (this is a value)
   • Optionally CUSTOM NAME MAP (...) (this is where the mapping goes)

   The CUSTOM NAME MAP itself contains a list of mappings, and it's this list that should follow the property format.

   Currently it's: ('jsonKey' AS 'CustomName', ...)
   It should be: (jsonKey: 'CustomName', ...) or similar.

   But wait, the guideline also says: "Reads as English — a business analyst understands the statement on first reading"

   "'jsonKey' AS 'CustomName'" reads very naturally as English.
   "jsonKey: 'CustomName'" is more technical.

   Let me check what the guideline says about this tension...

   Since I don't have access to .claude/skills/design-mdl-syntax.md, I need to make a judgment based on what I can see.

   Looking at the examples in the PR:

   CREATE JSON STRUCTURE Module.Name SNIPPET '...'
   CUSTOM NAME MAP ('jsonKey' AS 'CustomName');
   

   This reads as: "Create JSON structure Module.Name from snippet '...' with custom name map 'jsonKey' as 'CustomName'"

   Which is very readable.

   If it were:

   CREATE JSON STRUCTURE Module.Name SNIPPET '...'
   CUSTOM NAME MAP (jsonKey: 'CustomName');
   

   It would read as: "Create JSON structure Module.Name from snippet '...' with custom name map jsonKey: 'CustomName'"

   Which is less natural.

   Also, looking at the DESCRIBE output which should be re-executable, it shows the AS format.

   Let me check if there are any other examples of mappings in MDL that use AS...

   Actually, in the REST client skill that was also modified in this PR (.claude/skills/mendix/rest-client.md), I see in the diff:

   ## JSON Structures
   ...
   -- Custom name mapping for non-English keys
   CREATE JSON STRUCTURE Module.Name SNIPPET $${"kvkNummer": "123"}$$
   CUSTOM NAME MAP ('kvkNummer' AS 'ChamberOfCommerceNumber');
   

   This suggests that the AS syntax is intentional and considered readable.

   Let me also check the guideline about "LLM-friendly" — one example is sufficient for an LLM to generate correct variants.

   The example with AS is clear.

   And "Diff-friendly" — adding one property is a one-line diff.

   With the current syntax, adding a mapping would be:

   CUSTOM NAME MAP (
     'existingKey' AS 'ExistingName',
   + 'newKey' AS 'NewName'
   )
   

   Which is a one-line diff (plus adjusting the comma on the previous line).

   If it used colons, it would be similar.

   Given the readability aspect and the fact that AS is commonly used in mappings (like in SQL SELECT aliases), I think this might be acceptable despite the strict letter of the guideline.

   However, the guideline is quite explicit, so I should note this as a potential issue.

   Let me also check other aspects of the syntax:

   • No keyword overloading: Each keyword has one consistent meaning - ✓ (JSON, STRUCTURE, SNIPPET, CUSTOM, NAME, MAP all seem to be used consistently)
   • Statement reads as English: ✓ (Very readable)
   • Diff-friendly: Adding one property is a one-line diff - For the CUSTOM NAME MAP, adding one entry requires changing one line and adjusting a comma on another line, which is mostly one-line diff
   • Consistent with existing MDL patterns: Need to check

   Let me look at the existing MDL patterns for similar constructs...

   From the CLAUDE.md, I see references to ALTER PAGE/SNIPPET with SET, INSERT, DROP, REPLACE operations.

   Let me imagine what an ALTER JSON STRUCTURE might look like (though this PR doesn't add it, following the "no ALERT needed" rationale in the plan).

   But for existing patterns, let's consider how properties are specified in other statements.

   Actually, looking at the skill files mentioned in CLAUDE.md, particularly .claude/skills/create-page.md or .claude/skills/alter-page.md, I might find examples.

   Since I don't have those, let me look at the existing doctype test files for clues.

   From the list, I see 03-page-examples.mdl, 14-project-settings-examples.mdl, etc.

   But without content, it's hard.

   Let me check if there are any examples in the MDL_QUICK_REFERENCE.md that show property lists...

   In the diff to MDL_QUICK_REFERENCE.md, I see the Navigation section:

   ## Navigation Menus
   
   | Statement | Syntax | Notes |
   |-----------|--------|-------|
   | Create navigation | `CREATE MENU Module.Name [ ( Caption: 'text', Key: 'key' ) ];` | Create a new navigation menu |
   

   Ah! Here we see the property format: ( Caption: 'text', Key: 'key' ) with colons.

   This confirms that the standard MDL property format uses colons.

   Therefore, the CUSTOM NAME MAP syntax using AS instead of colons is inconsistent with the established pattern.

   It should be:

   CUSTOM NAME MAP (jsonKey: 'CustomName', ...)
   

   Or perhaps:

   CUSTOM NAME MAP ('jsonKey': 'CustomName', ...)
   

   But given the Navigation example shows ( Caption: 'text', Key: 'key' ) without quotes around the keys, it should probably be:

   CUSTOM NAME MAP (jsonKey: 'CustomName', ...)
   

   However, jsonKey might need to be quoted if it contains special characters or starts with a number, but in JSON keys, they are strings.

   Looking at the example in the PR: 'kvkNummer' AS 'ChamberOfCommerceNumber'

   If we convert to the colon format, it would be: kvkNummer: 'ChamberOfCommerceNumber'

   But kvkNummer is a JSON key, which in the snippet is a string, so in the MDL it might need to be quoted when used as a key in the map.

   Let me check how other MDL statements handle keys that might need quoting...

   In the Navigation example, Caption and Key are identifiers, not strings that need quoting.

   For a JSON key like "kvkNummer", which is a string, it might need to be quoted in the MDL map key.

   Looking at existing MDL, I don't see a perfect parallel, but in some contexts, keys are quoted.

   Alternatively, the syntax could be:

   CUSTOM NAME MAP ( "kvkNummer": 'ChamberOfCommerceNumber', ... )
   

   But the Navigation example doesn't quote the keys.

   Another possibility is that JSON keys in this context are treated as identifiers, so:

   CUSTOM NAME MAP ( kvkNummer: 'ChamberOfCommerceNumber', ... )
   

   But kvkNummer starts with a lowercase letter, and Mendix identifiers typically start with uppercase? Not sure.

   Given the Navigation example uses ( Caption: 'text', Key: 'key' ), I think the intended format for CUSTOM NAME MAP should be:

   CUSTOM NAME MAP ( jsonKey: 'CustomName', ... )
   

   With the understanding that jsonKey here refers to the JSON key (which would be an identifier in MDL context).

   However, JSON keys can contain spaces, hyphens, etc., which wouldn't be valid MDL identifiers.

   This suggests that the JSON key might need to be quoted when used as a key in the map.

   Let me think about how this would work...

   If the JSON snippet is: { "kvk-nummer": "123" }

   Then the map entry might need to be: "kvk-nummer": 'ChamberOfCommerceNumber'

   Or if MDL allows hyphens in identifiers: kvk-nummer: 'ChamberOfCommerceNumber'

   But I don't know if MDL allows hyphens in identifiers.

   Given the complexity, and that the current AS syntax works and is readable, but violates the explicit guideline, I need to make a call.

   Looking back at the CLAUDE.md checklist item for MDL syntax design:

   • Property format — uses ( Key: value, ... ) with colon separators, one per line

   This is clearly not followed by the current implementation.

   However, let me check if there are any exceptions or if this case is special...

   The plan document (docs/plans/2026-04-01-show-describe-json-structures.md) shows in the Grammar section:

   customNameMapping
       : STRING_LITERAL AS STRING_LITERAL   // 'jsonKey' AS 'CustomName'
       ;
   

   And in the MDL syntax examples:

   -- Override auto-generated element names
   CREATE JSON STRUCTURE Module.Name SNIPPET '...'
   CUSTOM NAME MAP ('jsonKey' AS 'CustomName');
   

   This suggests that the designers intentionally chose this format.

   Let me check if the guideline has any nuance I'm missing...

   The guideline says: "Property format — uses ( Key: value, ... ) with colon separators, one per line"

   But perhaps for mappings, a different format is acceptable?

   The guideline also says: "Follows standard patterns — uses CREATE/ALTER/DROP/SHOW/DESCRIBE, not custom verbs"

   Which we do follow.

   And: "Reads as English — a business analyst understands the statement on first reading"

   The AS format reads very well as English.

   Let me see what the design skill says (which I don't have access to, but is referenced in the checklist).

   Since I can't access `.claude/skills//

---

Automated review via OpenRouter (Nemotron Super 120B) — workflow source

[peterjumpnl]

Issue 1: CUSTOM NAME MAP syntax: AS is the established MDL keyword for name mapping/aliasing. It's used consistently throughout the codebase: PUBLISH ENTITY ... AS 'name', MAP (Old AS New), SQL CONNECT ... AS alias, OldPlaceholder AS NewPlaceholder. The Key: value colon format is for properties, not name mappings. This is correct as-is.

Issue 2: VS Code extension: No new LSP capabilities were added (just keyword completions, which are already handled by the
existing completion provider). No package.json change needed.

Issue 3: UTF-8 bug: This is a valid finding. strings.ToUpper(string(runes[0]))[0] indexes a byte, not a rune. For multi-byte uppercase characters this would truncate.

[ako]

@peterjumpnl i think the review bot can also use some improvements. The as keyword is fine for mappings. I'll update the review bot.

[ako]

Review

Well-structured PR — single feature, full pipeline coverage, good documentation. This is one of the better-scoped PRs in the series.

Full-stack checklist

Step	Status
Grammar (lexer + parser)	✅ STRUCTURES, CUSTOM_NAME_MAP tokens; createJsonStructureStatement, customNameMapping rules
Parser regenerated	✅
AST	✅ CreateJsonStructureStmt, DropJsonStructureStmt
Visitor	✅ visitor_jsonstructure.go
Executor	✅ cmd_jsonstructures.go
BSON reader/writer	✅ parser_misc.go, writer_jsonstructure.go
DESCRIBE roundtrip	✅ Outputs re-executable CREATE OR REPLACE MDL
Catalog	✅ json_structures table
Autocomplete	✅ DESCRIBE/DROP completion
LSP keywords	✅ lsp_completions_gen.go
Quick reference	✅
Skill docs	✅ rest-client.md updated
Test examples	✅ 20-json-structure-examples.mdl

Issues

1. int32 for property values — contradicts PR #71 (needs verification)

The writer uses int32 for FractionDigits, MaxLength, MaxOccurs, MinOccurs, TotalDigits:

{Key: "FractionDigits", Value: int32(elem.FractionDigits)},
{Key: "MaxLength", Value: int32(elem.MaxLength)},

PR #71 (just merged) established that property values must be int64 because Studio Pro's type cache matches by BSON wire type. The PR description says "Verified against Studio Pro" — if Studio Pro genuinely stores JsonStructures$JsonElement properties as int32, this is fine but should have a comment explaining why this type differs from the int64 convention. Otherwise, these should be int64 for consistency.

The parser also reads these as int32:

if v, ok := raw["MinOccurs"].(int32); ok {

If Studio Pro actually writes int64 here, the parser will silently skip these fields (they'll keep the default -1 values).

2. Non-deterministic DESCRIBE output for custom name maps

for jsonKey, customName := range customMappings {

Go map iteration order is random. If a structure has multiple custom name mappings, DESCRIBE will produce different output on each run. This breaks the "re-executable" guarantee and makes diffs noisy. Use a sorted key slice instead.

3. Unchecked errors in JSON parsing

In buildElementFromRawObject:

dec.Token() // opening {   — error ignored
// ...
dec.Decode(&rawVal)          — error ignored

If the JSON is malformed partway through (already validated at the top level, but sub-objects aren't re-validated), these will silently produce incomplete element trees. At minimum, check the dec.Token() error.

4. Whitespace-only changes

reader_types.go and cmd_catalog.go contain alignment-only changes (reformatting struct tags and map literals). These add noise to the diff without functional change. Prefer keeping these in a separate commit or omitting them.

Minor observations

• singularize is intentionally naive ("Addresses" → "Addresse") to match Studio Pro behavior — good, and documented
• iso8601Pattern regex handles common ISO 8601 variants; edge cases (week dates, ordinal dates) won't match, which is correct since Studio Pro doesn't recognize those either
• normalizeDateTimeValue pads fractional seconds to 7 digits (.NET-style) — good attention to detail
• Reserved name handling (Id → _id, Type → _type) matches Studio Pro — the comment says "Name" is also reserved but reservedExposedNames only has Id and Type
• The 444-line design doc (docs/plans/2026-04-01-show-describe-json-structures.md) is committed — same question as prior PRs: is this intended to be permanent documentation?
• CUSTOM_NAME_MAP lexer token uses WS+ to match multi-word keywords with embedded whitespace — this works but is unusual for this grammar (most multi-word constructs use separate tokens). Not a problem, just noting the pattern.

Recommendation

Address the int32/int64 question (verify against Studio Pro BSON) and fix the map iteration order in DESCRIBE. The rest is solid.