Skip to content

Implement declare decimal-format (XQuery 3.1)#6077

Closed
joewiz wants to merge 2 commits intoeXist-db:developfrom
joewiz:feature/declare-decimal-format
Closed

Implement declare decimal-format (XQuery 3.1)#6077
joewiz wants to merge 2 commits intoeXist-db:developfrom
joewiz:feature/declare-decimal-format

Conversation

@joewiz
Copy link
Copy Markdown
Member

@joewiz joewiz commented Mar 3, 2026

Summary

  • Add parser support for XQuery 3.1 declare decimal-format and declare default decimal-format prolog declarations (spec section 4.10)
  • The runtime infrastructure was already in place (DecimalFormat, XQueryContext storage, FnFormatNumbers 3-arg support) — this adds the missing parser recognition and tree walker processing
  • Enables users to customize number formatting (e.g., European comma-as-decimal-separator conventions) via fn:format-number

Changes

File Change
XQuery.g Add tokens, grammar rules for named/default forms, property keywords in reservedKeywords
XQueryTree.g Walk AST, validate properties (single-char, zero-digit, distinctness), register formats in context
ErrorCodes.java Add XQST0097 (duplicate declarations) and XQST0098 (invalid/conflicting property values)
XQueryContext.java Add setDefaultStaticDecimalFormat() convenience method
format-numbers.xql 8 new tests for named/default formats, custom NaN/infinity, and error cases

Closes #56

See also: eXist-db/exist-xqts-runner#44 (fix copy-paste typo in XQTS runner for exponent-separator attribute)

Test plan

  • All 161 existing format-numbers.xql tests pass (0 failures, 0 errors)
  • New tests pass: named format, default format, custom NaN, custom infinity
  • New error tests pass: duplicate named (XQST0097), duplicate default (XQST0097), non-distinct properties (XQST0098), invalid zero-digit (XQST0098)
  • ANTLR grammar generates without errors
  • Full project compiles successfully

🤖 Generated with Claude Code

@joewiz joewiz requested a review from a team as a code owner March 3, 2026 01:07
@joewiz joewiz force-pushed the feature/declare-decimal-format branch 2 times, most recently from 4963be3 to 15c8748 Compare March 5, 2026 01:40
Copy link
Copy Markdown
Member

@line-o line-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @joewiz for this welcome and valuable addition to exist-db.
From what I can see and understand the changes make sense.

A few questions I asked myself going through the changes:

  • Why is the default UNNAMED?
  • The newly added function requireSingleChar sounds like a general purpose one but it is very closely tied to decimal formats looking at the exception that is thrown
  • Same is true for validateZeroDigit
  • validateDistinctPictureChars is less ambiguous but I think all new functions would benefit from a clear prefix like df or so

@joewiz
Copy link
Copy Markdown
Member Author

joewiz commented Mar 5, 2026

(CI note: Codacy keeps stalling on this PR, despite repeated restarts of the job. If only there were a way to restart one stalled job without restarting them all.)

@joewiz
Copy link
Copy Markdown
Member Author

joewiz commented Mar 5, 2026

[Co-authored with Claude Code.]

Thanks for the review, @line-o! I've pushed a commit (bb4a1a1) addressing your points:

df prefix on helper methods — Great suggestion. Renamed all three validation helpers in XQueryTree.g:

  • requireSingleChardfRequireSingleChar
  • validateZeroDigitdfValidateZeroDigit
  • validateDistinctPictureCharsdfValidateDistinctPictureChars

Why UNNAMED? — The name comes directly from the XQuery 3.1 spec (§4.10), which defines the concept of a "default unnamed decimal format" that is always available. I've added Javadoc comments on both DecimalFormat.UNNAMED and UNNAMED_DECIMAL_FORMAT in XQueryContext citing the spec section. That said, if you think DEFAULT would be clearer as a constant name (while keeping the spec reference in Javadoc), I'm happy to rename it — just let me know.

@joewiz joewiz force-pushed the feature/declare-decimal-format branch from 34ec782 to bb4a1a1 Compare March 5, 2026 14:59
@joewiz
Copy link
Copy Markdown
Member Author

joewiz commented Mar 5, 2026

(CI observation: Looks like the XQTS failure needs us to implement one of the workarounds/fixes we discussed yesterday.)

@line-o
Copy link
Copy Markdown
Member

line-o commented Mar 6, 2026

@joewiz Please rebase this onto current develop. I am curious about the XQTS results

joewiz and others added 2 commits March 6, 2026 08:20
Add parser support for the XQuery 3.1 `declare decimal-format` and
`declare default decimal-format` prolog declarations (spec section 4.10),
enabling users to customize number formatting via fn:format-number.

The runtime infrastructure (DecimalFormat class, XQueryContext storage,
FnFormatNumbers 3-arg support) was already in place — this adds the
missing parser recognition and tree walker processing.

Changes:
- XQuery.g: Add DECIMAL_FORMAT_DECL/DEF_DECIMAL_FORMAT_DECL tokens,
  grammar rules for named and default forms, property keywords
- XQueryTree.g: Walk AST, validate properties (single-char, zero-digit,
  distinctness), register formats in XQueryContext
- ErrorCodes.java: Add XQST0097 (duplicate) and XQST0098 (invalid)
- XQueryContext.java: Add setDefaultStaticDecimalFormat() convenience
- format-numbers.xql: Add tests for named/default formats, custom
  NaN/infinity, and error cases

Closes eXist-db#56

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…dd Javadoc

Rename the three decimal-format validation helper methods in XQueryTree.g
with a `df` prefix to clarify their scope:

- requireSingleChar → dfRequireSingleChar
- validateZeroDigit → dfValidateZeroDigit
- validateDistinctPictureChars → dfValidateDistinctPictureChars

Add Javadoc comments on DecimalFormat.UNNAMED and UNNAMED_DECIMAL_FORMAT
explaining the XPath 3.1 spec origin of the "unnamed" terminology.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@joewiz joewiz force-pushed the feature/declare-decimal-format branch from bb4a1a1 to 549fffd Compare March 6, 2026 13:21
@line-o
Copy link
Copy Markdown
Member

line-o commented Mar 6, 2026

This PR lets us pass 36 additional test cases of XQTS.

The two new failures are

  • re00062: this test case is flaky - there is an open issue check XQTS case re00062 #6056
  • numberformat127 - this one looks related at first but then it is apparent that it only applies to XQ30 and not XQ31
  <test-case name="numberformat127" covers-30="fn-format-number">
      <description>
         PURPOSE: exponent separator rejected in 3.0
      </description>
      <created by="Michael Kay" on="2015-04-22"/>
      <modified by="O'Neil Delpratt" on="2016-09-08" change="Changed covers attribute to covers-30" />
    <dependency type="spec" value="XQ30"/>
    <test>
      declare decimal-format exp-E exponent-separator="E";
      fn:format-number(12345.678, '9.9999E999', 'exp-E')
    </test>
    <result>
      <error code="XPST0003"/>
    </result>
  </test-case>

@line-o
Copy link
Copy Markdown
Member

line-o commented Mar 6, 2026

joewiz added a commit to joewiz/exist that referenced this pull request Mar 28, 2026
Full implementation of XQuery 3.1 decimal-format declarations:
- declare decimal-format name property = value ... ;
- declare default decimal-format property = value ... ;

All 12 properties supported: decimal-separator, grouping-separator,
infinity, minus-sign, NaN, percent, per-mille, zero-digit, digit,
pattern-separator, exponent-separator.

Validation:
- XQST0097: duplicate named or default decimal-format declarations
- XQST0098: non-distinct picture-string character values
- Zero-digit must be a Unicode digit with numeric value 0
- Single-character property validation

Complementary to PR eXist-db#6077, which adds the same support to the ANTLR 2
grammar for the develop branch. The ANTLR 2 grammar changes are already
on this branch via the parser branch lineage; this commit adds the
equivalent rd parser implementation.

NumberTests: 160/161 (was 153/161, +7). All format-number tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@joewiz
Copy link
Copy Markdown
Member Author

joewiz commented Apr 6, 2026

[This comment was co-authored with Claude Code. -Joe]

Closing — superseded by #6217 (v2/declare-decimal-format).

This work has been consolidated into a clean v2/ branch as part of the eXist-db 7.0 PR reorganization. The new PR includes all commits from this PR plus additional related work, with reviewer feedback incorporated where applicable. See the reviewer guide for the full context.

@joewiz joewiz closed this Apr 6, 2026
joewiz added a commit to joewiz/exist that referenced this pull request Apr 7, 2026
Full implementation of XQuery 3.1 decimal-format declarations:
- declare decimal-format name property = value ... ;
- declare default decimal-format property = value ... ;

All 12 properties supported: decimal-separator, grouping-separator,
infinity, minus-sign, NaN, percent, per-mille, zero-digit, digit,
pattern-separator, exponent-separator.

Validation:
- XQST0097: duplicate named or default decimal-format declarations
- XQST0098: non-distinct picture-string character values
- Zero-digit must be a Unicode digit with numeric value 0
- Single-character property validation

Complementary to PR eXist-db#6077, which adds the same support to the ANTLR 2
grammar for the develop branch. The ANTLR 2 grammar changes are already
on this branch via the parser branch lineage; this commit adds the
equivalent rd parser implementation.

NumberTests: 160/161 (was 153/161, +7). All format-number tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

I18N: format-number and decimal separator

2 participants