#452 Data Generator private package to generate test data from Dictionaries and Schemas - #454
Open
joneubank wants to merge 11 commits into
Open
#452 Data Generator private package to generate test data from Dictionaries and Schemas#454joneubank wants to merge 11 commits into
joneubank wants to merge 11 commits into
Conversation
joneubank
force-pushed
the
feat/iterative-validators
branch
from
September 1, 2026 18:20
019166b to
a24587c
Compare
Fully implement typed value generation for boolean, integer, number, and string schema fields, with deterministic seeding, conditional restriction resolution, and conflict reporting when restrictions cannot be reconciled. - Add `FieldGeneratorResult<T>`, `FieldGeneratorFailureData<T>`, `FieldGeneratorOptions`, and `FieldGenerator<TField>` types - Export `testConditionalRestriction` from `@overture-stack/lectern-validation` - Add `@overture-stack/lectern-validation` as a dependency of `data-generator` - Add test suites for `fieldGenerators`, `resolveRestrictions`, and `restrictionReducers` (93 tests passing)
- reorganizes generation files into /fields and /records dirs - includes field generation order based on conditional restriction relationships - includes ability to provide values from a parent entity to help satisfy FK restrictions
…is not required Based on the presence of a `required:true` restriction. If required, always return a value. If not required, return an empty (undefined) value at a default 25% rate. - the rate can be specified in the generator options. rate will be between 0 and 1, with 0 being never empty and 1 being always empty. - record generator also gets the emptyRate option and this is threaded into the field generators - all tests are updated to have NO_EMPTY (emptyRate: 0) added to their generators to ensure they are testing a generated value and not a potentially empty result - empty results are determinsitic based on seed
- Fix Set mutation in extractFieldDependencies: build filtered set instead of deleting from Set during iteration - Fix cycle detection in resolveGenerationOrder: only lump genuinely cyclic fields in fallback tier; independent zero-in-degree fields now correctly get their own tier - Optimize resolveGenerationOrder to O(N+E) using a ready queue (Kahn's algorithm as intended) - Replace fc.sample with lightweight xorshift32 seeded draw in resolveForeignKeyOverrides - Rename mergedRange/effectiveRange to validatedRange/fallbackRange in resolveNumericConstraints - Extract buildRequiredEmptyConflict helper and apply required+empty conflict check to all field generators (was silently ignored for numeric and string fields) - Guard resolveArrayLength against min>max when integer exclusive bounds produce an impossible range - Narrow reduceRegex return type from RestrictionRegex|undefined to string|undefined; simplify filterCodeListByRegex accordingly - Add tests for cycle+independent field interaction and impossible array length exclusive bounds
joneubank
force-pushed
the
feat/data-generator
branch
from
September 1, 2026 18:30
e35630c to
5ccf0cf
Compare
joneubank
requested review from
MitchellShiell,
justincorrigible,
leoraba and
mistryrn
September 1, 2026 18:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a private package for Lectern data-generator which can generate data that conforms to a Lectern dictionary or schema. It is created to help with performance testing of lectern systems (data validation, submission, etc.) with inputs at a representative scale for real world applications. This attempts to create data of any scale that will be valid for the restrictions defined in the lectern dictionary.
Includes:
Issues
This is the first step in a plan to refactor the Lectern validation libraries to function with large scale data. It is necessary to prepare performance testing for large data collections.