-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
fix: numeric literals can be set as object keys and used to access objects #3498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from 7 commits
cedc04f
70de523
384e0fe
90cb3f6
349f39d
eafba64
e8377d1
9d854ba
f95f57f
8edbfe4
12b23bf
59060fd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| import { factory } from '../utils/factory.js' | ||
| import { isAccessorNode, isConstantNode, isFunctionNode, isOperatorNode, isSymbolNode, rule2Node } from '../utils/is.js' | ||
| import { isAccessorNode, isConstantNode, isFunctionNode, isObjectNode, isOperatorNode, isSymbolNode, rule2Node } from '../utils/is.js' | ||
| import { deepMap } from '../utils/collection.js' | ||
| import { safeNumberType } from '../utils/number.js' | ||
| import { hasOwnProperty } from '../utils/object.js' | ||
|
|
@@ -1413,6 +1413,12 @@ export const createParse = /* #__PURE__ */ factory(name, dependencies, ({ | |
| closeParams(state) | ||
| getToken(state) | ||
|
|
||
| // If param value is number and node is object node, make param value a string | ||
| if (typeof params[0].value === 'number' && isObjectNode(node) && params.length === 1 && isConstantNode(params[0])) { | ||
| // Number constructor is first used to manage situations of numbers with preceding zero digit(s) | ||
| params[0].value = String(Number(params[0].value)) | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since it can be the case that the actual (numeric or string) key is evaluated at runtime and not parse time, I think this logic should not be located here but when evaluating the actual key, in functions like
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alright, I will look into effecting the change from IndexNode instead
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right...making the changes in IndexNode will affect assignments too, which is something I overlooked. I'm thinking of introducing an optional property like 'forObjectNode'. But since there's already an optional property for dot notation, I'm now thinking of having an optional options object that would contain dotNotation and forObjectNode.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've been able to mostly move this logic to IndexNode.isObjectProperty and .getObjectProperty I did attach a boolean property, 'forObjectNode' to params[0] here I opted for this against modifying the structure of arguments to the IndexNode constructor as I initially intended in my earlier comment, because this felt more controlled and still got the job done However, using that forObjectNode property I attached to the nodes being fed into the IndexNode constructor, I've been able to recreate this logic you commented on here in IndexNode's isObjectProperty and getObjectProperty
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @josdejong Can you please review this update?
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it makes sense to await reviewing until you've tried out the latest ideas, or is there a specific part of the code on which you already would like to have more feedback?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @josdejong I've pushed my latest changes and I would like your review. The earlier logic I had in parseAccessor has been completely moved out. To be able to handle numbers, operations and expressions in matrix indexes for object accessing and assignments, I relied on a singular DenseMatrix being present in the index._dimensions array recieved in the access and assign functions Problems are that
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the updates, I'll do some testing and debugging with the latest version of your PR soon (I expect tomorrow). It can indeed be that some of the security tests now give differing error messages, that is fine.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you very much
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. About (2): the tests for names will need to change if we drop support for these "smart" names. It will be a breaking change. Thinking about it however, it may be better to leave it as it is for now and mark it deprecated, then we don't introduce a breaking change right now, and it may be better to try focus on getting numbers working as index, which is complex enough already. Let's try one step at a time 😅 . |
||
| } | ||
|
|
||
| node = new AccessorNode(node, new IndexNode(params)) | ||
| } else { | ||
| // dot notation like variable.prop | ||
|
|
@@ -1615,11 +1621,11 @@ export const createParse = /* #__PURE__ */ factory(name, dependencies, ({ | |
| // parse key | ||
| if (state.token === '"' || state.token === "'") { | ||
| key = parseStringToken(state, state.token) | ||
| } else if (state.tokenType === TOKENTYPE.SYMBOL || (state.tokenType === TOKENTYPE.DELIMITER && state.token in NAMED_DELIMITERS)) { | ||
| key = state.token | ||
| } else if (state.tokenType === TOKENTYPE.SYMBOL || (state.tokenType === TOKENTYPE.DELIMITER && state.token in NAMED_DELIMITERS) || state.tokenType === TOKENTYPE.NUMBER) { | ||
| key = state.tokenType === TOKENTYPE.NUMBER ? String(Number(state.token)) : state.token | ||
| getToken(state) | ||
| } else { | ||
| throw createSyntaxError(state, 'Symbol or string expected as object key') | ||
| throw createSyntaxError(state, 'Symbol, numeric literal or string expected as object key') | ||
| } | ||
|
|
||
| // parse key/value separator | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -919,6 +919,42 @@ describe('parse', function () { | |
| }, /Cannot apply a numeric index as object property/) | ||
| }) | ||
|
|
||
| it('should coerce numbers to string when trying to apply a numeric key in an object expression', function () { | ||
| assert.deepStrictEqual(parseAndEval('{2: 6}'), { 2: 6 }) | ||
| }) | ||
|
|
||
| it('should throw an error when negative numbers are applied as keys in an object expression', function () { | ||
| assert.throws(function () { | ||
| parseAndEval('{-1: 34}') | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with Glen: I cannot come up with a reason why we wouldn't support negative numbers. Can you implement support for negative values too? Or do you have an other opinion?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I believe I can. I left it out earlier because javascript also didn't seem to accept that
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @josdejong I was able to make this happen as well by modifying a part of the parseObject function. Right before it would throw an error due to not finding a symbol, string or number as an object key, if the current token is '-', it would try to parseUnary and if the resulting node is an operator node with just a constant node with a number value, then it would proceed to use the number preceeded by a '-' as the key
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you're idea is the same as Glen implemented for the |
||
| }, /Symbol, numeric literal or string expected as object key \(char 2\)/) | ||
| }) | ||
|
|
||
| it('should coerce numbers to string when trying to access an object expression property with matrix index', function () { | ||
| assert.strictEqual(parseAndEval('{2: 6}[2]'), 6) | ||
| assert.strictEqual(parseAndEval('{2.5: 6}[2.5]'), 6) | ||
| assert.strictEqual(parseAndEval('{5: 16}[3]'), undefined) | ||
| }) | ||
|
|
||
| it('should accept operations in a matrix when trying to access an object expression property', function () { | ||
| assert.strictEqual(parseAndEval('{2: 6}[2 * 1]'), 6) | ||
| assert.strictEqual(parseAndEval('{31: 7 - 4}[0.2 + 0.8]'), undefined) | ||
| assert.strictEqual(parseAndEval('{4: 11 * 4}[(2 ^ 2) * 1]'), 44) | ||
| }) | ||
|
|
||
| it('should ignore leading zeros when trying to apply numeric keys in an object expression', function () { | ||
| assert.deepStrictEqual(parseAndEval('{02: 6}'), { 2: 6 }) | ||
| assert.deepStrictEqual(parseAndEval('{0070: 6}'), { 70: 6 }) | ||
| assert.deepStrictEqual(parseAndEval('{0.2: 6}'), { 0.2: 6 }) | ||
| assert.deepStrictEqual(parseAndEval('{0010.0501: "haha"}'), { 10.0501: 'haha' }) | ||
| }) | ||
|
|
||
| it('should ignore leading zeros in a matrix index when trying to access an object expression property', function () { | ||
| assert.strictEqual(parseAndEval('{2: 6}[02]'), 6) | ||
| assert.strictEqual(parseAndEval('{70: 1 - 6}[0070]'), -5) | ||
| assert.strictEqual(parseAndEval('{0.2: 6}[000.2]'), 6) | ||
| assert.strictEqual(parseAndEval('{10.0501: "haha"}[0010.0501]'), 'haha') | ||
| }) | ||
|
|
||
| it('should set a nested matrix subset from an object property (1)', function () { | ||
| const scope = { obj: { foo: [1, 2, 3] } } | ||
| assert.deepStrictEqual(parseAndEval('obj.foo[2] = 6', scope), 6) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new section in
AccessorNodeadds support for operators inside the accessor like{2: 6}[2 * 1], but it doesn't allow any expression in general, for example it fails with{2: 6}[multiply(2, 1)].I think the right place to update this behavior is not here in
AccessorNode, but inIndexNode.isObjectPropertyandIndexNode.getObjectProperty, see also my other comment inparse.js.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm having trouble updating this behaviour in IndexNode.getObjectProperty
I was able to compile the result of the operation because this was happening in a _compile method and I had access to math and argNames, so could call the compile method of the OperatorNode
In IndexNode.getObjectProperty, I don't have access to math and argNames, so can't use the _compile method on an OperatorNode in there. And I can't find another way to compile a result to the operation in an OperatorNode
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've looked into this a bit more, it's more complex than I had expected.
In
AccessorNode._compile, there is anif (this.index.isObjectProperty()) { ... }. I think that part of the code must be removed, since it only works for static strings and not expressions inside the brackets. And also, we cannot detect anymore whether we're dealing with an object key (vs a matrix subset) by just looking at the index: when the index now contains just one dimension with one number, it could be an object key too. Therefore, this should be handled by theaccess(...)function. The logic in the functionaccessthat handles the case of an object can be extended to correctly evaluate numeric properties too. To do that, instead of theindex.getObjectProperty()we need a method likeindex.evalObjectProperty(scope, args, context). We cannot use the normalIndexNode._compile(...)because that changes numeric indexes from 1-based to 0-based, and we don't want that in case of an object key.Similarly, we need to update/extend the logic in
AssignmentNode.Maybe we should also get rid of the "smart" name function of
AccessorNode, seeget name() { ... }. It's not really needed in practice I think, and is not reliable anymore when it only works in "some" cases.Does this direction sort of make sense?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it all make sense. Thank you for the directions. I will try to work it out and revert