Compile literal binary patterns in the loader - #11468
Open
josevalim wants to merge 2 commits into
Open
Conversation
Contributor
CT Test Results 5 files 545 suites 2h 3m 10s ⏱️ Results for commit 2ba5a31. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
Allow non-empty lists of inclusive byte-range tuples
in binary:compile_pattern/1 and the binary match, matches,
split, and replace operations. Normalize overlapping
ranges and compile them to the new br pattern type.
Use SIMD for up to 16 normalized entries, with equality
checks for singletons and one-sided comparisons for ranges
touching 0 or 255. Use the 256-byte membership table for
tails, unsupported SIMD targets, and larger range sets.
ASCII validation throughput on arm64 macOS (million calls/s):
SIMD Unicode SWAR Guard
Valid ASCII 14.40 2.74 2.74 0.41
Invalid last 13.78 3.00 2.74 0.41
Invalid first 42.30 148.81 91.83 104.38
SIMD is the one added by this pull request, Unicode stands
for the current unicode:bin_is_7bit/1. Guard is the equivalent
guard imlpementation and the SWAR one is an optimized version
of that. The native unicode:bin_is_7bit/1 can be removed in
a future commit.
As shown in the benchmarks above, the new byte ranges are
clear winners for validating data, as most times you are
expecting to fully traverse binaries without matches.
For parsing, byte ranges can be faster than guards, but may
be beaten by SWAR on certain payloads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As we improve performance of
binary:match/2,compiling the pattern ends-up taking a higher cost of the
overall operation, so we automatically handle them in
the loader.
PS: I am not familiar with the loader and I cannot really assess
the quality of this patch but hopefully it is good enough to
start the conversation. @sverker has proposed a similar patch
for re:import/1 in the past.