Skip to content

enable optimizer in tests and procmacros#67

Merged
alexpyattaev merged 8 commits into
anza-xyz:masterfrom
alexpyattaev:enable_optimizer_in_tests
Mar 7, 2025
Merged

enable optimizer in tests and procmacros#67
alexpyattaev merged 8 commits into
anza-xyz:masterfrom
alexpyattaev:enable_optimizer_in_tests

Conversation

@alexpyattaev
Copy link
Copy Markdown
Contributor

Problem

Same as anza-xyz/agave#5155 but for sdk

  • Unittests can be faster: optimization is not enabled for them at all, so we wait long time for CI to complete
  • Builds are slow: optimizations for procmacros are not enabled either

Summary of Changes

  • Enable optimizer for unittests
  • Enable optimizer for procmacros

If this fails to build there are bugs in tests=)

@alexpyattaev alexpyattaev force-pushed the enable_optimizer_in_tests branch from 8013d9a to 0a64077 Compare March 7, 2025 09:58
@alexpyattaev alexpyattaev force-pushed the enable_optimizer_in_tests branch from 3a7d0b7 to 7424850 Compare March 7, 2025 10:51
@alexpyattaev alexpyattaev marked this pull request as ready for review March 7, 2025 11:42
@alexpyattaev alexpyattaev requested a review from joncinque March 7, 2025 11:42
Copy link
Copy Markdown
Collaborator

@joncinque joncinque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks great overall, just a little point and a question

Comment thread program-entrypoint/Cargo.toml Outdated
description = "The Solana BPF program entrypoint supported by the latest BPF loader."
documentation = "https://docs.rs/solana-program-entrypoint"
version = "2.2.1"
version = "2.2.2"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The publish job takes care of bumping the version, so there's no need to do it here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok should I revert it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted

Comment on lines +316 to +320
// create a pointer to the start of the arena
// that will hold an address of the byte following free space
let pos_ptr = arena.as_mut_ptr() as *mut usize;
// initialize the data there
*pos_ptr = pos_ptr as usize + arena.len();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good, but for my own understanding, how does this avoid undefined behavior exactly?

Is the idea that the new call forces the optimizer to keep heap around in the tests because some part of it is written to? Would there still be UB if this code here were removed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fn new() correctly initializes the arena content where the allocated amount is, which was not done before, as the code assumed all arenas come zeroed. If you were to instantiate the BumpAllocator over a non-zeroed slice, it would cause UB the moment you call alloc().

The reason bug UB was triggered before this change was due to heap variable not marked as mut, so compiler assumed that it does not need to reinitialize it (since noone would write into it).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah very interesting, thanks for the explanation!

Copy link
Copy Markdown
Collaborator

@joncinque joncinque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@alexpyattaev alexpyattaev merged commit 4e30766 into anza-xyz:master Mar 7, 2025
@alexpyattaev alexpyattaev deleted the enable_optimizer_in_tests branch March 7, 2025 13:25
Comment on lines +310 to +320
pub unsafe fn new(arena: &mut [u8]) -> Self {
debug_assert!(
arena.len() > size_of::<usize>(),
"Arena should be larger than usize"
);

// create a pointer to the start of the arena
// that will hold an address of the byte following free space
let pos_ptr = arena.as_mut_ptr() as *mut usize;
// initialize the data there
*pos_ptr = pos_ptr as usize + arena.len();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a potential alignment issue here.
arena: &mut [u8] is not guaranteed to have proper alignment for it to be cast to a pointer to a usize.
A [u8] can have any alignment, while a pointer to usize needs to have an alignment of 4 or higher.

You can use write_unaligned() to avoid the undefined behavior.
write_unaligned() example is almost literary the code you are writing here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write unaligned could work, yes, but I think to properly address this, the entire BumpAllocator should be rewritten to use some form of interior mutability in the struct itself rather than storing the position pointer inline with the allocated data. It would make code a lot cleaner too. I will make a PR for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants