prov/efa: Minor changes necessary for the protocol path refactor#12010
Merged
sunkuamzn merged 4 commits intoofiwg:mainfrom Apr 24, 2026
Merged
prov/efa: Minor changes necessary for the protocol path refactor#12010sunkuamzn merged 4 commits intoofiwg:mainfrom
sunkuamzn merged 4 commits intoofiwg:mainfrom
Conversation
sunkuamzn
force-pushed
the
proto-callback-prereq
branch
from
a486551 to
76b9f74
Compare
talavr-amazon
requested changes
Mar 18, 2026
sunkuamzn
force-pushed
the
proto-callback-prereq
branch
7 times, most recently
from
88ed8af to
7d9e360
Compare
| @@ -195,7 +195,8 @@ struct efa_rdm_ep { | |||
| struct efa_rdm_pke **pke_vec; | |||
| /* Work arrays for efa_rdm_ope_post_send to avoid stack allocation */ | |||
| struct efa_rdm_pke **send_pkt_entry_vec; | |||
There was a problem hiding this comment.
Will it be cleaner if we create a send_pkt_entry struct to wrap these 3 together?
Contributor
Author
There was a problem hiding this comment.
I thought about this. It will add another layer of redirection and I'm not sure if it helps. I did not add a new struct.
There was a problem hiding this comment.
Having an nested struct does not cause an extra layer of indirection. It does however help in organizing this giant and messy struct
| /* Work arrays for efa_rdm_ope_post_send to avoid stack allocation */ | ||
| struct efa_rdm_pke **send_pkt_entry_vec; | ||
| int *send_pkt_entry_size_vec; | ||
| size_t *send_pkt_entry_data_sizes; |
There was a problem hiding this comment.
nit: if they're not bundled together, maybe name it send_pkt_entry_vec_data/payload_sizes
talavr-amazon
approved these changes
Apr 15, 2026
| ssize_t err; | ||
| struct efa_rdm_ope *txe; | ||
| struct efa_rdm_peer *peer; | ||
| int available_tx_pkts; |
There was a problem hiding this comment.
unsigned or add an assert that it's >0?
talavr-amazon
approved these changes
Apr 15, 2026
| if (!txe) | ||
| return NULL; | ||
|
|
||
| efa_rdm_txe_construct(txe, efa_rdm_ep, peer, &msg, op, 0); |
There was a problem hiding this comment.
should we not check for ofi_op_tagged here?
Contributor
Author
There was a problem hiding this comment.
Why? We're just constructing the tx entry
There was a problem hiding this comment.
Seems like the rest of the code are setting the tag right after the construct call
talavr-amazon
previously approved these changes
Apr 15, 2026
| /* | ||
| * The default memory alignment | ||
| */ | ||
| #define EFA_RDM_DEFAULT_MEMORY_ALIGNMENT (8) |
There was a problem hiding this comment.
if these are place in this header and are endpoint specific values, they should be prefixed with EFA_RDM_EP_
shijin-aws
reviewed
Apr 15, 2026
shijin-aws
reviewed
Apr 15, 2026
The pkt entries are stored in send_pkt_entry_vec, so the size should also be stored in the endpoint Also rename send_pkt_entry_size_vec to send_pkt_entry_data_sizes to avoid confusion between the two members Signed-off-by: Sai Sunku <sunkusa@amazon.com>
If the send queue is full, no point creating txe and pkes. Moved the available_tx_pkts function to a static inline function so that it can be re-used without copying. Signed-off-by: Sai Sunku <sunkusa@amazon.com>
Remove the efa_rdm_ep_alloc_txe helper and inline its logic at each call site. The function combined pool allocation, TXE construction, tagged field setup, and list insertion in a way that made it difficult to separate allocation from initialization for the new protocol interface. Move the dlist_insert_tail for ep->txe_list into efa_rdm_txe_construct so that all callers get consistent list management without duplicating the insertion call. Signed-off-by: Sai Sunku <sunkusa@amazon.com>
Move efa_rdm_ep_get_memory_alignment from efa_rdm_ep_utils.c to efa_rdm_ep.h as a static inline function. The function is small and called in the packet construction hot path, so inlining avoids function call overhead. Also move the EFA_RDM_*_ALIGNMENT constant definitions from efa.h to efa_rdm_ep.h to keep them co-located with the function that uses them. Add EP to the prefix to reflect their new location. Signed-off-by: Sai Sunku <sunkusa@amazon.com>
sunkuamzn
force-pushed
the
proto-callback-prereq
branch
from
7d9e360 to
b8c0665
Compare
shijin-aws
approved these changes
Apr 22, 2026
talavr-amazon
approved these changes
Apr 24, 2026
talavr-amazon
approved these changes
Apr 24, 2026
| if (!txe) | ||
| return NULL; | ||
|
|
||
| efa_rdm_txe_construct(txe, efa_rdm_ep, peer, &msg, op, 0); |
There was a problem hiding this comment.
Seems like the rest of the code are setting the tag right after the construct call
talavr-amazon
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisite for #12002