Skip to content

Document scheduler memory fencing flags #2242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shawnwang18 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Document scheduler memory fencing flags #2242

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/GPU_performance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Please note that some of these flags are experimental. All combinations of flags

- The `xla_gpu_memory_limit_slop_factor` flag controls the memory used by XLA for determining its default heuristics for scheduling, and rematerialization. Default is recommended.

- The experimental `xla_gpu_experimental_scheduler_memory_fencing_threshold_bytes` flag enables scheduler memory fencing, which limits peak memory when the latency-hiding scheduler would otherwise defer users of large buffers and keep those buffers live too long. `-1` disables the feature (default), `0` fences buffers at least 1% of the scheduler memory limit, and a positive value specifies the threshold in bytes. The optional `xla_gpu_experimental_scheduler_memory_fencing_slack_windows` flag controls how many asynchronous-operation windows fenced buffer users may be deferred (default: `1`).

## General CUDA/NCCL flags

Expand Down