From ee62cffb78a337afa27b45f880ae35bacf163316 Mon Sep 17 00:00:00 2001 From: Seonghee Lee Date: Tue, 25 Aug 2026 11:12:56 -0700 Subject: [PATCH 1/2] docs: scaffold tensor parallelism optimization blog post Adds a placeholder page and Tech Blogs navigation section so the TP optimization write-up has a home before the content lands. --- docs/blog/tensor-parallelism-optimization.md | 27 ++++++++++++++++++++ docs/index.yml | 6 +++++ 2 files changed, 33 insertions(+) create mode 100644 docs/blog/tensor-parallelism-optimization.md diff --git a/docs/blog/tensor-parallelism-optimization.md b/docs/blog/tensor-parallelism-optimization.md new file mode 100644 index 000000000..341dcc9ea --- /dev/null +++ b/docs/blog/tensor-parallelism-optimization.md @@ -0,0 +1,27 @@ +# Optimizing Tensor Parallelism in JAX and XLA + +> **Draft** — this page is a placeholder. Content is in progress. + +## Background + + + +## Where tensor parallelism costs performance + + + +## Optimizations + + + +## Results + + + +## Reproducing + + + +## Takeaways diff --git a/docs/index.yml b/docs/index.yml index ccfac3e37..ce319a1d5 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -55,6 +55,12 @@ navigation: - page: nsys-jax path: nsys-jax.md + # ==================== Tech Blogs ==================== + - section: Tech Blogs + contents: + - page: Optimizing Tensor Parallelism in JAX and XLA + path: blog/tensor-parallelism-optimization.md + # ==================== Resiliency ==================== - section: Resiliency contents: From ecbac5f04b8402623f1ade976eabfd39ee4a37cb Mon Sep 17 00:00:00 2001 From: Seonghee Lee Date: Tue, 25 Aug 2026 11:17:55 -0700 Subject: [PATCH 2/2] docs: use MDX comment syntax in blog placeholder Fern parses .md pages as MDX, so HTML comments fail to parse and blank the page. No other docs page uses HTML comments. --- docs/blog/tensor-parallelism-optimization.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/blog/tensor-parallelism-optimization.md b/docs/blog/tensor-parallelism-optimization.md index 341dcc9ea..a1a88b408 100644 --- a/docs/blog/tensor-parallelism-optimization.md +++ b/docs/blog/tensor-parallelism-optimization.md @@ -4,24 +4,24 @@ ## Background - +{/* Why TP matters: model sizes past single-GPU memory, the compute/communication + tradeoff versus FSDP and pipeline parallelism. */} ## Where tensor parallelism costs performance - +{/* All-gather / reduce-scatter on the critical path, exposed communication, + small GEMM tiles at high TP degree, NVLink versus inter-node bandwidth. */} ## Optimizations - +{/* One subsection per optimization, with the mechanism and how to enable it. */} ## Results - +{/* Model, hardware, TP degrees swept, throughput and step-time deltas. */} ## Reproducing - +{/* Container tag, framework config flags, launch command. */} ## Takeaways