feat(docs): add Megatron parameter tuning guide

[guapisolo]

Summary

Add a concise Megatron capacity-planning guide to the Advanced documentation.

Motivation

Document practical HBM, host-memory, and token-capacity estimates for Megatron training, and expose the guide through the Advanced documentation navigation.

Usage

Take N, N_t, and Q from the most heavily loaded Megatron rank, apply the documented HBM or CPU-offload formula, then calibrate --max-tokens-per-gpu with two measured training runs.

Design Notes

• Key choices: Base estimates on three explicit input counts.
• Key choices: Put the memory formulas on the main path.
• Key choices: Defer supporting details to existing guides.

Verification

• Tests added: None; this is a documentation-only change.
• Build validation: npx --yes mintlify@latest validate passed.
• Link validation: npx --yes mintlify@latest broken-links found no broken links.

Review Focus

• Scrutinize the memory formulas in docs/advanced/megatron_param_tuning.md against the current Megatron defaults.
• Verify that docs/advanced/index.md and docs/docs.json expose the new page consistently.

[gemini-code-assist]

Code Review

This pull request adds a new documentation guide, "Megatron Parameter Tuning" (docs/advanced/megatron_param_tuning.md), which provides formulas and guidelines to estimate model-state memory, host memory, and token capacity for Megatron training. It also updates the advanced features index and the docs.json configuration to integrate the new page. The review feedback suggests clarifying the MoE formula in the guide by explicitly defining the trainable dense and expert parameter elements and their respective optimizer sharding degrees.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

To make the MoE formula more precise and actionable, we should clarify that $N_{\text{dense}}$ and $N_{\text{expert}}$ refer to the trainable parameter elements on the rank (to align with the definition of $Q$ as trainable elements). Additionally, we should briefly define what $S_{\text{dense}}$ and $S_{\text{expert}}$ represent (typically the optimizer sharding degrees or data-parallel sizes for the respective groups).\n\nHere is a suggested replacement:\n\nmarkdown\nFor full-parameter training with one optimizer shard group of size `S`, use `N_t = N` and `Q = N / S`. MoE models can have different dense and expert shard groups; in that case use `Q = N_dense_t / S_dense + N_expert_t / S_expert`, where `N_dense_t` and `N_expert_t` are the trainable dense and expert parameter elements on the rank, and `S_dense` and `S_expert` are their respective optimizer sharding degrees (typically the data-parallel sizes of those groups).\n