Fully address memory management issues: profiling, KV cache lifecycle docs, tunable chunking, tests at scale (#2190) - #2300

Fully address memory management issues: profiling, KV cache lifecycle docs, tunable chunking, tests at scale (#2190)#2300

OnePunchMonk wants to merge 13 commits into

Lightning-AI:mainLightning-AI/litgpt:mainfrom

OnePunchMonk:fix/kv-cache-stale-errorOnePunchMonk/litgpt:fix/kv-cache-stale-errorCopy head branch name to clipboard