Replace public Recipe.job usage with purpose-built Recipe APIs

[nvkevlu]

Replaces public Recipe.job usage with purpose-built Recipe APIs.

Description

Builds on #4862 to remove direct recipe.job usage from public examples, documentation, and research jobs.

• Adds recipe.enable_tensor_streaming(...) for configuring matching server and client tensor streamers.
• Adds add_final_global_evaluation(...) for evaluating a PyTorch recipe’s persisted global model without requesting client model submissions.
• Migrates log streaming, file packaging, server configuration, export, experiment tracking, and final evaluation examples to supported Recipe APIs.
• Expands the client-side MLflow documentation to cover local stores, remote servers, simulation behavior, and per-site configuration.
• Updates the new Kubernetes Recipe example to use add_server_file() and targeted add_client_file().

After these changes, there are no remaining recipe.job references under examples/, docs/, or research/.

This PR intentionally does not rename the internal Recipe.job attribute to _job. That mechanical/internal migration will be handled in a separate PR to keep this change focused and easier to review.

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Quick tests passed locally by running ./runtest.sh.
• In-line docstrings updated.
• Documentation updated.

[Copilot]

Pull request overview

This PR removes public-facing reliance on the internal recipe.job attribute by introducing purpose-built Recipe APIs and migrating examples/docs/research jobs to those APIs, keeping Recipe.job usage confined to internal implementation details.

Changes:

• Added new Recipe/utility APIs: Recipe.enable_tensor_streaming(...) and add_final_global_evaluation(...).
• Migrated multiple examples/research jobs/docs from recipe.job.* usage to supported Recipe helpers (add_server_config, add_*_file, export, enable_log_streaming, etc.).
• Updated documentation to clarify client-side experiment tracking patterns (including per-site targeting constraints) and revised TensorFlow validator guidance.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
research/brats18/job.py	Switches server config placement from recipe.job.to_server to recipe.add_server_config.
research/auto-fl-research/tasks/vlm_med/job.py	Replaces custom final-eval helper (and recipe.job usage) with add_final_global_evaluation.
research/auto-fl-research/tasks/cifar10/job.py	Replaces custom final-eval helper (and recipe.job usage) with add_final_global_evaluation.
nvflare/recipe/utils.py	Adds add_final_global_evaluation and adjusts cross-site evaluation docs wording.
nvflare/recipe/spec.py	Adds Recipe.enable_tensor_streaming(...) to configure matching server/client tensor streamers.
nvflare/recipe/init.py	Exposes add_final_global_evaluation via package exports.
nvflare/app_opt/tf/tf_validator.py	Updates TFValidator doc example to use the lower-level FedJob API for component placement.
examples/hello-world/hello-log-streaming/README.md	Updates Recipe log-streaming example to use recipe.enable_log_streaming("log.txt").
examples/advanced/tensor-stream/README.md	Updates tensor streaming setup docs to use recipe.enable_tensor_streaming().
examples/advanced/tensor-stream/job.py	Uses recipe.enable_tensor_streaming() instead of direct streamer placement.
examples/advanced/recipe-k8s/job.py	Migrates file bundling to recipe.add_server_file / recipe.add_client_file.
examples/advanced/kaplan-meier-he/job.py	Migrates export from recipe.job.export_job(...) to recipe.export(...).
examples/advanced/experiment-tracking/wandb/README.md	Migrates client-side targeting example to add_experiment_tracking(...).
examples/advanced/experiment-tracking/README.md	Updates MLflow client-side tracking docs and explains per-site targeting requirements.
examples/advanced/experiment-tracking/mlflow/hello-pt-mlflow-client/README.md	Expands client-side MLflow docs and migrates to add_experiment_tracking(...).
docs/user_guide/data_scientist_guide/recipe_api.rst	Documents the new enable_tensor_streaming and add_final_global_evaluation helpers.
docs/user_guide/data_scientist_guide/job_recipe.rst	Clarifies guidance around mutating generated recipe/job metadata.
docs/design/nvflare_cli.md	Updates log-streaming guidance for Recipe jobs to use recipe.enable_log_streaming("log.txt").

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[greptile-apps]

Greptile Summary

This PR completes the migration of public-facing examples, documentation, and research jobs away from direct recipe.job usage by introducing two new purpose-built Recipe APIs and switching the default MLflow local store from a file-based directory to a SQLite database.

• Adds Recipe.enable_tensor_streaming(...) to configure matching server and client tensor streamers with idempotency protection and format validation, and migrates the tensor-stream example to use it.
• Adds add_final_global_evaluation(...) (and exports it from nvflare.recipe) to run final global-model validation without requesting client model submissions; removes the now-redundant local copies from the cifar10 and vlm_med research jobs.
• Changes MLflowReceiver._get_tracking_uri default from a file://mlflow/ directory store to sqlite:///mlflow.db; users relying on the old default local path will see a new storage location for results going forward.

Confidence Score: 5/5

Safe to merge; all new Recipe APIs are well-guarded and thoroughly unit-tested.

The two new Recipe APIs both have idempotency guards, input validation, and matching unit tests covering error paths. The MLflow defaults are correctly isolated via deep copies so the caller's dict is never mutated. The default MLflow store change is intentional and documented. No logic errors were found in the changed code paths.

The integration test helper _add_model_to_apps is the one spot in the changed files that still calls recipe.job.add_file_to_server / recipe.job.add_file_to directly, which is the pattern this PR is retiring elsewhere.

Important Files Changed

Filename	Overview
nvflare/recipe/spec.py	Adds enable_tensor_streaming with idempotency guard and format-match validation; initializes _cse_added flag in __init__; well-tested.
nvflare/recipe/utils.py	Adds add_final_global_evaluation with PyTorch-only guard, idempotency check, and component-ID tracking; expands add_experiment_tracking with zero-config MLflow defaults; logic is correct and thoroughly unit-tested.
nvflare/app_opt/tracking/mlflow/mlflow_receiver.py	Switches default local tracking URI from a file:// directory store to sqlite:///mlflow.db; docstring updated to match; covered by a new unit test.
tests/integration_test/slow/experiment_tracking_recipes_test.py	Adds zero-config MLflow integration tests verifying SQLite store creation and derived experiment/run names; _add_model_to_apps was extended but still uses recipe.job directly instead of the new Recipe APIs.
tests/unit_test/recipe/spec_test.py	New TestRecipeTensorStreaming class covers idempotency guard, format mismatch, tasks validation, and default task assignment for enable_tensor_streaming.
tests/unit_test/recipe/utils_test.py	Comprehensive tests for add_final_global_evaluation and MLflow default name derivation; all cases covered.
research/auto-fl-research/tasks/cifar10/job.py	Removes local add_final_global_evaluation duplicate and replaces it with the new canonical Recipe API import.
examples/advanced/tensor-stream/job.py	Replaces direct TensorServerStreamer/TensorClientStreamer construction with recipe.enable_tensor_streaming().
examples/advanced/recipe-k8s/job.py	Migrates recipe.job.add_file_to_server / recipe.job.add_file_to to recipe.add_server_file / recipe.add_client_file.

_{Reviews (8): Last reviewed commit: "Merge branch 'main' into codex/recipe-pu..." | Re-trigger Greptile}

[codecov-commenter]

Codecov Report

❌ Patch coverage is 98.52941% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 61.06%. Comparing base (387a4c9) to head (6cc22c8).

Files with missing lines	Patch %	Lines
nvflare/recipe/utils.py	97.77%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4873      +/-   ##
==========================================
+ Coverage   60.99%   61.06%   +0.07%     
==========================================
  Files         980      980              
  Lines       93754    93818      +64     
==========================================
+ Hits        57184    57294     +110     
+ Misses      36570    36524      -46     

Flag	Coverage Δ
unit-tests	61.06% <98.52%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[YuanTingHsieh]

Changes LGTM, make sure you tested these updated examples.
And for some utils can @holgerroth and @ZiyueXu77 comment on if that util makes sense?
otherwise we could just use the recipe.add_server_component or recipe.add_client_component

[holgerroth]

A few non-blocking suggestions from code review. The overall direction is great — removing recipe.job from public examples is the right cleanup.

[holgerroth]

This makes sense. Nice cleanup — getting recipe.job out of the public examples and docs is a good step, and the new enable_tensor_streaming and add_final_global_evaluation APIs are good additions. All my review comments were addressed in the latest commits, including the matching-defaults test for the validation timeout.