feat(annotate-mlflow-trace): add trace tagging and feedback skill#20
Open
V-3604 wants to merge 2 commits into
Open
feat(annotate-mlflow-trace): add trace tagging and feedback skill#20V-3604 wants to merge 2 commits into
V-3604 wants to merge 2 commits into
Conversation
Signed-off-by: Varshith <kvarshithgowda@gmail.com>
Contributor
|
@B-Step62 can we route this to someone to look at please |
Contributor
Author
…ject Signed-off-by: Varshith <kvarshithgowda@gmail.com>
Contributor
Author
|
@B-Step62 Hi, I updated this to fit MLflow 3.14. Trace selection now scopes to the current project using the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
annotate-mlflow-tracelets a coding agent tag and leave feedback on the current session's MLflow trace from the same conversation, so sessions can be labeled for evaluation without opening the UI. A single script (trace_annotate.py) exposestag,feedback, andlist.Fixes #13
Trace selection (updated for MLflow 3.14)
tagandfeedbackdefault to the most recent trace from the current project, resolved by themlflow.trace.working_directorymetadata thatmlflow.claude_codewrites in 3.14, so a shared experiment does not surface another project's trace. Selection falls back to the most recent trace on older versions, and--trace-idtargets a specific trace. This is the agent and CLI path for writing assessments; it complements Review Queues, which is the UI path for the same data.Decisions on the open questions in #13
tag-traceandfeedback-traceshare a workflow and a script, so they are a singleannotate-mlflow-traceskill with subcommands rather than two directories.skip_skill_traces.pyis included here as an optional Stop-hook pre-filter so annotation turns do not log their own trace. A native skip list instop_hook_handleris probably the better long-term home, and I can move it there if that is preferred. The skill works without the hook; the hook only suppresses the administrative trace.Notes
Base directory for this skill:marker Claude Code injects, so it is specific to Claude Code. Other agents can still annotate by--trace-id.