Thank you for improving TACO-Benchmark. This project welcomes documentation fixes, structural cleanup, new baselines, and bug fixes.
- Read docs/ARCHITECTURE.md for repository layout and change boundaries.
- Read docs/README.md for the documentation layering convention.
- Install the environment:
python scripts/setup_env.py→taco data download.
| Layer | Location | Write about |
|---|---|---|
| User guides | docs/*.md |
Setup, workflows, CLI, concepts |
| Module refs | **/README.md next to code |
Script args, paths, module-specific notes |
| Reviewer / paper | experiments/docs/ |
Fairness, prompts, reviewer notes |
Avoid duplicating the same quick-start block in both docs/ and module READMEs — link instead.
- Python 3.10+, English comments and docstrings.
- Semantic script names in
benchmark/generation/; use thin shims when renaming. - Paths — prefer
taco.core.pathsorPath(__file__)-based resolution; no hardcoded home directories. - Secrets — never commit real API keys; use
*.exampletemplates. - Artifacts — do not add generation intermediate files (graphs, large JSON) to git.
- Changes match the scope of the PR (no unrelated refactors)
- No secrets or large generated files in the diff
- User-facing behavior changes are documented in
docs/ - Module-only changes update the relevant
README.md -
python -m compileall taco benchmark/generation experimentspasses
source .venv/bin/activate
python -m compileall taco benchmark/generation experiments -q
taco data verify # after downloading data
bash examples/quick_eval.sh # optional; requires API keyOpen a GitHub issue for bugs, documentation gaps, or design discussions.
By contributing, you agree that your contributions will be released under the MIT License.