Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/agents/test.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ You are a **Software Development Engineer in Test (SDET)** for a Python data wra
## 📚 Project Knowledge

### Tech Stack
- **Core:** Python 3.11–3.13, pytest, pandas, numpy, polars
- **Core:** Python 3.11–3.14, pytest, pandas, numpy, polars
- **CI coverage:** Main-targeted PRs and main pushes test Python 3.11, 3.13, and 3.14 on Ubuntu and Windows; tagged releases test Python 3.13 and 3.14 on Ubuntu and macOS.
- **Development baseline:** The standard local environment, development container, and CI test image use Python 3.13.
- **Data:** SQLAlchemy, boto3, pymongo, Pydantic
- **Templates:** Jinja2

Expand Down Expand Up @@ -311,4 +313,4 @@ Tests should:

Every test you write is an investment in the project's long-term maintainability and reliability.

---
---
11 changes: 7 additions & 4 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

## Tech Stack

- **Python:** 3.11, 3.12, 3.13 (multi-version support)
- **Python:** 3.11, 3.12, 3.13, 3.14 (package support); the standard local development environment uses Python 3.13
- **Core Dependencies:** pandas (>=2.0,<3.0), numpy, polars (1.33.0), pyyaml
- **Database Connectors:** sqlalchemy, pymssql, psycopg2-binary, pymysql, pymongo
- **Cloud/External:** boto3 (AWS S3), simple-salesforce, fabric (SFTP)
Expand Down Expand Up @@ -243,7 +243,7 @@ current `main`; never merge `dev` wholesale into `main`.

**Pushing to a feature branch currently runs nothing.** Open a pull request to
`main` (Draft is fine) to trigger `ci.yml`. A PR to `main` runs the full Ubuntu
and Windows, Python 3.11 and 3.13 matrix.
and Windows, Python 3.11, 3.13, and 3.14 matrix.

PR #1115 established the current workflows, but some triggers still mention the
legacy `dev` branch. That support is transitional compatibility and does not
Expand Down Expand Up @@ -281,8 +281,8 @@ to define.
### GitHub Actions Workflows
- **ci.yml** (*CI*)**:** PRs into `main` and pushes to `main`; legacy `dev`
triggers remain temporarily while recovery is completed
- Pytest on Ubuntu + Windows across Python 3.11 + 3.13 for `main` PRs
- Test pip installation
- Pytest on Ubuntu + Windows across Python 3.11 + 3.13 + 3.14 for `main` PRs and pushes
- Test pip installation across the same matrix
- Generate and test JSON schema
- Build the CI test image and, on PRs, run smoke checks and local recipes against that exact image
- Push the image on merges to `main` under the new policy
Expand All @@ -293,6 +293,9 @@ to define.
- **publish-tagged.yml** (*Deploy Prod*)**:** `v*` tag push. GHCR, then
CodeArtifact, then PyPI. The file name is pinned by PyPI Trusted Publishing
and cannot be renamed without updating the publisher on PyPI first.
- Pytest on Ubuntu + macOS across Python 3.13 + 3.14
- Test pip installation on Ubuntu across Python 3.13 + 3.14
- Validate the Python 3.13 CI test image before package publication

### Workflow Jobs
1. **pytest:** Run test suite across OS/Python matrix
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
FULL=false
[[ "$GITHUB_REF_NAME" == "main" || "$BASE_REF" == "main" ]] && FULL=true
if [[ "$FULL" == "true" ]]; then
echo 'matrix={"os":["ubuntu-latest","windows-latest"],"python-version":["3.11","3.13"]}' >> "$GITHUB_OUTPUT"
echo 'matrix={"os":["ubuntu-latest","windows-latest"],"python-version":["3.11","3.13","3.14"]}' >> "$GITHUB_OUTPUT"
Comment thread
thomasstvr marked this conversation as resolved.
else
echo 'matrix={"os":["ubuntu-latest"],"python-version":["3.11"]}' >> "$GITHUB_OUTPUT"
fi
Expand Down
15 changes: 12 additions & 3 deletions .github/workflows/publish-tagged.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@ jobs:
pytest:
runs-on: ubuntu-latest
needs: [validate]
strategy:
matrix:
python-version: ["3.13", "3.14"]
permissions:
contents: read
steps:
Expand All @@ -99,7 +102,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
python-version: ${{ matrix.python-version }}

- name: Install Dependencies
run: |
Expand All @@ -124,6 +127,9 @@ jobs:
test-pip-install:
runs-on: ubuntu-latest
needs: [validate]
strategy:
matrix:
python-version: ["3.13", "3.14"]
permissions:
contents: read
steps:
Expand All @@ -133,7 +139,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
python-version: ${{ matrix.python-version }}

- name: Test pip install
run: |
Expand All @@ -144,6 +150,9 @@ jobs:
pytest-macos:
runs-on: macos-latest
needs: [validate]
strategy:
matrix:
python-version: ["3.13", "3.14"]
permissions:
contents: read
steps:
Expand All @@ -153,7 +162,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.13"
python-version: ${{ matrix.python-version }}

- name: Install Dependencies
run: |
Expand Down
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,18 @@

Full documentation available at [wrangles.io](https://wrangles.io/python).

## Python support

Wrangles supports Python 3.11, 3.12, 3.13, and 3.14. Pull requests targeting
`main` and pushes to `main` run tests and package-install checks on Python 3.11,
3.13, and 3.14 on Ubuntu and Windows. Tagged releases run tests on Python 3.13
and 3.14 on Ubuntu and macOS, plus package-install checks on Ubuntu.

## Local development

Supported local development uses Python 3.13. On Windows, create or refresh the
complete test and tooling environment with one command from the repository
root:
The standard local development environment uses Python 3.13. On Windows,
create or refresh the complete test and tooling environment with one command
from the repository root:

```powershell
.\scripts\bootstrap-dev.ps1
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent'
],
Expand Down
27 changes: 27 additions & 0 deletions tests/test_ai_definition.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import pandas as pd
import pytest
import wrangles
from pydantic import ValidationError

import wrangles.ai_definition as ai_definition
import wrangles.ai_cache as ai_cache
Expand Down Expand Up @@ -334,6 +335,32 @@ def test_outputs_are_nullable_by_default_but_keys_remain_required():
)


@pytest.mark.parametrize(
"field_schema",
[
{"type": ["string", "integer", "null"]},
{"anyOf": [{"type": "string"}, {"type": "integer"}, {"type": "null"}]},
],
ids=["type-list", "anyOf"],
)
@pytest.mark.parametrize("value", ["25mm", 25, None, [], {}])
def test_structured_output_union_validation(field_schema, value):
"""Union fields must build and validate on every supported Python version."""
schema = {
"type": "object",
"properties": {"value": field_schema},
"required": ["value"],
"additionalProperties": False,
}
parsed = {"value": value}

if isinstance(value, (list, dict)):
with pytest.raises(ValidationError):
openai_responses.validate_structured_output(parsed, schema)
else:
assert openai_responses.validate_structured_output(parsed, schema) == parsed


def test_nullable_false_is_an_explicit_opt_out():
compiled = ai_definition.compile_definition(
{
Expand Down
4 changes: 2 additions & 2 deletions wrangles/openai_responses.py
Original file line number Diff line number Diff line change
Expand Up @@ -429,7 +429,7 @@ def schema_type_to_python(schema: dict, name: str) -> _Any:
if isinstance(option, dict)
)
if python_types:
return _Union.__getitem__(python_types)
return _Union[python_types]

if schema.get("enum"):
return _Literal.__getitem__(tuple(schema["enum"]))
Expand All @@ -440,7 +440,7 @@ def schema_type_to_python(schema: dict, name: str) -> _Any:
schema_type_to_python({**schema, "type": item}, name)
for item in schema_type
)
return _Union.__getitem__(python_types)
return _Union[python_types]

if schema_type == "array":
return _List[schema_type_to_python(schema.get("items", {}), f"{name}Item")]
Expand Down
Loading