Add explicit lookup.key and lookup.semantic operations

[mborodii-prog]

Related discussion: wrangleworks/Wrangles-Docs#34

What changes

Adds explicit lookup.key and lookup.semantic operations so callers can state which saved-model lookup behavior they expect. Previously, recipes used the generic lookup operation and the model's stored variant determined the behavior.

The new operations validate the model before lookup execution:

Operation	Required purpose	Required stored variant
lookup.key	lookup	key
lookup.semantic	lookup	embedding

• Exposes both operations through Python, YAML recipes, and the Wrangles DataFrame accessor.
• Adds recipe contract definitions used by the existing schema generator, including required arguments, defaults, examples, and mode constraints.
• Reuses the existing lookup implementation and /wrangles/lookup API.
• Keeps the generic lookup callable and its existing behavior available.
• Supports the existing by_row, by_dataframe, and by_matrix recipe modes. by_matrix requires matrix_variables.
• Rejects a non-null n outside by_row for the new recipe operations, instead of silently ignoring it.

Examples

Use existing saved-model execution IDs for KEY_MODEL_ID and SEMANTIC_MODEL_ID. In Excel, define these recipe variables or replace the placeholders with the actual IDs. Catalog IDs such as 101 and 102 are not execution model IDs.

The examples below use the actual field names from the manual Excel fixtures: Column1 contains values such as Bolt and Nut, and the semantic model also exposes Score. If a model stores its value under Output, use Output instead of Column1 on the left side of the mapping.

Select the input table, including headers, before running each Excel recipe.

Key lookup with a named output field

For an input column SKU containing ABC-001 and ABC-002:

wrangles:
  - lookup.key:
      input: SKU
      output:
        - Column1: Output
      model_id: ${KEY_MODEL_ID}

write:
  - excel.sheet:
      name: Test_Lookup_Key
      action: increment
      cell: A1

For the test model, Output contains Bolt and Nut as individual field values.

Semantic lookup with a value and score

wrangles:
  - lookup.semantic:
      input: Description
      output:
        - Column1: Output
        - Score: Score
      model_id: ${SEMANTIC_MODEL_ID}

write:
  - excel.sheet:
      name: Test_Lookup_Semantic
      action: increment
      cell: A1

In the manual Excel test, both steel bolt and steel bolt eight millimetre matched the stored bolt description and returned Bolt. The value and score appeared in separate columns, with all six input rows and their RowID order preserved.

Full records and multiple semantic matches

An output name that does not match a stored field receives the complete record. For example, output: MatchRecord returns a dictionary containing the model's fields rather than selecting one field.

To distribute three semantic matches into separate columns:

wrangles:
  - lookup.semantic:
      input: Description
      output: "Match *"
      model_id: ${SEMANTIC_MODEL_ID}
      n: 3

write:
  - excel.sheet:
      name: Test_Semantic_Top3
      action: increment
      cell: A1

Match 1, Match 2, and Match 3 contain match dictionaries. This uses the existing backend ranking and does not introduce a new scoring algorithm.

Direct Python calls

import os
import wrangles

key_model_id = os.environ["KEY_MODEL_ID"]
semantic_model_id = os.environ["SEMANTIC_MODEL_ID"]

values = wrangles.lookup.key(
    ["ABC-001", "ABC-002"],
    model_id=key_model_id,
    columns="Column1",
)

matches = wrangles.lookup.semantic(
    ["steel bolt", "brass nut M8"],
    model_id=semantic_model_id,
    columns=["Column1", "Score"],
)

# The existing entry point remains available.
legacy_matches = wrangles.lookup(
    ["steel bolt", "brass nut M8"],
    model_id=semantic_model_id,
    columns=["Column1", "Score"],
)

For a single match, passing one column name returns its values; passing a list of column names returns rows of values. Omitting columns returns complete records. With n > 1, matches remain dictionaries.

How it was verified

Automated

Verified against commit d8db39ba with local Python 3.12:

python -B -m pytest tests/test_lookup_variants.py tests/recipes/wrangles/test_standardize.py tests/recipes/wrangles/test_main.py::TestWrangleSchema -q -p no:cacheprovider

Result: 75 passed. These are offline tests with stubbed model metadata and API responses; they do not establish live-service or Python 3.14 CI results.

Coverage includes explicit variant validation, wrong-purpose models, legacy lookup behavior, Python return shapes, forwarded arguments and batching, all three recipe modes, output renaming, wildcard match expansion, empty inputs, where filtering, DataFrame accessors, and generated recipe schemas. The branch diff also passes git diff --check.

Manual Excel verification

• Key lookup returned the expected records for all six input rows, including duplicate SKUs.
• Semantic lookup returned the expected Bolt/Nut matches for exact and paraphrased descriptions.
• Explicit semantic field mapping produced separate Output and Score columns.
• Input row count and RowID order were preserved.
• Replacing lookup.semantic with legacy lookup returned the same Output values and row order for the tested model.

The manual checks above are a subset of the prepared test plan. They do not claim that the full Excel suite, including top-n and negative cases, has passed.

Compatibility and risk

• Both new operations require an existing lookup model and the exact stored variant shown above. Missing/null variants and the literal variant semantic are rejected by the new names; generic lookup retains its existing handling.
• Semantic models are created with stored variant=embedding; the operation name is lookup.semantic.
• Model IDs, training data, API endpoints, database schema, and catalog allocation are unchanged by this PR. Registry and standalone documentation updates remain separate work.
• Excel must execute a WranglesPY version containing this change. A local branch alone does not update the deployed recipe runtime. Existing model permissions and service credentials still apply.
• Rollback: revert the code change and use generic lookup in recipes that adopted the explicit names. Existing model data needs no migration.

Remaining verification and known limitations

• One manual comparison showed Score=1.001 for the explicit semantic call and Score=1 for legacy lookup on the first row; the other displayed scores and all output labels matched. Both paths share the same API implementation, but the cause of this observed difference has not been established. Exact score parity remains unverified.
• An inherited recipe limitation remains: n > 1 with a single named model-field output can raise Columns must be same length as key. It was reproduced offline with both generic and explicit lookup. Use a wildcard such as Match * or an unnamed output such as Matches for that case.

Ready-for-review checklist

• One human delivery owner is assigned
• The linked issue and intended milestone are correct
• The branch is current with main and has no merge conflicts
• Focused tests pass
• New or changed behavior has direct test coverage
• Documentation/schema/configuration is updated where applicable: recipe contract definitions and schema-generation tests are included
• The PR contains no unrelated changes
• The PR description reflects the branch's current scope and latest validation
• The observed semantic score difference is investigated or explicitly accepted as remaining work
• One primary reviewer is requested only when this PR is ready