diff --git a/registry/CONTRACT.md b/registry/CONTRACT.md index 94476f6..a36f135 100644 --- a/registry/CONTRACT.md +++ b/registry/CONTRACT.md @@ -1,6 +1,6 @@ # Wrangles Registry Contract -Status: pilot +Status: pre-production Contract version: 0.1 @@ -45,7 +45,7 @@ manifest and any normalized Registry entry. Aggregate category pages, the index, and the template are inventoried separately as supporting Markdown; they are not treated as one-to-one wrangle records. -For the first migration pass, conflicting or incomplete source content is +During the initial migration, conflicting or incomplete source content is resolved in this order: 1. WranglesPY callable signature, implementation, and tests @@ -83,10 +83,10 @@ required for Registry inclusion. Its signature and tests may be used for parity checks, but the Registry pipeline must not depend on finding one. Registry Markdown keeps the full parameter records, including descriptions, -roles, accepted-value constraints that cannot be inferred reliably from Python -annotations, and examples. The compiler merges those records with the runtime -manifest, rejects name/required/default drift, and generates the recipe JSON -Schema from the merged result. +presentation groups, accepted-value constraints that cannot be inferred +reliably from Python annotations, and examples. The compiler merges those +records with the runtime manifest, rejects name/required/default drift, and +generates the recipe JSON Schema from the merged result. The current embedded JSON Schema docstrings in `recipe_wrangles` are migration input, not a second permanent authoring source. Their complete overview and @@ -157,7 +157,7 @@ Each entry separates identity, executable naming, compatibility, and routing: - `slug` is the public documentation path below `/wrangles/`. Changing it requires a redirect. -`aliases` contains still-supported legacy recipe keys. The first-pass Registry +`aliases` contains still-supported legacy recipe keys. The initial Registry keeps `classify`, `lookup`, and other ungrouped keys exactly as the current code exposes them. When the dotted migration is implemented, the new key can become canonical without changing `id`, and the former key can move to `aliases` for @@ -197,6 +197,22 @@ Parameter records contain human guidance plus a JSON Schema fragment. Every runtime-owned parameter fact must reconcile with the pinned manifest and be traceable to the named runtime symbol. +Every parameter also declares one `param_group` from a small shared vocabulary: + +- `I/O` identifies input and output columns or structures. +- `Options` contains wrangle-specific settings users need to achieve the + desired behavior. +- `Formatting` controls the shape or presentation of results. +- `Conditions` contains the shared `if`, `where`, and `where_params` controls. +- `Execution` controls batching, concurrency, time limits, and nested execution. +- `Errors` controls fallbacks, retries, and failure handling. +- `Details` contains model, provider, connection, credential, cache, and other + supporting technical settings. + +The groups are intentionally fixed rather than specialized by wrangle. They +organize the human documentation and remain available to agent and product +consumers in the compiled JSON contracts. + `runtime_default` means the value used when the argument is omitted. It is not a playground placeholder. Example or UI starter values belong in examples, not in `runtime_default`. @@ -222,7 +238,7 @@ verification level: - `live`: a separate controlled job executes an external service. - `manual`: the example requires a documented human check. -The pilot compiler implements `static` verification. The entries remain +The compiler currently implements `static` verification. The entries remain `static` until an automated WranglesPY execution check is added; output fixtures do not by themselves constitute execution evidence. @@ -241,8 +257,8 @@ The compiler produces: - one comprehensive JSON contract per wrangle under `wrangles-docs/static/registry/contracts/`, including all parameters - raw source Markdown and sanitized example fixtures -- a pilot recipe JSON Schema under - `wrangles-docs/static/schemas/recipes/pilot/schema.json` +- a pre-production recipe JSON Schema under + `wrangles-docs/static/schemas/recipes/registry/schema.json` - deterministic JSON and Markdown reconciliation reports under `registry/reports/` @@ -256,7 +272,8 @@ and environment-specific values are excluded. A clean compile followed by ## Versioning and lifecycle -The pilot Registry version is `0.1.0-pilot`. A production release will contain: +The pre-production Registry version is `0.1.0`. A production release will +contain: - an immutable Registry version - the compatible WranglesPY version or version range @@ -265,8 +282,12 @@ The pilot Registry version is `0.1.0-pilot`. A production release will contain: - a convenience `latest` alias that is not used for reproducible execution Wrangles progress through `draft`, `active`, `deprecated`, and `removed`. -Deprecated records stay discoverable and identify their replacement. Removed -records remain in versioned artifacts needed to understand historical recipes. +Deprecated records stay discoverable and must identify their canonical +replacement with `replaced_by`. The compiler rejects a missing, self-referential, +or unknown replacement key and generates a visible migration notice in the +human documentation. Deprecated entries sort after active entries within their +documentation group. Removed records remain in versioned artifacts needed to +understand historical recipes. ## Change workflow @@ -277,17 +298,31 @@ the database projection are invalid. The current wiki and database extraction scripts are migration inputs. Once the cutover is complete, they must not remain ongoing authoring paths. -## Pilot exit criteria - -The pilot is complete when: - -1. All callable recipe wrangles compile into deterministic Markdown, manifest, - and recipe-schema artifacts; curated entries are preserved during bootstrap. -2. Invalid frontmatter, duplicate keys, invalid recipes, missing fixtures, and - stale generated outputs fail CI. -3. The pages build successfully in Docusaurus at stable per-wrangle routes. -4. A pinned WranglesPY contract manifest is reconciled with all existing - per-wrangle quasi-registry Markdown and the normalized pilot entries. The - report distinguishes records awaiting normalization from runtime wrangles - that genuinely lack Markdown, and preserves embedded-schema differences. -5. An offline example runner is designed for the next implementation slice. +## Production readiness + +The contract remains `pre-production`. The detailed migration sequence is in +[README.md](README.md#migration-plan), and production requires all of the +following outcomes: + +1. Every callable recipe wrangle and public parameter reconciles with a pinned + WranglesPY runtime manifest, with no unexplained key, required-state, + default, common-control, or accepted-value conflicts. +2. Registry records contain reviewed descriptions, constraints, examples, + access and lifecycle metadata, provenance, and canonical database UUIDs. +3. The entry schema, compiled contract, recipe-schema URLs, compatibility + policy, and change-management rules are stable and versioned. +4. Recipe JSON Schema and any temporary WranglesPY `_schema` compatibility view + are generated from the Registry and protected by deterministic parity checks. +5. WranglesXL, VS Code, APIs, the docs site, and Recipe Writer clients consume + the Registry-generated contracts instead of hand-maintained schema copies. +6. Hand-maintained `_schema` docstrings have been removed, or only a generated + compatibility view remains for explicitly supported legacy readers. +7. Local examples have offline execution verification, and service-backed + examples have controlled live verification. +8. Invalid source, duplicate keys, invalid recipes, missing fixtures, runtime + drift, and stale generated outputs fail CI. +9. Registry pages build at stable routes, and immutable public artifacts have + been deployed and verified at canonical URLs. +10. Ownership, release, rollback, and deprecation procedures are documented; + the first production Registry version is tagged before this status changes + to `production`. diff --git a/registry/README.md b/registry/README.md index 26097c9..ea62808 100644 --- a/registry/README.md +++ b/registry/README.md @@ -4,16 +4,35 @@ This directory is the authoring source for the Wrangles knowledge registry. It contains curated, reviewable Markdown records and the small supporting schemas and fixtures needed to compile those records into product artifacts. -Start with [CONTRACT.md](CONTRACT.md). The first-pass pilot covers all callable -recipe wrangles reported by the pinned WranglesPY runtime manifest. Three -entries (`convert.case`, `convert.data_type`, and `convert.from_json`) remain -hand-curated; the migration bootstrap creates the rest without overwriting -those records. +Start with [CONTRACT.md](CONTRACT.md). The Registry currently covers all +callable recipe wrangles reported by the pinned WranglesPY runtime manifest. +Every existing Registry record is authoritative editorial content; +`bootstrap:registry` creates missing records but never overwrites an existing +one. The generated outputs are intentionally committed so changes to public docs, the agent-facing bundle, and recipe hinting can be reviewed in the same pull request as their source records. +## Source and generated directories + +The similarly named Registry directories have different responsibilities: + +| Directory | Responsibility | Edit directly? | +| --- | --- | --- | +| `registry/wrangles/` | Authoritative Markdown records for wrangle metadata, parameters, guidance, examples, and lifecycle | Yes | +| `wrangles-docs/registry-docs/` | Docusaurus pages generated from the authoritative records | No | +| `wrangles-docs/static/registry/` | Public machine-readable Markdown, JSON contracts, schemas, fixtures, and discovery metadata | No | + +The normal flow is therefore: + +```text +registry/wrangles/ + -> npm run compile:registry +wrangles-docs/registry-docs/ (human-facing pages) +wrangles-docs/static/registry/ (machine-facing artifacts) +``` + `runtime/wranglespy.json` is the pinned mechanical contract exported from the WranglesPY recipe namespace. Its producer-owned schema is copied alongside the Registry schema under `schema/`. The compiler reconciles the manifest with the @@ -35,17 +54,103 @@ npm run check:registry npm run build ``` -`bootstrap:registry` creates only missing Registry records. Use -`refresh:registry-bootstrap` to regenerate the non-curated first-pass records -after updating the pinned runtime manifest or migration logic. +`bootstrap:registry` creates only missing Registry records. It deliberately +does not regenerate existing records because `registry/wrangles/` is the +authoritative editorial source. -`compile:registry` produces the pilot Docusaurus pages and public raw +`compile:registry` produces the Docusaurus pages and public raw artifacts. `check:registry` fails when the source is invalid or the committed generated files are stale. +## Migration Plan + +The documentation-source migration and the recipe-schema migration are +separate activities. Neither is the same as deprecating or removing a callable +wrangle from the product. + +Progress is tracked in +[#27: Complete Docs Registry migration and production cutover](https://github.com/wrangleworks/Wrangles-Docs/issues/27). + +### Retiring Baver's local source files + +The committed files under `wrangles-docs/wrangle-docs/**/_sources/` came from +the previous documentation workflow. They are currently retained as migration +input and reconciliation evidence; they are not the ongoing authoring source. + +Complete this migration in the following order: + +1. Inventory every `_sources` file and confirm that useful descriptions, + examples, access requirements, and provenance are represented in + `registry/wrangles/` or intentionally omitted. +2. Review the Registry-generated pages and establish the final routes and any + required redirects from the previous documentation pages. +3. Remove `_sources` from Registry reconciliation and retire the old + `_sources`-based generation and synchronization workflow, including + `.github/agents/sync.md`. +4. Preserve a tagged or archived rollback snapshot, deploy the Registry pages, + and then remove the obsolete `_sources` files. + +This activity is complete when no supported build, compiler, or authoring +workflow reads from or writes to `_sources`. + +### Migrating the schema from WranglesPY to the Docs Registry + +The embedded `_schema` JSON Schema docstrings in WranglesPY remain migration +input today. Runtime behavior and callable signatures continue to be owned by +WranglesPY; descriptions, parameter guidance, accepted-value constraints, and +examples move to `registry/wrangles/`. + +The Registry contract remains **pre-production** until these steps are +complete: + +1. Reconcile every callable wrangle and public parameter against a pinned + WranglesPY runtime manifest, resolving every unexplained key, required-state, + default, common-control, and accepted-value conflict. +2. Complete the Registry descriptions, constraints, examples, access metadata, + lifecycle metadata, and provenance; assign canonical database UUIDs where + they are still missing. +3. Stabilize and version the Registry entry schema, compiled contract, and + recipe-schema URLs, including compatibility and change-management rules. +4. Generate the recipe JSON Schema and, where still needed, a compatible + WranglesPY `_schema` view from the Registry; add deterministic parity checks + so manually maintained copies cannot drift. +5. Cut over supported consumers—including WranglesXL, VS Code, APIs, the docs + site, and Recipe Writer clients—to the Registry-generated schema and + contracts. +6. Remove the hand-maintained WranglesPY `_schema` content, or retain only a + generated compatibility view until every supported direct reader has + migrated. +7. Add offline execution verification for local examples and controlled live + verification for service-backed examples. +8. Publish and verify immutable Registry artifacts at their canonical public + URLs, document ownership and release procedures, and tag the first + production Registry release. + +After all eight steps are complete, change the contract status from +`pre-production` to `production`. Until then, generated artifacts are suitable +for integration and review but are not the final production authority for all +consumers. + +## Deprecating or removing a callable wrangle + +This is the lifecycle process for an executable recipe key. It is unrelated to +retiring `_sources` or migrating the WranglesPY `_schema` content. + +1. Add or verify the canonical replacement in WranglesPY, retain the old key + for the supported compatibility period, and test that both paths behave as + intended. +2. Export the refreshed WranglesPY runtime manifest, then set the old Registry + record to `status: deprecated` and add `replaced_by: `. +3. Run `compile:registry`, `check:registry`, and `build`. The compiler validates + the replacement and automatically labels, links, and sorts the deprecated + entry in both human and machine artifacts. + +Do not model a deprecated callable as an alias of its replacement: keeping its +own Registry record preserves compatibility and historical recipe discovery. + Do not edit these generated directories by hand: - `wrangles-docs/registry-docs/` - `wrangles-docs/static/registry/` -- `wrangles-docs/static/schemas/recipes/pilot/` +- `wrangles-docs/static/schemas/recipes/registry/` - `registry/reports/` diff --git a/registry/common/wrangle-controls.yaml b/registry/common/wrangle-controls.yaml index 00712e3..44f51dd 100644 --- a/registry/common/wrangle-controls.yaml +++ b/registry/common/wrangle-controls.yaml @@ -3,11 +3,13 @@ controls: where: description: Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. required: false + param_group: Conditions schema: type: string where_params: description: Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. required: false + param_group: Conditions schema: type: - array @@ -15,5 +17,6 @@ controls: if: description: Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. required: false + param_group: Conditions schema: type: string diff --git a/registry/index.md b/registry/index.md index b9daaab..ea95487 100644 --- a/registry/index.md +++ b/registry/index.md @@ -3,7 +3,7 @@ okf_version: "0.2" type: collection title: Wrangles Registry description: Versioned knowledge and machine contracts for Wrangles recipe primitives. -status: pilot +status: pre-production tags: - wrangles - recipes @@ -15,6 +15,7 @@ tags: The Registry describes the recipe vocabulary shared by WranglesPY, WranglesXL, the documentation site, and future Recipe Writer clients. -The pilot contains three Convert wrangles. Generated indexes and manifests -should be used for discovery; individual files provide the detailed contract, -guidance, examples, provenance, and lifecycle state. +The Registry contains the callable recipe wrangles reported by the pinned +WranglesPY runtime manifest. Generated indexes and manifests should be used for +discovery; individual files provide the detailed contract, guidance, examples, +provenance, and lifecycle state. diff --git a/registry/reports/runtime-reconciliation.json b/registry/reports/runtime-reconciliation.json index 342065e..7caecbb 100644 --- a/registry/reports/runtime-reconciliation.json +++ b/registry/reports/runtime-reconciliation.json @@ -1,7 +1,7 @@ { "format": "wrangles-registry-reconciliation", "format_version": "0.2", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "runtime_source": { "repository": "https://github.com/wrangleworks/WranglesPY", "revision": "7916bf158e8b7e561270a1bea7b808f88956edc4" diff --git a/registry/reports/runtime-reconciliation.md b/registry/reports/runtime-reconciliation.md index 509243a..d9fb46c 100644 --- a/registry/reports/runtime-reconciliation.md +++ b/registry/reports/runtime-reconciliation.md @@ -3,7 +3,7 @@ Generated file. Do not edit directly. - Runtime source: [`7916bf158e8b7e561270a1bea7b808f88956edc4`](https://github.com/wrangleworks/WranglesPY/commit/7916bf158e8b7e561270a1bea7b808f88956edc4) -- Registry version: `0.1.0-pilot` +- Registry version: `0.1.0` - Runtime entries: 98 - Embedded Python schema docstrings: 96 - Existing quasi-registry records: 93 diff --git a/registry/schema/wrangle-entry.schema.json b/registry/schema/wrangle-entry.schema.json index bfaa13b..a1380e9 100644 --- a/registry/schema/wrangle-entry.schema.json +++ b/registry/schema/wrangle-entry.schema.json @@ -63,6 +63,11 @@ "status": { "enum": ["draft", "active", "deprecated", "removed"] }, + "replaced_by": { + "type": "string", + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)*$", + "description": "Canonical wrangle_key that replaces a deprecated or removed entry." + }, "visibility": { "enum": ["public", "internal"] }, @@ -118,11 +123,20 @@ "items": {"$ref": "#/$defs/source"} } }, + "allOf": [ + { + "if": { + "properties": {"status": {"const": "deprecated"}}, + "required": ["status"] + }, + "then": {"required": ["replaced_by"]} + } + ], "$defs": { "parameter": { "type": "object", "additionalProperties": false, - "required": ["name", "description", "required", "role", "schema"], + "required": ["name", "description", "required", "param_group", "schema"], "properties": { "name": { "type": "string", @@ -135,7 +149,18 @@ }, "description": {"type": "string", "minLength": 1}, "required": {"type": "boolean"}, - "role": {"type": "string", "minLength": 1}, + "param_group": { + "type": "string", + "enum": [ + "I/O", + "Options", + "Formatting", + "Conditions", + "Execution", + "Errors", + "Details" + ] + }, "runtime_default": {}, "schema": {"type": "object", "minProperties": 1} } diff --git a/registry/wrangles/_root/accordion.md b/registry/wrangles/_root/accordion.md index 73e57e8..163025b 100644 --- a/registry/wrangles/_root/accordion.md +++ b/registry/wrangles/_root/accordion.md @@ -33,7 +33,7 @@ parameters: - name: wrangles description: List of wrangles to apply. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -42,7 +42,7 @@ parameters: - name: input description: The column(s) containing the list(s) that the wrangles will be applied to the elements of. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -51,7 +51,7 @@ parameters: - name: output description: Output of the wrangles to save back to the dataframe. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -64,7 +64,7 @@ parameters: not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets. required: false - role: option + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/_root/batch.md b/registry/wrangles/_root/batch.md index f17bbec..7794310 100644 --- a/registry/wrangles/_root/batch.md +++ b/registry/wrangles/_root/batch.md @@ -35,7 +35,7 @@ parameters: The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -44,14 +44,14 @@ parameters: - name: batch_size description: The number of rows to split each batch into. required: false - role: option + param_group: Execution runtime_default: 1000 schema: type: integer - name: threads description: The number of threads to use for parallel processing. Default 1. required: false - role: option + param_group: Execution runtime_default: 1 schema: type: integer @@ -60,7 +60,7 @@ parameters: A dictionary of column_name: value to return if an error occurs while attempting to run a batch. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: @@ -69,7 +69,7 @@ parameters: - name: timeout description: The number of seconds to wait for a batch to complete before raising an error. required: false - role: option + param_group: Execution runtime_default: null schema: type: @@ -78,7 +78,7 @@ parameters: - name: use_multiprocessing description: Use process-based workers instead of threads. This is an advanced runtime option. required: false - role: option + param_group: Execution runtime_default: false schema: type: boolean diff --git a/registry/wrangles/_root/classify.md b/registry/wrangles/_root/classify.md index 9dffa3c..498ab61 100644 --- a/registry/wrangles/_root/classify.md +++ b/registry/wrangles/_root/classify.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,13 +48,13 @@ parameters: - name: model_id description: ID of the classification model to be used. required: true - role: model-reference + param_group: Details schema: type: string - name: include_confidence description: For models that support it, include the confidence level in the output. required: false - role: option + param_group: Formatting schema: type: boolean examples: [] diff --git a/registry/wrangles/_root/clean-whitespaces.md b/registry/wrangles/_root/clean-whitespaces.md index 5be4c43..0a76469 100644 --- a/registry/wrangles/_root/clean-whitespaces.md +++ b/registry/wrangles/_root/clean-whitespaces.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,14 +52,14 @@ parameters: - name: trim description: Whether to trim leading and trailing spaces. Default True. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: remove_literals description: Whether to remove special space characters such as new lines etc. Default True. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/registry/wrangles/_root/concurrent.md b/registry/wrangles/_root/concurrent.md index 05a1f18..ce5187f 100644 --- a/registry/wrangles/_root/concurrent.md +++ b/registry/wrangles/_root/concurrent.md @@ -34,7 +34,7 @@ parameters: - name: wrangles description: The wrangles section of a recipe to execute for each combination of variables. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -43,7 +43,7 @@ parameters: - name: max_concurrency description: The maximum number of wrangles to execute in parallel. required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer @@ -51,7 +51,7 @@ parameters: - name: use_multiprocessing description: Use process-based workers instead of threads. This is an advanced runtime option. required: false - role: option + param_group: Execution runtime_default: false schema: type: boolean diff --git a/registry/wrangles/_root/copy.md b/registry/wrangles/_root/copy.md index f9d8764..34c320a 100644 --- a/registry/wrangles/_root/copy.md +++ b/registry/wrangles/_root/copy.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input columns or columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output columns or columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/_root/date-calculator.md b/registry/wrangles/_root/date-calculator.md index d0d7987..ba25a44 100644 --- a/registry/wrangles/_root/date-calculator.md +++ b/registry/wrangles/_root/date-calculator.md @@ -31,13 +31,13 @@ parameters: - name: input description: Name of the dates column. required: true - role: column-selector + param_group: I/O schema: type: string - name: operation description: Date operation. required: false - role: option + param_group: Options runtime_default: add schema: type: string @@ -47,7 +47,7 @@ parameters: - name: output description: Name of the output column of dates. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -56,7 +56,7 @@ parameters: - name: time_unit description: Time unit for operation. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -74,7 +74,7 @@ parameters: - name: time_value description: Time unit value for operation. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/drop.md b/registry/wrangles/_root/drop.md index f27d9ec..f24ac35 100644 --- a/registry/wrangles/_root/drop.md +++ b/registry/wrangles/_root/drop.md @@ -31,7 +31,7 @@ parameters: - name: columns description: Name of the column(s) to drop. required: true - role: option + param_group: I/O schema: type: - string diff --git a/registry/wrangles/_root/explode.md b/registry/wrangles/_root/explode.md index f743d1c..2b18fc3 100644 --- a/registry/wrangles/_root/explode.md +++ b/registry/wrangles/_root/explode.md @@ -33,7 +33,7 @@ parameters: Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: reset_index description: Reset the index after exploding. Default True. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean @@ -51,7 +51,7 @@ parameters: If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/_root/filter.md b/registry/wrangles/_root/filter.md index 453b311..d95698d 100644 --- a/registry/wrangles/_root/filter.md +++ b/registry/wrangles/_root/filter.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column to filter on. If multiple are provided, all must match the criteria. required: false - role: column-selector + param_group: I/O runtime_default: [] schema: type: @@ -43,7 +43,7 @@ parameters: - name: equal description: Select rows where the values equal a given value. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: not_equal description: Select rows where the values do not equal a given value. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -63,7 +63,7 @@ parameters: - name: is_in description: Select rows where the values are in a given list. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -73,7 +73,7 @@ parameters: - name: not_in description: Select rows where the values are not in a given list. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -85,7 +85,7 @@ parameters: Select rows where the values are greater than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -97,7 +97,7 @@ parameters: Select rows where the values are greater than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -109,7 +109,7 @@ parameters: Select rows where the values are less than a specified value. Does not include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -119,7 +119,7 @@ parameters: - name: less_than_equal_to description: Select rows where the values are less than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -129,7 +129,7 @@ parameters: - name: between description: Value or list of values to filter that are in between two parameter values. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -138,7 +138,7 @@ parameters: - name: contains description: Select rows where the input contains the value. Allows regular expressions. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -147,7 +147,7 @@ parameters: - name: not_contains description: Select rows where the input does not contain the value. Allows regular expressions. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -156,7 +156,7 @@ parameters: - name: is_null description: If true, select all rows where the value is NULL. If false, where is not NULL. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/huggingface.md b/registry/wrangles/_root/huggingface.md index f684cc6..1d058ce 100644 --- a/registry/wrangles/_root/huggingface.md +++ b/registry/wrangles/_root/huggingface.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,19 +40,19 @@ parameters: - name: api_token description: Huggingface API Token. required: true - role: option + param_group: Details schema: type: string - name: model description: Name of the model to use. e.g. facebook/bart-large-cnn. required: true - role: option + param_group: Details schema: type: string - name: output description: Name of the output column. If not provided, will overwrite the input column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -62,7 +62,7 @@ parameters: - name: parameters description: Optionally, provide additional parameters to define the model behaviour. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/log.md b/registry/wrangles/_root/log.md index de9c3af..55f78ab 100644 --- a/registry/wrangles/_root/log.md +++ b/registry/wrangles/_root/log.md @@ -31,7 +31,7 @@ parameters: - name: columns description: (Optional, default all columns) List of specific columns to log. required: false - role: option + param_group: I/O runtime_default: null schema: type: @@ -40,7 +40,7 @@ parameters: - name: write description: (Optional) Allows for an intermediate output to a file/dataframe/database etc. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: warning description: Log a warning to the console. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -61,7 +61,7 @@ parameters: - name: info description: Log info to the console. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -81,7 +81,7 @@ parameters: - name: error description: Log an error to the console. required: false - role: option + param_group: Options schema: type: string examples: [] diff --git a/registry/wrangles/_root/lookup.md b/registry/wrangles/_root/lookup.md index d219a3d..13652cb 100644 --- a/registry/wrangles/_root/lookup.md +++ b/registry/wrangles/_root/lookup.md @@ -30,7 +30,7 @@ parameters: - name: input description: Name of the column(s) to lookup. required: true - role: column-selector + param_group: I/O schema: type: string - name: output @@ -39,7 +39,7 @@ parameters: output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. "Top *" with n: 3 becomes "Top 1", "Top 2", "Top 3". required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -49,7 +49,7 @@ parameters: - name: model_id description: The model_id to use lookup against. required: false - role: model-reference + param_group: Details runtime_default: null schema: type: @@ -61,7 +61,7 @@ parameters: lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation. required: false - role: option + param_group: Options runtime_default: by_row schema: type: string @@ -75,7 +75,7 @@ parameters: column receives the corresponding match. Otherwise all n matches are stored as a list in each output column. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/math.md b/registry/wrangles/_root/math.md index 14bb1de..9e6df49 100644 --- a/registry/wrangles/_root/math.md +++ b/registry/wrangles/_root/math.md @@ -33,13 +33,13 @@ parameters: The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_). required: true - role: column-selector + param_group: I/O schema: type: string - name: output description: The column to output the results to. required: true - role: column-output + param_group: I/O schema: type: string examples: [] diff --git a/registry/wrangles/_root/maths.md b/registry/wrangles/_root/maths.md index 320743b..188e429 100644 --- a/registry/wrangles/_root/maths.md +++ b/registry/wrangles/_root/maths.md @@ -10,6 +10,7 @@ wrangle_key: maths aliases: [] slug: maths status: deprecated +replaced_by: math visibility: public tags: - maths @@ -30,13 +31,13 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: string - name: output description: Name or list of output columns. required: true - role: column-output + param_group: I/O schema: type: string examples: [] @@ -53,4 +54,4 @@ sources: Deprecated alias for `math`; evaluate an expression and write its result to an output column. -This first-pass guidance is derived from the callable signature. +This guidance was derived from the callable signature. diff --git a/registry/wrangles/_root/matrix.md b/registry/wrangles/_root/matrix.md index efe654b..5394e36 100644 --- a/registry/wrangles/_root/matrix.md +++ b/registry/wrangles/_root/matrix.md @@ -35,7 +35,7 @@ parameters: A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values. required: true - role: variables + param_group: Execution schema: type: object - name: wrangles @@ -43,7 +43,7 @@ parameters: The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -55,7 +55,7 @@ parameters: set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables. required: false - role: option + param_group: Options runtime_default: loop schema: type: string diff --git a/registry/wrangles/_root/python.md b/registry/wrangles/_root/python.md index 8e4ce10..d6fc9a1 100644 --- a/registry/wrangles/_root/python.md +++ b/registry/wrangles/_root/python.md @@ -39,7 +39,7 @@ parameters: Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_). required: true - role: option + param_group: Options schema: type: string - name: output @@ -47,7 +47,7 @@ parameters: Name or list of output column(s). To output multiple columns, return a list of the corresponding length. required: true - role: column-output + param_group: I/O schema: type: - string @@ -58,7 +58,7 @@ parameters: conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: exception will be raised as normal. If multiple output columns are specified, this must match the length. required: false - role: option + param_group: Errors schema: type: - string diff --git a/registry/wrangles/_root/recipe.md b/registry/wrangles/_root/recipe.md index 296f36f..8e794a1 100644 --- a/registry/wrangles/_root/recipe.md +++ b/registry/wrangles/_root/recipe.md @@ -30,7 +30,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -41,7 +41,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -51,7 +51,7 @@ parameters: - name: name description: File name of the recipe. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: - name: variables description: A dictionary of variables to pass to the recipe. required: false - role: variables + param_group: Execution runtime_default: null schema: type: @@ -80,4 +80,4 @@ sources: Run another recipe as a wrangle against the current dataframe. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/registry/wrangles/_root/reindex.md b/registry/wrangles/_root/reindex.md index 4191c70..0a0fff9 100644 --- a/registry/wrangles/_root/reindex.md +++ b/registry/wrangles/_root/reindex.md @@ -31,7 +31,7 @@ parameters: - name: labels description: New labels / index to conform the axis specified by ‘axis’ to. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -40,7 +40,7 @@ parameters: - name: index description: New labels for the index. Preferably an Index object to avoid duplicating data. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -49,7 +49,7 @@ parameters: - name: columns description: New labels for the columns. Preferably an Index object to avoid duplicating data. required: false - role: option + param_group: I/O runtime_default: null schema: type: @@ -58,7 +58,7 @@ parameters: - name: axis description: Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1). required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/remove-words.md b/registry/wrangles/_root/remove-words.md index d84baf3..d93dc0c 100644 --- a/registry/wrangles/_root/remove-words.md +++ b/registry/wrangles/_root/remove-words.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of column to remove words from. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: to_remove description: Column or list of columns with a list of words to be removed. required: true - role: option + param_group: I/O schema: type: string - name: output description: Name of the output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -56,14 +56,14 @@ parameters: - name: tokenize_to_remove description: Tokenize all to_remove inputs. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore input and to_remove case. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/registry/wrangles/_root/rename.md b/registry/wrangles/_root/rename.md index 882075d..3235df0 100644 --- a/registry/wrangles/_root/rename.md +++ b/registry/wrangles/_root/rename.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -55,7 +55,7 @@ parameters: must also include the column named 'columns'. This can only be used instead of the standard rename. required: false - role: nested-wrangles + param_group: Execution runtime_default: null schema: type: diff --git a/registry/wrangles/_root/replace.md b/registry/wrangles/_root/replace.md index 452a7fa..53a8f31 100644 --- a/registry/wrangles/_root/replace.md +++ b/registry/wrangles/_root/replace.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,19 +42,19 @@ parameters: - name: find description: Pattern to find using regex. required: true - role: option + param_group: Options schema: type: string - name: replace description: Value to replace the pattern found. required: true - role: option + param_group: Options schema: type: string - name: output description: Name or list of output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/_root/round.md b/registry/wrangles/_root/round.md index 9f29e10..a2aca9d 100644 --- a/registry/wrangles/_root/round.md +++ b/registry/wrangles/_root/round.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: decimals description: Number of decimal places to round column. required: false - role: option + param_group: Formatting runtime_default: 0 schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/_root/similarity.md b/registry/wrangles/_root/similarity.md index 03f5f5e..7502f6f 100644 --- a/registry/wrangles/_root/similarity.md +++ b/registry/wrangles/_root/similarity.md @@ -31,7 +31,7 @@ parameters: - name: input description: Two columns of vectors to compare the similarity of. required: true - role: column-selector + param_group: I/O schema: type: array minItems: 2 @@ -39,7 +39,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: method @@ -47,7 +47,7 @@ parameters: The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons. required: false - role: option + param_group: Options runtime_default: cosine schema: type: string diff --git a/registry/wrangles/_root/sort.md b/registry/wrangles/_root/sort.md index 211ec33..7f53055 100644 --- a/registry/wrangles/_root/sort.md +++ b/registry/wrangles/_root/sort.md @@ -31,14 +31,14 @@ parameters: - name: ignore_index description: Ignore Index value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: by description: Name or list of the column(s) to sort by. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by. required: false - role: option + param_group: Options schema: type: - boolean diff --git a/registry/wrangles/_root/sql.md b/registry/wrangles/_root/sql.md index 4c3178d..c75242f 100644 --- a/registry/wrangles/_root/sql.md +++ b/registry/wrangles/_root/sql.md @@ -33,7 +33,7 @@ parameters: - name: command description: SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect. required: true - role: option + param_group: Options schema: type: string - name: params @@ -41,7 +41,7 @@ parameters: Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name). required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -51,14 +51,14 @@ parameters: - name: preserve_index description: Preserve Index value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: preserve_data_types description: Preserve Data Types value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/registry/wrangles/_root/standardize.md b/registry/wrangles/_root/standardize.md index e414d26..a6b4555 100644 --- a/registry/wrangles/_root/standardize.md +++ b/registry/wrangles/_root/standardize.md @@ -6,12 +6,13 @@ wrangle_name: standardize namespace: null title: Standardize description: >- - Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and - Subscription. + Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY + or bespoke model. wrangle_key: standardize aliases: [] slug: standardize -status: active +status: deprecated +replaced_by: standardize.custom visibility: public tags: - format @@ -33,7 +34,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +43,7 @@ parameters: - name: model_id description: The ID of the wrangle to use (do not include 'find' and 'replace'). required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +51,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +61,7 @@ parameters: - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/_root/translate.md b/registry/wrangles/_root/translate.md index ddba2cc..2339235 100644 --- a/registry/wrangles/_root/translate.md +++ b/registry/wrangles/_root/translate.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column to translate. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: target_language description: Code of the language to translate to. required: true - role: option + param_group: Options schema: type: string enum: @@ -83,7 +83,7 @@ parameters: - name: source_language description: Code of the language to translate from. If omitted, automatically detects the input language. required: false - role: option + param_group: Options runtime_default: AUTO schema: type: string @@ -116,7 +116,7 @@ parameters: - name: case description: Allow changing the case of the input prior to translation. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/_root/transpose.md b/registry/wrangles/_root/transpose.md index b9fa957..f116141 100644 --- a/registry/wrangles/_root/transpose.md +++ b/registry/wrangles/_root/transpose.md @@ -33,7 +33,7 @@ parameters: Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header. required: false - role: column-selector + param_group: I/O runtime_default: 0 schema: type: diff --git a/registry/wrangles/_root/try.md b/registry/wrangles/_root/try.md index 34daeb8..ce46640 100644 --- a/registry/wrangles/_root/try.md +++ b/registry/wrangles/_root/try.md @@ -31,7 +31,7 @@ parameters: - name: wrangles description: List of wrangles to apply. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -40,7 +40,7 @@ parameters: - name: retries description: Number of times to retry the wrangles if an error occurs. Default 0. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer @@ -51,7 +51,7 @@ parameters: dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue. required: false - role: option + param_group: Errors schema: type: object minItems: 1 diff --git a/registry/wrangles/compare/lists.md b/registry/wrangles/compare/lists.md index 8e7f258..78ec5c4 100644 --- a/registry/wrangles/compare/lists.md +++ b/registry/wrangles/compare/lists.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns containing lists to compare. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: method description: Type of comparison to perform. required: false - role: option + param_group: Options runtime_default: intersection schema: type: string @@ -54,14 +54,14 @@ parameters: - name: remove_duplicates description: Remove duplicates from the result. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore case when comparing string items. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/compare/text.md b/registry/wrangles/compare/text.md index c726f75..1dc9781 100644 --- a/registry/wrangles/compare/text.md +++ b/registry/wrangles/compare/text.md @@ -33,7 +33,7 @@ parameters: - name: input description: The columns to compare. First column is the base column. required: true - role: column-selector + param_group: I/O schema: type: array - name: output @@ -42,7 +42,7 @@ parameters: ratio_column] when method is overlap and include_ratio is true; otherwise a single column name. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: method description: The type of comparison to perform (difference, intersection, overlap, similarity). required: false - role: option + param_group: Options runtime_default: difference schema: type: string @@ -62,35 +62,35 @@ parameters: - name: char description: Character to split strings on for difference and intersection. Defaults to a space. required: false - role: option + param_group: Options runtime_default: ' ' schema: type: string - name: non_match_char description: Character to use for non-matching characters when using overlap. required: false - role: option + param_group: Formatting runtime_default: '*' schema: type: string - name: include_ratio description: Include the ratio of matching characters when using overlap. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: decimal_places description: Number of decimal places to round the overlap ratio to. required: false - role: option + param_group: Formatting runtime_default: 3 schema: type: integer - name: exact_match description: Value to use for exact matches when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -99,7 +99,7 @@ parameters: - name: empty_a description: Value to use when input A is empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -108,7 +108,7 @@ parameters: - name: empty_b description: Value to use when input B is empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -117,7 +117,7 @@ parameters: - name: all_empty description: Value to use when both inputs are empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -126,14 +126,14 @@ parameters: - name: case_sensitive description: Whether the comparison is case sensitive. Defaults to true. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: metric description: Metric value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: token_sort schema: type: string diff --git a/registry/wrangles/compute/case-when.md b/registry/wrangles/compute/case-when.md index c93bf26..d0be1cf 100644 --- a/registry/wrangles/compute/case-when.md +++ b/registry/wrangles/compute/case-when.md @@ -31,13 +31,13 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: cases description: List of conditions and corresponding values. required: true - role: option + param_group: Options schema: type: array minItems: 1 @@ -58,7 +58,7 @@ parameters: - name: default description: Value to assign if no conditions are met. Default None. required: false - role: fallback-value + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/compute/score-search-results.md b/registry/wrangles/compute/score-search-results.md index 68dbc5d..e730a00 100644 --- a/registry/wrangles/compute/score-search-results.md +++ b/registry/wrangles/compute/score-search-results.md @@ -35,7 +35,7 @@ parameters: List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)]. required: true - role: column-selector + param_group: I/O schema: type: array - name: output @@ -43,7 +43,7 @@ parameters: Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. required: true - role: column-output + param_group: I/O schema: type: - string @@ -51,98 +51,98 @@ parameters: - name: must_match_part_code description: If true, filters out results that don't satisfy the allowed match types. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_mpn_exact description: Treat exact MPN matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_mpn_partial description: Treat partial MPN matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_other_exact description: Treat exact other part code matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_other_partial description: Treat partial other part code matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: blacklist_keywords description: Comma-separated list or array of keywords to filter out URLs containing them. required: false - role: option + param_group: Options runtime_default: '' schema: type: string - name: mpn_exact_score description: Mpn Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 8 schema: type: number - name: mpn_partial_base description: Mpn Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 4 schema: type: number - name: part_code_exact_score description: Part Code Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 6 schema: type: number - name: part_code_partial_base description: Part Code Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 2 schema: type: number - name: supplier_exact_score description: Supplier Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 3 schema: type: number - name: supplier_partial_base description: Supplier Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 1 schema: type: number - name: context_match_base description: Context Match Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 2 schema: type: number - name: fuzzy_match_threshold description: Fuzzy Match Threshold value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 0.8 schema: type: number diff --git a/registry/wrangles/convert/case.md b/registry/wrangles/convert/case.md index 2709ff6..986490c 100644 --- a/registry/wrangles/convert/case.md +++ b/registry/wrangles/convert/case.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: case description: Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts. required: false - role: option + param_group: Options runtime_default: lower schema: type: string diff --git a/registry/wrangles/convert/data-type.md b/registry/wrangles/convert/data-type.md index c74839c..51543fe 100644 --- a/registry/wrangles/convert/data-type.md +++ b/registry/wrangles/convert/data-type.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: data_type description: Data type to produce. required: false - role: option + param_group: Options runtime_default: str schema: type: string @@ -57,7 +57,7 @@ parameters: - name: default description: Value returned when conversion fails. If omitted, the original value is retained. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: [string, number, array, object, boolean, "null"] diff --git a/registry/wrangles/convert/fraction-to-decimal.md b/registry/wrangles/convert/fraction-to-decimal.md index 69ef4a5..74c50db 100644 --- a/registry/wrangles/convert/fraction-to-decimal.md +++ b/registry/wrangles/convert/fraction-to-decimal.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: decimals description: Number of decimals to round fraction. required: false - role: option + param_group: Formatting runtime_default: 4 schema: type: integer - name: output description: Name of the output colum. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/convert/from-json.md b/registry/wrangles/convert/from-json.md index bed4ad6..3e55d3b 100644 --- a/registry/wrangles/convert/from-json.md +++ b/registry/wrangles/convert/from-json.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of columns containing valid JSON text. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: default description: Value returned for empty or invalid JSON. A list may supply one fallback per input column. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: [string, number, array, object, boolean, "null"] diff --git a/registry/wrangles/convert/from-yaml.md b/registry/wrangles/convert/from-yaml.md index 8e56142..a8370a7 100644 --- a/registry/wrangles/convert/from-yaml.md +++ b/registry/wrangles/convert/from-yaml.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: may also be a list - either a single value to apply to all columns, or one value per input column. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: diff --git a/registry/wrangles/convert/to-json.md b/registry/wrangles/convert/to-json.md index 856dc5c..c9d4e5e 100644 --- a/registry/wrangles/convert/to-json.md +++ b/registry/wrangles/convert/to-json.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,7 +50,7 @@ parameters: - name: ensure_ascii description: If true, non-ASCII characters will be escaped. Default is false. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean @@ -62,7 +62,7 @@ parameters: integer indent indents that many spaces per level. If indent is a string (such as '\t'), that string is used to indent each level. required: false - role: option + param_group: Formatting schema: type: - string @@ -72,7 +72,7 @@ parameters: If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key. required: false - role: option + param_group: Formatting schema: type: boolean examples: [] diff --git a/registry/wrangles/convert/to-yaml.md b/registry/wrangles/convert/to-yaml.md index 003a7e9..d837118 100644 --- a/registry/wrangles/convert/to-yaml.md +++ b/registry/wrangles/convert/to-yaml.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,21 +50,21 @@ parameters: - name: sort_keys description: 'If sort_keys is true (default: False), then the output of dictionaries will be sorted by key.' required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: allow_unicode description: Allow Unicode value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: indent description: Specify the number of spaces for indentation to specify nested elements. required: false - role: option + param_group: Formatting schema: type: integer examples: [] diff --git a/registry/wrangles/create/bins.md b/registry/wrangles/create/bins.md index d4fce38..cd1bd69 100644 --- a/registry/wrangles/create/bins.md +++ b/registry/wrangles/create/bins.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of new column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: bins description: Defines the number of equal-width bins in the range. required: true - role: option + param_group: Options schema: type: - integer @@ -56,7 +56,7 @@ parameters: - name: labels description: Labels for the returned bins. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/create/column.md b/registry/wrangles/create/column.md index 396dc5c..bb0677b 100644 --- a/registry/wrangles/create/column.md +++ b/registry/wrangles/create/column.md @@ -31,7 +31,7 @@ parameters: - name: output description: 'Name or list of names of new columns or column_name: value pairs.' required: true - role: column-output + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: (Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -57,7 +57,7 @@ parameters: column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value. required: false - role: option + param_group: Options runtime_default: existing schema: type: string @@ -72,7 +72,7 @@ parameters: empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null. required: false - role: option + param_group: Options runtime_default: existing schema: type: string diff --git a/registry/wrangles/create/embeddings.md b/registry/wrangles/create/embeddings.md index da187e7..719f6c9 100644 --- a/registry/wrangles/create/embeddings.md +++ b/registry/wrangles/create/embeddings.md @@ -31,19 +31,19 @@ parameters: - name: input description: The column of text to create the embeddings for. required: true - role: column-selector + param_group: I/O schema: type: string - name: api_key description: The API key. required: true - role: credential + param_group: Details schema: type: string - name: output description: The output column the embeddings will be saved as. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: batch_size description: The number of rows to submit per individual request. required: false - role: option + param_group: Execution runtime_default: 100 schema: type: integer @@ -61,14 +61,14 @@ parameters: The number of requests to submit in parallel. Each request contains the number of rows set as batch_size. required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer - name: output_type description: Output the embeddings as a numpy array or a python list Default - python list. required: false - role: column-output + param_group: Formatting runtime_default: python list schema: type: string @@ -78,7 +78,7 @@ parameters: - name: model description: The specific model to use to generate the embeddings. required: false - role: option + param_group: Details runtime_default: text-embedding-3-small schema: type: string @@ -87,7 +87,7 @@ parameters: The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer @@ -97,7 +97,7 @@ parameters: provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format. required: false - role: option + param_group: Details runtime_default: https://api.openai.com/v1/embeddings schema: type: string @@ -106,7 +106,7 @@ parameters: The precision of the embeddings. Default is float32. This should be used with output_type numpy array. required: false - role: option + param_group: Formatting runtime_default: float32 schema: type: string @@ -121,7 +121,7 @@ parameters: only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy). required: false - role: option + param_group: Details runtime_default: null schema: type: @@ -135,7 +135,7 @@ parameters: The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/registry/wrangles/create/guid.md b/registry/wrangles/create/guid.md index 59d51d7..960ab60 100644 --- a/registry/wrangles/create/guid.md +++ b/registry/wrangles/create/guid.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/registry/wrangles/create/hash.md b/registry/wrangles/create/hash.md index 8dd0da6..f575a66 100644 --- a/registry/wrangles/create/hash.md +++ b/registry/wrangles/create/hash.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of new column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: method description: 'The method to use to hash the input (Default: md5).' required: false - role: option + param_group: Options runtime_default: md5 schema: type: string diff --git a/registry/wrangles/create/index.md b/registry/wrangles/create/index.md index 53f05c1..8fef5b5 100644 --- a/registry/wrangles/create/index.md +++ b/registry/wrangles/create/index.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -39,21 +39,21 @@ parameters: - name: start description: (Optional; default 1) Starting number for the index. required: false - role: option + param_group: Options runtime_default: 1 schema: type: integer - name: step description: (Optional; default 1) Step between successive rows. required: false - role: option + param_group: Options runtime_default: 1 schema: type: integer - name: by description: Optional. Cluster the created indexes by one or more columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/create/jinja.md b/registry/wrangles/create/jinja.md index 219cac8..8a3b507 100644 --- a/registry/wrangles/create/jinja.md +++ b/registry/wrangles/create/jinja.md @@ -34,7 +34,7 @@ parameters: input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_). required: true - role: option + param_group: Options schema: type: object additionalProperties: false @@ -48,7 +48,7 @@ parameters: - name: output description: Name of the column to be output to. required: true - role: column-output + param_group: I/O schema: type: array - name: input @@ -56,7 +56,7 @@ parameters: Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/create/uuid.md b/registry/wrangles/create/uuid.md index 0e64405..56bc6fe 100644 --- a/registry/wrangles/create/uuid.md +++ b/registry/wrangles/create/uuid.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/registry/wrangles/extract/address.md b/registry/wrangles/extract/address.md index 89b1da8..e8e3c38 100644 --- a/registry/wrangles/extract/address.md +++ b/registry/wrangles/extract/address.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: dataType description: Specific part of the address to extract. required: true - role: option + param_group: Options schema: type: string enum: @@ -59,7 +59,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/ai.md b/registry/wrangles/extract/ai.md index 41ef1a3..e116587 100644 --- a/registry/wrangles/extract/ai.md +++ b/registry/wrangles/extract/ai.md @@ -33,7 +33,7 @@ parameters: - name: api_key description: OpenAI API key used for this wrangle, normally supplied through a recipe variable. required: true - role: credential + param_group: Details schema: type: string - name: input @@ -41,7 +41,7 @@ parameters: Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -57,7 +57,7 @@ parameters: for one prompted value, or an array of field names/definitions. Each field may use the schema options below. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -139,7 +139,7 @@ parameters: is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields. required: false - role: model-reference + param_group: Details runtime_default: null schema: type: @@ -153,7 +153,7 @@ parameters: Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -187,7 +187,7 @@ parameters: How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -200,7 +200,7 @@ parameters: - name: char description: Separator used only when output_format is concatenate. Defaults to comma-space. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string @@ -211,7 +211,7 @@ parameters: order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean @@ -221,7 +221,7 @@ parameters: priorities, normalization requirements, or other behavior that applies to the complete extraction. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -235,13 +235,13 @@ parameters: OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model. required: false - role: option + param_group: Details schema: type: string - name: threads description: Maximum number of row-level requests sent in parallel. The configured default is 32. required: false - role: option + param_group: Execution schema: type: integer minimum: 1 @@ -250,7 +250,7 @@ parameters: Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner. required: false - role: option + param_group: Execution schema: type: number exclusiveMinimum: 0 @@ -259,7 +259,7 @@ parameters: Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline. required: false - role: option + param_group: Errors schema: type: integer minimum: 0 @@ -269,13 +269,13 @@ parameters: protocol only when protocol is omitted; new recipes should use the configured Responses endpoint. required: false - role: option + param_group: Details schema: type: string - name: provider description: AI service provider. Currently only OpenAI is supported. required: false - role: option + param_group: Details schema: type: string enum: @@ -285,7 +285,7 @@ parameters: OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions. required: false - role: option + param_group: Details schema: type: string enum: @@ -296,14 +296,14 @@ parameters: Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15. required: false - role: option + param_group: Execution schema: type: number exclusiveMinimum: 0 - name: store description: Whether OpenAI may store Responses API results. Defaults to false. required: false - role: option + param_group: Details schema: type: boolean - name: cache @@ -311,7 +311,7 @@ parameters: Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required. required: false - role: option + param_group: Details schema: type: boolean - name: cache_ttl @@ -319,7 +319,7 @@ parameters: Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together. required: false - role: option + param_group: Details schema: type: number exclusiveMinimum: 0 @@ -329,7 +329,7 @@ parameters: dictionary keys automatically switch to non-strict provider mode and are still validated locally. required: false - role: option + param_group: Details schema: type: boolean - name: reasoning @@ -337,7 +337,7 @@ parameters: Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies. required: false - role: option + param_group: Details schema: type: object properties: @@ -355,7 +355,7 @@ parameters: Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models. required: false - role: option + param_group: Details schema: type: string enum: diff --git a/registry/wrangles/extract/attributes.md b/registry/wrangles/extract/attributes.md index 3bb2db0..be13e4a 100644 --- a/registry/wrangles/extract/attributes.md +++ b/registry/wrangles/extract/attributes.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: responseContent description: Span - returns the text found. object - returns an object with the value and unit. required: false - role: option + param_group: Options runtime_default: span schema: type: string @@ -60,7 +60,7 @@ parameters: - name: attribute_type description: Request only a specific type of attribute. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -95,7 +95,7 @@ parameters: - name: desired_unit description: Convert the extracted unit to the desired unit. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -106,7 +106,7 @@ parameters: When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid. required: false - role: option + param_group: Options runtime_default: mid schema: type: string @@ -117,14 +117,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -138,7 +138,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/brackets.md b/registry/wrangles/extract/brackets.md index 03b9f40..a3e8b6c 100644 --- a/registry/wrangles/extract/brackets.md +++ b/registry/wrangles/extract/brackets.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: (Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets. required: false - role: option + param_group: Options runtime_default: all schema: type: @@ -59,14 +59,14 @@ parameters: - name: include_brackets description: (Optional) Include the brackets in the output. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -79,7 +79,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/codes.md b/registry/wrangles/extract/codes.md index 7a8fb3b..e293cd9 100644 --- a/registry/wrangles/extract/codes.md +++ b/registry/wrangles/extract/codes.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,14 +48,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -68,14 +68,14 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string - name: min_length description: Minimum length of allowed results. required: false - role: option + param_group: Options schema: type: - integer @@ -83,7 +83,7 @@ parameters: - name: max_length description: Maximum length of allowed results. required: false - role: option + param_group: Options schema: type: - integer @@ -94,7 +94,7 @@ parameters: balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict. required: false - role: option + param_group: Options schema: type: string enum: @@ -104,7 +104,7 @@ parameters: - name: sort_order description: Default is input order. Also allows longest or shortest. required: false - role: option + param_group: Formatting schema: type: string enum: @@ -114,13 +114,13 @@ parameters: - name: disallowed_patterns description: A pattern or JSON array of regex patterns to not include in the found codes. required: false - role: option + param_group: Options schema: type: string - name: include_multi_part_tokens description: Whether to include multi-part tokens that have a space. Default True. required: false - role: option + param_group: Options schema: type: boolean - name: extract_raw @@ -128,7 +128,7 @@ parameters: Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False. required: false - role: option + param_group: Options schema: type: boolean examples: [] diff --git a/registry/wrangles/extract/custom.md b/registry/wrangles/extract/custom.md index 05dc9f9..a9d612b 100644 --- a/registry/wrangles/extract/custom.md +++ b/registry/wrangles/extract/custom.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: model_id description: The ID of the wrangle to use. required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,49 +60,49 @@ parameters: - name: use_labels description: 'Use Labels in the extract output {label: value}.' required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: extract_raw description: Extract the raw data from the wrangle. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: use_spellcheck description: Use spellcheck to also find minor mispellings compared to the reference data. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: include_empty_labels description: Include labels with no found values in the output when using use_labels=True. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: sort description: Sort the results. required: false - role: option + param_group: Options runtime_default: training_order schema: type: string @@ -118,7 +118,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -132,7 +132,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/date-properties.md b/registry/wrangles/extract/date-properties.md index 22d1a39..e094780 100644 --- a/registry/wrangles/extract/date-properties.md +++ b/registry/wrangles/extract/date-properties.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: property description: Property to extract from date. required: true - role: option + param_group: Options schema: type: string enum: @@ -55,7 +55,7 @@ parameters: - name: output description: Name of the output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/extract/date-range.md b/registry/wrangles/extract/date-range.md index 89811d8..e131fae 100644 --- a/registry/wrangles/extract/date-range.md +++ b/registry/wrangles/extract/date-range.md @@ -31,25 +31,25 @@ parameters: - name: start_time description: Name of the start date column. required: true - role: option + param_group: I/O schema: type: string - name: end_time description: Name of the end date column. required: true - role: option + param_group: I/O schema: type: string - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: range description: Type of frequency to count. required: false - role: option + param_group: Options runtime_default: day schema: type: string diff --git a/registry/wrangles/extract/html.md b/registry/wrangles/extract/html.md index 938610c..4f07693 100644 --- a/registry/wrangles/extract/html.md +++ b/registry/wrangles/extract/html.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: data_type description: The type of data to extract. required: true - role: option + param_group: Options schema: type: string enum: @@ -49,7 +49,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/properties.md b/registry/wrangles/extract/properties.md index 4c4198c..22ad0c9 100644 --- a/registry/wrangles/extract/properties.md +++ b/registry/wrangles/extract/properties.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: property_type description: The specific type of properties to extract. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -62,7 +62,7 @@ parameters: - name: return_data_type description: Legacy format option. Prefer output_format. required: false - role: option + param_group: Formatting runtime_default: list schema: type: string @@ -72,14 +72,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -93,7 +93,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/extract/regex.md b/registry/wrangles/extract/regex.md index 8529f1a..d546de0 100644 --- a/registry/wrangles/extract/regex.md +++ b/registry/wrangles/extract/regex.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: find description: Pattern to find using regex. required: true - role: option + param_group: Options schema: type: string - name: output description: Name of the output column(s). required: true - role: column-output + param_group: I/O schema: type: - string @@ -58,7 +58,7 @@ parameters: `r'(\d+)\s(\w+)'` and `output_pattern = '\2 \1'`, with input `'120 volt'`, the output would be `'volt 120'`. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -67,14 +67,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -87,7 +87,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/registry/wrangles/format/dates.md b/registry/wrangles/format/dates.md index 519bb66..a5ed72d 100644 --- a/registry/wrangles/format/dates.md +++ b/registry/wrangles/format/dates.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: format description: String pattern to format date. required: true - role: option + param_group: Formatting schema: type: string - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/format/pad.md b/registry/wrangles/format/pad.md index 84de5da..337795e 100644 --- a/registry/wrangles/format/pad.md +++ b/registry/wrangles/format/pad.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,25 +40,25 @@ parameters: - name: pad_length description: Length for the output. required: true - role: option + param_group: Formatting schema: type: integer - name: side description: Side from which to fill resulting string. required: true - role: option + param_group: Options schema: type: string - name: char description: The character to pad the input with. required: true - role: option + param_group: Formatting schema: type: string - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -68,7 +68,7 @@ parameters: - name: skip_empty description: If true, skip padding for empty or whitespace-only values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/format/prefix.md b/registry/wrangles/format/prefix.md index 6035fd6..3e6e96f 100644 --- a/registry/wrangles/format/prefix.md +++ b/registry/wrangles/format/prefix.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: value description: Prefix value to add. required: true - role: option + param_group: Options schema: type: - string @@ -49,7 +49,7 @@ parameters: - name: output description: (Optional) Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/format/price-breaks.md b/registry/wrangles/format/price-breaks.md index 3ca2e39..3908183 100644 --- a/registry/wrangles/format/price-breaks.md +++ b/registry/wrangles/format/price-breaks.md @@ -31,19 +31,19 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: categoryLabel description: Prefix for output columns that identify the source price-break category. required: true - role: option + param_group: Formatting schema: type: string - name: valueLabel description: Prefix for output columns that contain the corresponding price-break value. required: true - role: option + param_group: Formatting schema: type: string examples: [] @@ -60,4 +60,4 @@ sources: Expand non-empty price-break cells into paired category and value columns. -This first-pass guidance is derived from the callable signature. +This guidance was derived from the callable signature. diff --git a/registry/wrangles/format/remove-duplicates.md b/registry/wrangles/format/remove-duplicates.md index 2729093..897c6cf 100644 --- a/registry/wrangles/format/remove-duplicates.md +++ b/registry/wrangles/format/remove-duplicates.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,7 +50,7 @@ parameters: - name: ignore_case description: Ignore case when removing duplicates. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/format/significant-figures.md b/registry/wrangles/format/significant-figures.md index aead2ba..68c6587 100644 --- a/registry/wrangles/format/significant-figures.md +++ b/registry/wrangles/format/significant-figures.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: significant_figures description: Number of significant figures to format to. Default is 3. required: false - role: option + param_group: Formatting runtime_default: 3 schema: type: integer - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/format/suffix.md b/registry/wrangles/format/suffix.md index 2134406..e53346a 100644 --- a/registry/wrangles/format/suffix.md +++ b/registry/wrangles/format/suffix.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: value description: Suffix value to add. required: true - role: option + param_group: Options schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: (Optional) Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/format/trim.md b/registry/wrangles/format/trim.md index fcc36bc..a05c2d0 100644 --- a/registry/wrangles/format/trim.md +++ b/registry/wrangles/format/trim.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/generate/ai.md b/registry/wrangles/generate/ai.md index 9690616..50fdeef 100644 --- a/registry/wrangles/generate/ai.md +++ b/registry/wrangles/generate/ai.md @@ -31,13 +31,13 @@ parameters: - name: api_key description: OpenAI-compatible API key. required: true - role: credential + param_group: Details schema: type: string - name: output description: Target schema; string/array shorthands are expanded automatically. required: true - role: column-output + param_group: I/O schema: type: - string @@ -46,7 +46,7 @@ parameters: - name: input description: Column(s) to concatenate into the prompt (defaults to all columns). required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -56,35 +56,35 @@ parameters: - name: model description: Responses model name (e.g. gpt-5-mini). required: false - role: option + param_group: Details runtime_default: gpt-5 schema: type: string - name: threads description: Maximum concurrent requests (default 20). required: false - role: option + param_group: Execution runtime_default: 20 schema: type: integer - name: timeout description: Per-request timeout in seconds. required: false - role: option + param_group: Execution runtime_default: 90 schema: type: integer - name: retries description: Number of retry attempts on failure. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer - name: messages description: Optional extra messages forwarded to the inner generate helper. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -94,28 +94,28 @@ parameters: - name: url description: Override for the OpenAI-compatible endpoint. required: false - role: option + param_group: Details runtime_default: https://api.openai.com/v1/responses schema: type: string - name: strict description: Enforce JSON-schema validation on the response. required: false - role: option + param_group: Details runtime_default: false schema: type: boolean - name: web_search description: Enable DuckDuckGo context lookup per row. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: reasoning description: Responses API reasoning options (forwarded verbatim). required: false - role: option + param_group: Details runtime_default: effort: low schema: @@ -125,14 +125,14 @@ parameters: - name: previous_response description: Chain responses by reusing previous_response_id for field-by-field calls. required: false - role: option + param_group: Details runtime_default: false schema: type: boolean - name: summary description: Request summary text to be merged into the output. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/merge/coalesce.md b/registry/wrangles/merge/coalesce.md index 5178b47..276ee34 100644 --- a/registry/wrangles/merge/coalesce.md +++ b/registry/wrangles/merge/coalesce.md @@ -31,13 +31,13 @@ parameters: - name: input description: List of input columns or a single column containing lists. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output columns. This is required if multiple input columns are provided. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/merge/concatenate.md b/registry/wrangles/merge/concatenate.md index fa69003..cd5efec 100644 --- a/registry/wrangles/merge/concatenate.md +++ b/registry/wrangles/merge/concatenate.md @@ -31,7 +31,7 @@ parameters: - name: input description: Either a single column name or list of columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,20 +40,20 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: char description: (Optional) Character to add between successive values. required: false - role: option + param_group: Formatting runtime_default: ',' schema: type: string - name: skip_empty description: Whether to skip empty values, defaults to false. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/merge/dictionaries.md b/registry/wrangles/merge/dictionaries.md index 97e424f..e68412a 100644 --- a/registry/wrangles/merge/dictionaries.md +++ b/registry/wrangles/merge/dictionaries.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: skip_empty description: Whether to skip empty dictionaries when merging. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/merge/key-value-pairs.md b/registry/wrangles/merge/key-value-pairs.md index 167aa94..896566f 100644 --- a/registry/wrangles/merge/key-value-pairs.md +++ b/registry/wrangles/merge/key-value-pairs.md @@ -33,19 +33,19 @@ parameters: - name: input description: Matched pairs of key and value columns. required: true - role: column-selector + param_group: I/O schema: type: object - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: skip_empty description: Whether to skip empty keys or values when creating the dictionary. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/merge/lists.md b/registry/wrangles/merge/lists.md index b20fbaf..04a8c50 100644 --- a/registry/wrangles/merge/lists.md +++ b/registry/wrangles/merge/lists.md @@ -31,33 +31,33 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: remove_duplicates description: Whether to remove duplicates from the created list. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore case when removing duplicates. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: include_empty description: Whether to include empty values in the created list. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/registry/wrangles/merge/to-dict.md b/registry/wrangles/merge/to-dict.md index 509779b..137a8bb 100644 --- a/registry/wrangles/merge/to-dict.md +++ b/registry/wrangles/merge/to-dict.md @@ -33,19 +33,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: include_empty description: Whether to include empty columns in the created dictionary. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/merge/to-list.md b/registry/wrangles/merge/to-list.md index 62ebd22..3566655 100644 --- a/registry/wrangles/merge/to-list.md +++ b/registry/wrangles/merge/to-list.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: include_empty description: Whether to include empty columns in the created list. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/search/find-links.md b/registry/wrangles/search/find-links.md index 2cde605..11561e7 100644 --- a/registry/wrangles/search/find-links.md +++ b/registry/wrangles/search/find-links.md @@ -33,7 +33,7 @@ parameters: - name: queries description: Name or list of input columns containing search queries. required: true - role: option + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: - name: id description: Name of the column containing the row ID to append to each search result. required: true - role: option + param_group: I/O schema: type: string - name: output @@ -49,7 +49,7 @@ parameters: Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: client description: The search provider to use. required: false - role: option + param_group: Details runtime_default: serpapi schema: type: string @@ -70,7 +70,7 @@ parameters: API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY). required: false - role: credential + param_group: Details runtime_default: null schema: type: @@ -79,39 +79,39 @@ parameters: - name: n_results description: Number of search results to return per query (default 10, max 100). required: false - role: option + param_group: Options runtime_default: 10 schema: type: integer - name: threads description: Number of concurrent threads for parallel processing (default 10). required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer - name: country description: 'Country code for search results (default ''us''). Alias: gl.' required: false - role: option + param_group: Options schema: type: string - name: language description: 'Language code for search results (default ''en''). Alias: hl.' required: false - role: option + param_group: Options schema: type: string - name: location description: Location for search results (e.g., 'Austin, Texas'). required: false - role: option + param_group: Options schema: type: string - name: device description: Device type for search results. required: false - role: option + param_group: Options schema: type: string enum: diff --git a/registry/wrangles/search/retrieve-link-content.md b/registry/wrangles/search/retrieve-link-content.md index c83f25c..c8a87ee 100644 --- a/registry/wrangles/search/retrieve-link-content.md +++ b/registry/wrangles/search/retrieve-link-content.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns containing URLs or Scored Search Result dictionaries. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -43,7 +43,7 @@ parameters: Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: client description: The retrieval provider to use. required: false - role: option + param_group: Details runtime_default: google_url_context schema: type: string @@ -62,7 +62,7 @@ parameters: - name: api_key description: API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY). required: false - role: credential + param_group: Details runtime_default: null schema: type: @@ -71,7 +71,7 @@ parameters: - name: prompt description: Optional custom system prompt to guide the extraction behavior and output format. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -80,14 +80,14 @@ parameters: - name: model_id description: The specific model ID to use (default models/gemini-3-flash-preview). required: false - role: model-reference + param_group: Details runtime_default: models/gemini-3-flash-preview schema: type: string - name: output_format description: The desired format for the extracted content. required: false - role: column-output + param_group: Formatting runtime_default: json schema: type: string @@ -97,7 +97,7 @@ parameters: - name: threads description: Number of concurrent threads for parallel processing (default 10). required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer diff --git a/registry/wrangles/select/columns.md b/registry/wrangles/select/columns.md index eaaa709..5e43303 100644 --- a/registry/wrangles/select/columns.md +++ b/registry/wrangles/select/columns.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column(s) to select. required: true - role: column-selector + param_group: I/O schema: type: - string diff --git a/registry/wrangles/select/dictionary-element.md b/registry/wrangles/select/dictionary-element.md index 577ffe5..de37e3f 100644 --- a/registry/wrangles/select/dictionary-element.md +++ b/registry/wrangles/select/dictionary-element.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,13 +42,13 @@ parameters: The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary. required: true - role: option + param_group: Options schema: type: string - name: output description: Name of the output column. If omitted, the input column will be replaced. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set. required: false - role: fallback-value + param_group: Errors runtime_default: '' schema: type: diff --git a/registry/wrangles/select/element.md b/registry/wrangles/select/element.md index 9cc5232..e8321b7 100644 --- a/registry/wrangles/select/element.md +++ b/registry/wrangles/select/element.md @@ -33,7 +33,7 @@ parameters: Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{"key":"val"}] -> "val". required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: default description: Set the default value to return if the specified element doesn't exist. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: diff --git a/registry/wrangles/select/group-by.md b/registry/wrangles/select/group-by.md index d1cbb50..af00a5b 100644 --- a/registry/wrangles/select/group-by.md +++ b/registry/wrangles/select/group-by.md @@ -31,7 +31,7 @@ parameters: - name: by description: List of the input columns to group on. required: false - role: column-selector + param_group: I/O runtime_default: [] schema: type: @@ -43,14 +43,14 @@ parameters: If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total). required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: list description: Group and return all values for these column(s) as a list. required: false - role: option + param_group: Options schema: type: - string @@ -58,7 +58,7 @@ parameters: - name: first description: The first value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -66,7 +66,7 @@ parameters: - name: last description: The last value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -74,7 +74,7 @@ parameters: - name: min description: The minimum value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -82,7 +82,7 @@ parameters: - name: max description: The maximum value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -90,7 +90,7 @@ parameters: - name: mean description: The mean (average) value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -98,7 +98,7 @@ parameters: - name: median description: The median value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -106,7 +106,7 @@ parameters: - name: nunique description: The count of unique values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -114,7 +114,7 @@ parameters: - name: count description: The count of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -125,7 +125,7 @@ parameters: converted to JSON-safe strings; missing values use the key "null" and booleans use lowercase "true"/"false". required: false - role: option + param_group: Options schema: type: - string @@ -133,7 +133,7 @@ parameters: - name: std description: The standard deviation of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -141,7 +141,7 @@ parameters: - name: sum description: The total of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -149,7 +149,7 @@ parameters: - name: any description: Return true if any of the values for these column(s) are true. required: false - role: option + param_group: Options schema: type: - string @@ -157,7 +157,7 @@ parameters: - name: all description: Return true if all of the values for these column(s) are true. required: false - role: option + param_group: Options schema: type: - string @@ -165,7 +165,7 @@ parameters: - name: p75 description: Get a percentile. Note, you can use any integer here for the corresponding percentile. required: false - role: option + param_group: Options schema: type: - string @@ -174,7 +174,7 @@ parameters: name_pattern: ^custom\.[A-Za-z_][A-Za-z0-9_]*$ description: Placeholder for custom functions. Replace 'placeholder' with the name of the function. required: false - role: option + param_group: Options schema: type: - string diff --git a/registry/wrangles/select/head.md b/registry/wrangles/select/head.md index 7a1c41b..e72e0ca 100644 --- a/registry/wrangles/select/head.md +++ b/registry/wrangles/select/head.md @@ -31,7 +31,7 @@ parameters: - name: n description: Number of rows to return. required: true - role: option + param_group: Options schema: type: integer examples: [] diff --git a/registry/wrangles/select/highest-confidence.md b/registry/wrangles/select/highest-confidence.md index 57a8b0b..20958b2 100644 --- a/registry/wrangles/select/highest-confidence.md +++ b/registry/wrangles/select/highest-confidence.md @@ -33,13 +33,13 @@ parameters: - name: input description: List of the input columns to select from. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: If two columns; the result and confidence. If one column; [result, confidence]. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/registry/wrangles/select/left.md b/registry/wrangles/select/left.md index 6b97981..4fb625f 100644 --- a/registry/wrangles/select/left.md +++ b/registry/wrangles/select/left.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -44,13 +44,13 @@ parameters: Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0. required: true - role: option + param_group: Options schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/select/length.md b/registry/wrangles/select/length.md index 3bbbe19..00ee362 100644 --- a/registry/wrangles/select/length.md +++ b/registry/wrangles/select/length.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/select/list-element.md b/registry/wrangles/select/list-element.md index be97b68..c83902b 100644 --- a/registry/wrangles/select/list-element.md +++ b/registry/wrangles/select/list-element.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,14 +52,14 @@ parameters: The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list. required: false - role: option + param_group: Options runtime_default: 0 schema: type: integer - name: default description: Set the default value to return if the specified element doesn't exist. required: false - role: fallback-value + param_group: Errors runtime_default: '' schema: type: diff --git a/registry/wrangles/select/right.md b/registry/wrangles/select/right.md index c080b2c..c46a334 100644 --- a/registry/wrangles/select/right.md +++ b/registry/wrangles/select/right.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -44,13 +44,13 @@ parameters: Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0. required: true - role: option + param_group: Options schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/select/sample.md b/registry/wrangles/select/sample.md index b200b89..d663762 100644 --- a/registry/wrangles/select/sample.md +++ b/registry/wrangles/select/sample.md @@ -33,7 +33,7 @@ parameters: If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned. required: true - role: option + param_group: Options schema: type: - integer diff --git a/registry/wrangles/select/substring.md b/registry/wrangles/select/substring.md index 7e10467..a2461c0 100644 --- a/registry/wrangles/select/substring.md +++ b/registry/wrangles/select/substring.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: The position of the first character to select. If ommited will start from the beginning and length must be provided. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -54,7 +54,7 @@ parameters: The length of the string to select. If ommited will select to the end of the string and start must be provided. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -64,7 +64,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/registry/wrangles/select/tail.md b/registry/wrangles/select/tail.md index 9464a8e..be85f43 100644 --- a/registry/wrangles/select/tail.md +++ b/registry/wrangles/select/tail.md @@ -31,7 +31,7 @@ parameters: - name: n description: Number of rows to return. required: true - role: option + param_group: Options schema: type: integer examples: [] diff --git a/registry/wrangles/select/threshold.md b/registry/wrangles/select/threshold.md index 342440e..58dcb94 100644 --- a/registry/wrangles/select/threshold.md +++ b/registry/wrangles/select/threshold.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of the input columns to select from. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: threshold description: Threshold above which to choose the first option, otherwise the second. required: true - role: option + param_group: Options schema: type: number minimum: 0 diff --git a/registry/wrangles/split/dictionary.md b/registry/wrangles/split/dictionary.md index 131ac37..163b991 100644 --- a/registry/wrangles/split/dictionary.md +++ b/registry/wrangles/split/dictionary.md @@ -35,7 +35,7 @@ parameters: Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -49,7 +49,7 @@ parameters: be two output columns for the keys and values lists. If not provided, Keys and Values will be used. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: default description: Provide a set of default headings and values if they are not found within the input. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: @@ -70,7 +70,7 @@ parameters: How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values. required: false - role: column-output + param_group: Formatting runtime_default: columns schema: type: string diff --git a/registry/wrangles/split/list.md b/registry/wrangles/split/list.md index c96388e..889d858 100644 --- a/registry/wrangles/split/list.md +++ b/registry/wrangles/split/list.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column to be split. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/registry/wrangles/split/text.md b/registry/wrangles/split/text.md index 1d69806..968c26d 100644 --- a/registry/wrangles/split/text.md +++ b/registry/wrangles/split/text.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column to be split. required: true - role: column-selector + param_group: I/O schema: type: string - name: output @@ -40,7 +40,7 @@ parameters: list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: Set the character(s) to split on. Default comma (,) Can also prefix with "regex:" to split on a pattern. required: false - role: option + param_group: Options runtime_default: ',' schema: type: string @@ -61,7 +61,7 @@ parameters: Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists. required: false - role: option + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: Select a specific element or range after splitting using slicing syntax. e.g. 0, ":5", "5:", "2:8:2". required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -82,14 +82,14 @@ parameters: - name: inclusive description: If true, include the split character in the output. Default False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/registry/wrangles/split/tokenize.md b/registry/wrangles/split/tokenize.md index bfd3a10..00b6211 100644 --- a/registry/wrangles/split/tokenize.md +++ b/registry/wrangles/split/tokenize.md @@ -33,7 +33,7 @@ parameters: - name: input description: Column(s) to be split into tokens. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -54,7 +54,7 @@ parameters: Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`. required: false - role: option + param_group: Options runtime_default: space schema: anyOf: diff --git a/registry/wrangles/standardize/clean.md b/registry/wrangles/standardize/clean.md index f8a7a9b..4bcbff0 100644 --- a/registry/wrangles/standardize/clean.md +++ b/registry/wrangles/standardize/clean.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. Defaults to overwriting input. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: fix_encoding description: Repair mojibake and other reversible encoding errors. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean @@ -62,7 +62,7 @@ parameters: Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup. required: false - role: option + param_group: Options runtime_default: auto schema: anyOf: @@ -73,7 +73,7 @@ parameters: - name: normalization description: Unicode normalization form. required: false - role: option + param_group: Options runtime_default: NFC schema: type: string @@ -86,49 +86,49 @@ parameters: - name: fix_character_width description: Normalize fullwidth and halfwidth characters. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: uncurl_quotes description: Replace typographic quotes with straight quotes. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: remove_control_chars description: Remove C0 and C1 control characters. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: collapse_whitespace description: Collapse runs of Unicode whitespace. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: preserve_line_breaks description: Preserve line breaks while collapsing other whitespace. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: trim description: Remove leading and trailing whitespace. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: separator description: Text used to join multiple input columns into one output. required: false - role: option + param_group: Formatting runtime_default: ' ' schema: type: string @@ -146,4 +146,4 @@ sources: Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/registry/wrangles/standardize/custom.md b/registry/wrangles/standardize/custom.md index 5f9e07c..a8e98e2 100644 --- a/registry/wrangles/standardize/custom.md +++ b/registry/wrangles/standardize/custom.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: model_id description: The ID of the wrangle to use (do not include 'find' and 'replace'). required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean @@ -78,4 +78,4 @@ sources: Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/wrangles-docs/docusaurus.config.js b/wrangles-docs/docusaurus.config.js index 7ad7e5d..0531f9e 100644 --- a/wrangles-docs/docusaurus.config.js +++ b/wrangles-docs/docusaurus.config.js @@ -46,7 +46,7 @@ const config = { '@docusaurus/plugin-content-docs', /** @type {import('@docusaurus/plugin-content-docs').Options} */ ({ - id: 'registryPilot', + id: 'registry', path: 'registry-docs', routeBasePath: 'wrangles', sidebarPath: './sidebarsRegistry.js', @@ -113,7 +113,7 @@ const config = { { to: '/wrangles/', position: 'left', - label: 'Registry Pilot', + label: 'Registry', }, ], }, diff --git a/wrangles-docs/package.json b/wrangles-docs/package.json index 4763690..2d2eb1e 100644 --- a/wrangles-docs/package.json +++ b/wrangles-docs/package.json @@ -13,7 +13,6 @@ "clear": "docusaurus clear", "compile:registry": "node ./scripts/compile-registry.cjs", "bootstrap:registry": "node ./scripts/bootstrap-registry.cjs", - "refresh:registry-bootstrap": "node ./scripts/bootstrap-registry.cjs --refresh-generated", "check:registry": "node ./scripts/compile-registry.cjs --check", "prebuild": "npm run compile:registry", "generate:wrangle-catalog": "node ./scripts/generate-wrangle-catalog.cjs", diff --git a/wrangles-docs/registry-docs/accordion.md b/wrangles-docs/registry-docs/accordion.md index 9d022ed..2ef9058 100644 --- a/wrangles-docs/registry-docs/accordion.md +++ b/wrangles-docs/registry-docs/accordion.md @@ -3,49 +3,41 @@ title: "Accordion" description: "Apply a series of wrangles to column(s) containing lists. The wrangles will be applied to each element in the list and the results will be returned back as a list." sidebar_label: "Accordion" slug: "/accordion" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Accordion Apply a series of wrangles to column(s) containing lists. The wrangles will be applied to each element in the list and the results will be returned back as a list. -> Pilot Registry entry. Runtime contract status: `verified`. +Apply a series of wrangles to the individual elements of one or more lists. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `wrangles` | Yes | array | List of wrangles to apply. | — | -| `input` | Yes | string, integer, array | The column(s) containing the list(s) that the wrangles will be applied to the elements of. | — | -| `output` | No | string, array, null | Output of the wrangles to save back to the dataframe. | `null` | -| `propagate` | No | string, array, null | Limit the column(s) that will be available to the wrangles and replicated for each element. If not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The column(s) containing the list(s) that the wrangles will be applied to the elements of. | string, integer, array | — | Yes | +| `output` | Output of the wrangles to save back to the dataframe. | string, array, null | `null` | No | +| `propagate` | Limit the column(s) that will be available to the wrangles and replicated for each element. If not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | List of wrangles to apply. | array | — | Yes | -Apply a series of wrangles to the individual elements of one or more lists. +
-## Migrated examples -#### Apply Convert Case to List Elements +## Examples This example applies `convert.case` to each string in a list, where the wrangle would not normally operate on the list as a whole. -##### Recipe + ```yaml wrangles: @@ -61,9 +53,7 @@ wrangles:
-
- -##### Input Sample +
| list_column | | --- | @@ -72,30 +62,46 @@ wrangles:
-
- -##### Output Sample +
-| list_column | modified_lists | -| --- | --- | -| ["a", "b", "c"] | ["A", "B", "C"] | -| ["e", "f", "g"] | ["E", "F", "G"] | +| modified_lists | +| --- | +| ["A", "B", "C"] | +| ["E", "F", "G"] |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `accordion` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.accordion` | + +**Sources** - [WranglesPY accordion implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing accordion Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/accordion.md) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `accordion` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.accordion` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/batch.md b/wrangles-docs/registry-docs/batch.md index e67f6ac..e080f23 100644 --- a/wrangles-docs/registry-docs/batch.md +++ b/wrangles-docs/registry-docs/batch.md @@ -3,51 +3,43 @@ title: "Batch" description: "Split the data into batches for executing a list of wrangles. Use this in situations such as where the intermediate data is too large to fit in memory." sidebar_label: "Batch" slug: "/batch" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Batch Split the data into batches for executing a list of wrangles. Use this in situations such as where the intermediate data is too large to fit in memory. -> Pilot Registry entry. Runtime contract status: `verified`. +Execute a series of wrangles in batches. Batches can run in parallel with `threads` and can provide fallback output when an error occurs. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `wrangles` | Yes | array | The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size. | — | -| `batch_size` | No | integer | The number of rows to split each batch into. | `1000` | -| `threads` | No | integer | The number of threads to use for parallel processing. Default 1. | `1` | -| `on_error` | No | object, null | A dictionary of column_name: value to return if an error occurs while attempting to run a batch. | `null` | -| `timeout` | No | number, null | The number of seconds to wait for a batch to complete before raising an error. | `null` | -| `use_multiprocessing` | No | boolean | Use process-based workers instead of threads. This is an advanced runtime option. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size. | array | — | Yes | +| `batch_size` | The number of rows to split each batch into. | integer | `1000` | No | +| `threads` | The number of threads to use for parallel processing. Default 1. | integer | `1` | No | +| `timeout` | The number of seconds to wait for a batch to complete before raising an error. | number, null | `null` | No | +| `use_multiprocessing` | Use process-based workers instead of threads. This is an advanced runtime option. | boolean | `false` | No | +| Errors | | | | | +| `on_error` | A dictionary of column_name: value to return if an error occurs while attempting to run a batch. | object, null | `null` | No | -Execute a series of wrangles in batches. Batches can run in parallel with `threads` and can provide fallback output when an error occurs. +
-## Migrated examples -#### Batch an Extract AI Wrangle +## Examples This example processes product descriptions in batches of two rows. -##### Recipe + ```yaml wrangles: @@ -66,9 +58,7 @@ wrangles:
-
- -##### Input Sample +
| Product Description | | --- | @@ -79,32 +69,48 @@ wrangles:
-
- -##### Output Sample +
-| Product Description | Title | -| --- | --- | -| Sleep better with our Memory Foam Pillow, designed to contour to your head and neck. | Memory Foam Pillow | -| Stay comfortable and stylish with our Organic Cotton T-Shirt, made from soft, breathable fabric. | Organic Cotton T-Shirt | -| Keep drinks hot or cold with our Stainless Steel Water Bottle, featuring durable insulation. | Stainless Steel Water Bottle | -| Enjoy crisp sound and long battery life with our Wireless Bluetooth Earbuds. | Wireless Bluetooth Earbuds | +| Title | +| --- | +| Memory Foam Pillow | +| Organic Cotton T-Shirt | +| Stainless Steel Water Bottle | +| Wireless Bluetooth Earbuds |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `batch` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.batch` | + +**Sources** - [WranglesPY batch implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing batch Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/batch.md) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `batch` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.batch` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/classify.md b/wrangles-docs/registry-docs/classify.md index 13ea36a..76b92a0 100644 --- a/wrangles-docs/registry-docs/classify.md +++ b/wrangles-docs/registry-docs/classify.md @@ -3,47 +3,38 @@ title: "Classify" description: "Run classify wrangles on the specified columns. Requires WrangleWorks Account and Subscription." sidebar_label: "Classify" slug: "/classify" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Classify Run classify wrangles on the specified columns. Requires WrangleWorks Account and Subscription. -> Pilot Registry entry. Runtime contract status: `verified`. +Run a custom classification wrangle on the specified column or columns. A classification wrangle must be trained first. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | Yes | string, array | Name of the output column. | — | -| `model_id` | Yes | string | ID of the classification model to be used. | — | -| `include_confidence` | No | boolean | For models that support it, include the confidence level in the output. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Run a custom classification wrangle on the specified column or columns. A classification wrangle must be trained first. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Formatting | | | | | +| `include_confidence` | For models that support it, include the confidence level in the output. | boolean | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | ID of the classification model to be used. | string | — | Yes | -## Migrated examples -#### Food Type Example +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +47,7 @@ wrangles:
-
- -##### Input Sample +
| Products | | --- | @@ -67,30 +56,46 @@ wrangles:
-
- -##### Output Sample +
-| Products | Category | -| --- | --- | -| Rice | | -| Milk | Dairy | +| Category | +| --- | +| | +| Dairy |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `classify` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `ai` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.classify` | + +**Sources** - [WranglesPY classify implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing classify Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/classify.md) -## Registry metadata - -- Registry ID: `06669ef8-cdd1-42f7-8078-98e0b7a42c30` -- Namespace: root-level runtime key -- Recipe key: `classify` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.classify` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/clean-whitespaces.md b/wrangles-docs/registry-docs/clean-whitespaces.md index 415b73f..79d5e33 100644 --- a/wrangles-docs/registry-docs/clean-whitespaces.md +++ b/wrangles-docs/registry-docs/clean-whitespaces.md @@ -3,47 +3,37 @@ title: "Clean Whitespaces" description: "Condense multiple spaces to a single space and convert special space characters to a standard space." sidebar_label: "Clean Whitespaces" slug: "/clean-whitespaces" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Clean Whitespaces Condense multiple spaces to a single space and convert special space characters to a standard space. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `trim` | No | boolean | Whether to trim leading and trailing spaces. Default True. | `true` | -| `remove_literals` | No | boolean | Whether to remove special space characters such as new lines etc. Default True. | `true` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Condense multiple spaces to a single space and convert special space characters to a standard space. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `trim` | Whether to trim leading and trailing spaces. Default True. | boolean | `true` | No | +| `remove_literals` | Whether to remove special space characters such as new lines etc. Default True. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Food Type Example +
-##### Recipe +## Examples ```yaml wrangles: @@ -53,9 +43,7 @@ wrangles:
-
- -##### Input Sample +
| Products | | --- | @@ -64,9 +52,7 @@ wrangles:
-
- -##### Output Sample +
| Products | | --- | @@ -77,17 +63,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `clean_whitespaces` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.clean_whitespaces` | + +**Sources** - [WranglesPY clean_whitespaces implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing clean_whitespaces Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/clean-whitespaces.md) -## Registry metadata - -- Registry ID: `e36e15c4-f0ad-43f8-8555-ef683a8ab892` -- Namespace: root-level runtime key -- Recipe key: `clean_whitespaces` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.clean_whitespaces` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/compare/lists.md b/wrangles-docs/registry-docs/compare/lists.md index 910c1fd..8c1d67e 100644 --- a/wrangles-docs/registry-docs/compare/lists.md +++ b/wrangles-docs/registry-docs/compare/lists.md @@ -3,48 +3,38 @@ title: "Lists" description: "Compare multiple lists and return the intersection, difference, or union." sidebar_label: "Lists" slug: "/compare/lists" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Lists Compare multiple lists and return the intersection, difference, or union. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns containing lists to compare. | — | -| `output` | Yes | string | Name of the output column. | — | -| `method` | No | string; one of: intersection, difference, union | Type of comparison to perform. | `"intersection"` | -| `remove_duplicates` | No | boolean | Remove duplicates from the result. | `false` | -| `ignore_case` | No | boolean | Ignore case when comparing string items. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Compare multiple lists and return the intersection, difference, or union. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns containing lists to compare. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `method` | Type of comparison to perform. | string; one of:
  • intersection
  • difference
  • union
| `"intersection"` | No | +| `remove_duplicates` | Remove duplicates from the result. | boolean | `false` | No | +| `ignore_case` | Ignore case when comparing string items. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Comparing the difference between two columns of lists +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,9 +48,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -70,9 +58,7 @@ wrangles:
-
- -##### Output Sample +
| Difference | | --- | @@ -84,9 +70,9 @@ wrangles:
-#### Comparing the intersection between two columns of lists -##### Recipe + + ```yaml wrangles: @@ -100,9 +86,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -112,9 +96,7 @@ wrangles:
-
- -##### Output Sample +
| Intersection | | --- | @@ -126,9 +108,9 @@ wrangles:
-#### Comparing the union between two columns of lists -##### Recipe + + ```yaml wrangles: @@ -142,9 +124,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -154,9 +134,7 @@ wrangles:
-
- -##### Output Sample +
| Union | | --- | @@ -168,17 +146,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compare.lists` | +| Lifecycle status | active | +| Namespace | `compare` | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compare.lists` | + +**Sources** - [WranglesPY compare.lists implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compare.py) - [Existing compare.lists Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/lists.md) -## Registry metadata - -- Registry ID: `a393225e-7ccf-4708-83f8-d5abd6ba9b1e` -- Namespace: `compare` -- Recipe key: `compare.lists` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.compare.lists` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/compare/text.md b/wrangles-docs/registry-docs/compare/text.md index 727a9f6..86d3b08 100644 --- a/wrangles-docs/registry-docs/compare/text.md +++ b/wrangles-docs/registry-docs/compare/text.md @@ -3,56 +3,47 @@ title: "Text" description: "Compare two strings and return the intersection or difference, use overlap to find the matching characters between the two strings, or use similarity to get a numeric similarity score." sidebar_label: "Text" slug: "/compare/text" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Text Compare two strings and return the intersection or difference, use overlap to find the matching characters between the two strings, or use similarity to get a numeric similarity score. -> Pilot Registry entry. Runtime contract status: `verified`. +Compare two strings and return the intersection or difference, or use overlap to find the matching characters between the two strings. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | The columns to compare. First column is the base column. | — | -| `output` | Yes | string, array | The column to output the results to. Must be a list of two column names [mask_column, ratio_column] when method is overlap and include_ratio is true; otherwise a single column name. | — | -| `method` | No | string; one of: difference, intersection, overlap, similarity | The type of comparison to perform (difference, intersection, overlap, similarity). | `"difference"` | -| `char` | No | string | Character to split strings on for difference and intersection. Defaults to a space. | `" "` | -| `non_match_char` | No | string | Character to use for non-matching characters when using overlap. | `"*"` | -| `include_ratio` | No | boolean | Include the ratio of matching characters when using overlap. | `false` | -| `decimal_places` | No | integer | Number of decimal places to round the overlap ratio to. | `3` | -| `exact_match` | No | string, null | Value to use for exact matches when using overlap. | `null` | -| `empty_a` | No | string, null | Value to use when input A is empty when using overlap. | `null` | -| `empty_b` | No | string, null | Value to use when input B is empty when using overlap. | `null` | -| `all_empty` | No | string, null | Value to use when both inputs are empty when using overlap. | `null` | -| `case_sensitive` | No | boolean | Whether the comparison is case sensitive. Defaults to true. | `false` | -| `metric` | No | string | Metric value accepted by the runtime. | `"token_sort"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Compare two strings and return the intersection or difference, or use overlap to find the matching characters between the two strings. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The columns to compare. First column is the base column. | array | — | Yes | +| `output` | The column to output the results to. Must be a list of two column names [mask_column, ratio_column] when method is overlap and include_ratio is true; otherwise a single column name. | string, array | — | Yes | +| Options | | | | | +| `method` | The type of comparison to perform (difference, intersection, overlap, similarity). | string; one of:
  • difference
  • intersection
  • overlap
  • similarity
| `"difference"` | No | +| `char` | Character to split strings on for difference and intersection. Defaults to a space. | string | `" "` | No | +| `exact_match` | Value to use for exact matches when using overlap. | string, null | `null` | No | +| `empty_a` | Value to use when input A is empty when using overlap. | string, null | `null` | No | +| `empty_b` | Value to use when input B is empty when using overlap. | string, null | `null` | No | +| `all_empty` | Value to use when both inputs are empty when using overlap. | string, null | `null` | No | +| `case_sensitive` | Whether the comparison is case sensitive. Defaults to true. | boolean | `false` | No | +| `metric` | Metric value accepted by the runtime. | string | `"token_sort"` | No | +| Formatting | | | | | +| `non_match_char` | Character to use for non-matching characters when using overlap. | string | `"*"` | No | +| `include_ratio` | Include the ratio of matching characters when using overlap. | boolean | `false` | No | +| `decimal_places` | Number of decimal places to round the overlap ratio to. | integer | `3` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Comparing the difference between two columns of text +
-##### Recipe +## Examples ```yaml wrangles: @@ -66,9 +57,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -78,9 +67,7 @@ wrangles:
-
- -##### Output Sample +
| Difference | | --- | @@ -92,9 +79,9 @@ wrangles:
-#### Comparing the intersection of two columns of text -##### Recipe + + ```yaml wrangles: @@ -108,9 +95,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -120,9 +105,7 @@ wrangles:
-
- -##### Output Sample +
| Intersection | | --- | @@ -134,9 +117,9 @@ wrangles:
-#### Comparing the overlap of two columns of text -##### Recipe + + ```yaml wrangles: @@ -150,9 +133,7 @@ wrangles:
-
- -##### Input Sample +
| Part Code1 | Part Code2 | | --- | --- | @@ -162,9 +143,7 @@ wrangles:
-
- -##### Output Sample +
| Overlap | | --- | @@ -176,17 +155,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compare.text` | +| Lifecycle status | active | +| Namespace | `compare` | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compare.text` | + +**Sources** - [WranglesPY compare.text implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compare.py) - [Existing compare.text Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/text.md) -## Registry metadata - -- Registry ID: `31905b74-ce58-45cd-8add-821cc04ab946` -- Namespace: `compare` -- Recipe key: `compare.text` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.compare.text` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/compute/case-when.md b/wrangles-docs/registry-docs/compute/case-when.md index 628a9f2..c052e21 100644 --- a/wrangles-docs/registry-docs/compute/case-when.md +++ b/wrangles-docs/registry-docs/compute/case-when.md @@ -3,46 +3,36 @@ title: "Case When" description: "Assign values to a column based on conditional logic." sidebar_label: "Case When" slug: "/compute/case-when" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Case When Assign values to a column based on conditional logic. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `output` | Yes | string | Name of the output column. | — | -| `cases` | Yes | array | List of conditions and corresponding values. | — | -| `default` | No | string, number, integer, boolean, null | Value to assign if no conditions are met. Default None. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Assign values to a column based on conditional logic. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `cases` | List of conditions and corresponding values. | array | — | Yes | +| `default` | Value to assign if no conditions are met. Default None. | string, number, integer, boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Assigning Letter Grades +
-##### Recipe +## Examples ```yaml wrangles: @@ -61,9 +51,7 @@ wrangles:
-
- -##### Input Sample +
| Student | Grade | | --- | --- | @@ -74,26 +62,24 @@ wrangles:
-
- -##### Output Sample +
-| Student | Grade | Letter Grade | -| --- | --- | --- | -| Billy | 62 | F | -| Sarah | 91 | A | -| Timmy | 88 | B | -| Tammy | 74 | C | +| Letter Grade | +| --- | +| F | +| A | +| B | +| C |
-#### Assigning Letter Grades with Attendance + Conditions can be combined so multiple criteria must be met before assigning a value. -##### Recipe + ```yaml wrangles: @@ -112,9 +98,7 @@ wrangles:
-
- -##### Input Sample +
| Student | Grade | Attendance | | --- | --- | --- | @@ -125,32 +109,48 @@ wrangles:
-
- -##### Output Sample +
-| Student | Grade | Attendance | Letter Grade | -| --- | --- | --- | --- | -| Billy | 62 | Poor | F | -| Sarah | 91 | Poor | F | -| Timmy | 88 | Good | B | -| Tammy | 74 | Good | C | +| Letter Grade | +| --- | +| F | +| F | +| B | +| C |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compute.case_when` | +| Lifecycle status | active | +| Namespace | `compute` | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compute.case_when` | + +**Sources** - [WranglesPY compute.case_when implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compute.py) - [Existing compute.case_when Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/case-when.md) -## Registry metadata - -- Registry ID: `9a9662e4-53d1-4932-8adf-bc3e7aa364ad` -- Namespace: `compute` -- Recipe key: `compute.case_when` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.compute.case_when` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/compute/score-search-results.md b/wrangles-docs/registry-docs/compute/score-search-results.md index 3616d02..37e3aad 100644 --- a/wrangles-docs/registry-docs/compute/score-search-results.md +++ b/wrangles-docs/registry-docs/compute/score-search-results.md @@ -3,61 +3,53 @@ title: "Score Search Results" description: "Scores and filters search results based on progressive partial/exact matching. Can return dictionaries or a parallel list of formatted strings." sidebar_label: "Score Search Results" slug: "/compute/score-search-results" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Score Search Results Scores and filters search results based on progressive partial/exact matching. Can return dictionaries or a parallel list of formatted strings. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)]. | — | -| `output` | Yes | string, array | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | — | -| `must_match_part_code` | No | boolean | If true, filters out results that don't satisfy the allowed match types. | `true` | -| `allow_mpn_exact` | No | boolean | Treat exact MPN matches as valid part code matches. | `true` | -| `allow_mpn_partial` | No | boolean | Treat partial MPN matches as valid part code matches. | `true` | -| `allow_other_exact` | No | boolean | Treat exact other part code matches as valid part code matches. | `true` | -| `allow_other_partial` | No | boolean | Treat partial other part code matches as valid part code matches. | `true` | -| `blacklist_keywords` | No | string | Comma-separated list or array of keywords to filter out URLs containing them. | `""` | -| `mpn_exact_score` | No | number | Mpn Exact Score value accepted by the runtime. | `8` | -| `mpn_partial_base` | No | number | Mpn Partial Base value accepted by the runtime. | `4` | -| `part_code_exact_score` | No | number | Part Code Exact Score value accepted by the runtime. | `6` | -| `part_code_partial_base` | No | number | Part Code Partial Base value accepted by the runtime. | `2` | -| `supplier_exact_score` | No | number | Supplier Exact Score value accepted by the runtime. | `3` | -| `supplier_partial_base` | No | number | Supplier Partial Base value accepted by the runtime. | `1` | -| `context_match_base` | No | number | Context Match Base value accepted by the runtime. | `2` | -| `fuzzy_match_threshold` | No | number | Fuzzy Match Threshold value accepted by the runtime. | `0.8` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +## Parameters -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)]. | array | — | Yes | +| `output` | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | string, array | — | Yes | +| Options | | | | | +| `must_match_part_code` | If true, filters out results that don't satisfy the allowed match types. | boolean | `true` | No | +| `allow_mpn_exact` | Treat exact MPN matches as valid part code matches. | boolean | `true` | No | +| `allow_mpn_partial` | Treat partial MPN matches as valid part code matches. | boolean | `true` | No | +| `allow_other_exact` | Treat exact other part code matches as valid part code matches. | boolean | `true` | No | +| `allow_other_partial` | Treat partial other part code matches as valid part code matches. | boolean | `true` | No | +| `blacklist_keywords` | Comma-separated list or array of keywords to filter out URLs containing them. | string | `""` | No | +| `mpn_exact_score` | Mpn Exact Score value accepted by the runtime. | number | `8` | No | +| `mpn_partial_base` | Mpn Partial Base value accepted by the runtime. | number | `4` | No | +| `part_code_exact_score` | Part Code Exact Score value accepted by the runtime. | number | `6` | No | +| `part_code_partial_base` | Part Code Partial Base value accepted by the runtime. | number | `2` | No | +| `supplier_exact_score` | Supplier Exact Score value accepted by the runtime. | number | `3` | No | +| `supplier_partial_base` | Supplier Partial Base value accepted by the runtime. | number | `1` | No | +| `context_match_base` | Context Match Base value accepted by the runtime. | number | `2` | No | +| `fuzzy_match_threshold` | Fuzzy Match Threshold value accepted by the runtime. | number | `0.8` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -Scores and filters search results based on progressive partial/exact matching. Can return dictionaries or a parallel list of formatted strings. +
-## Migrated examples -#### Rank Part Search Results +## Examples This template scores search results using supplier and part-code context. The fields added to each result dictionary depend on the scoring configuration. -##### Recipe + ```yaml wrangles: @@ -74,9 +66,7 @@ wrangles:
-
- -##### Input Sample +
| Search Results | Suppliers | Part Codes | | --- | --- | --- | @@ -84,13 +74,11 @@ wrangles:
-
- -##### Output Sample +
-| Search Results | Suppliers | Part Codes | Scored Results | -| --- | --- | --- | --- | -| `[{"title": "SKF 6202 bearing", "link": "https://example.com/6202"}]` | `["SKF"]` | `["6202"]` | Ranked result dictionaries | +| Scored Results | +| --- | +| Ranked result dictionaries |
@@ -98,17 +86,35 @@ wrangles: _Template based on the documented parameters; no published source example is currently available._ -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compute.score_search_results` | +| Lifecycle status | active | +| Namespace | `compute` | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compute.score_search_results` | + +**Sources** - [WranglesPY compute.score_search_results implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compute.py) - [Existing compute.score_search_results Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/score-search-results.md) -## Registry metadata - -- Registry ID: `313a8ec0-cf13-4956-8d3b-5362b8641d0f` -- Namespace: `compute` -- Recipe key: `compute.score_search_results` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.compute.score_search_results` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/concurrent.md b/wrangles-docs/registry-docs/concurrent.md index 17e64a2..a9d17fa 100644 --- a/wrangles-docs/registry-docs/concurrent.md +++ b/wrangles-docs/registry-docs/concurrent.md @@ -3,48 +3,37 @@ title: "Concurrent" description: "Run multiple wrangles concurrently rather than sequentially. Wrangles must specify output columns to be used concurrently. When using concurrent, Wrangles may not complete in a predictable order and it is not recommended to update overlapping columns with different wrangles." sidebar_label: "Concurrent" slug: "/concurrent" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Concurrent Run multiple wrangles concurrently rather than sequentially. Wrangles must specify output columns to be used concurrently. When using concurrent, Wrangles may not complete in a predictable order and it is not recommended to update overlapping columns with different wrangles. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `wrangles` | Yes | array | The wrangles section of a recipe to execute for each combination of variables. | — | -| `max_concurrency` | No | integer | The maximum number of wrangles to execute in parallel. | `10` | -| `use_multiprocessing` | No | boolean | Use process-based workers instead of threads. This is an advanced runtime option. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +Run multiple wrangles concurrently instead of sequentially. Concurrent wrangles must declare output columns, may finish in any order, and should not update overlapping columns. -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +See the [Concurrent connector](/python/connectors/concurrent) for the connector equivalent. -## Guidance +## Parameters -Run multiple wrangles concurrently instead of sequentially. Concurrent wrangles must declare output columns, may finish in any order, and should not update overlapping columns. +
-See the [Concurrent connector](/python/connectors/concurrent) for the connector equivalent. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | The wrangles section of a recipe to execute for each combination of variables. | array | — | Yes | +| `max_concurrency` | The maximum number of wrangles to execute in parallel. | integer | `10` | No | +| `use_multiprocessing` | Use process-based workers instead of threads. This is an advanced runtime option. | boolean | `false` | No | -## Migrated examples -#### Run Extraction Wrangles Concurrently +
-##### Recipe +## Examples ```yaml wrangles: @@ -60,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| Products | | --- | @@ -72,31 +59,47 @@ wrangles:
-
+
-##### Output Sample - -| Products | Part Codes | Attributes | -| --- | --- | --- | -| SKF ball brg 2" od 6202 | 6202 | `{"length":["2in"]}` | -| brg seal 1" id 5493 | 5493 | `{"length":["1in"]}` | -| 3lb hammer 87102 | 87102 | `{"weight":["3lb"]}` | +| Part Codes | Attributes | +| --- | --- | +| 6202 | `{"length":["2in"]}` | +| 5493 | `{"length":["1in"]}` | +| 87102 | `{"weight":["3lb"]}` |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `concurrent` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.concurrent` | + +**Sources** - [WranglesPY concurrent implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing concurrent Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/concurrent.md) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `concurrent` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.concurrent` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/case.md b/wrangles-docs/registry-docs/convert/case.md index 8eb56c3..9138a41 100644 --- a/wrangles-docs/registry-docs/convert/case.md +++ b/wrangles-docs/registry-docs/convert/case.md @@ -3,30 +3,44 @@ title: "Convert Case" description: "Change the letter case of text values." sidebar_label: "Convert Case" slug: "/convert/case" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Convert Case Change the letter case of text values. -> Pilot Registry entry. Runtime contract status: `verified`. +Use `convert.case` to normalize capitalization while keeping the source column +or writing the result to a new column. + +## Behavior + +- Supported modes are `lower`, `upper`, `title`, and `sentence`. +- Omitting `output` overwrites the input column. +- Input and output lists must have equal lengths. +- Non-string values are passed through unchanged and produce a warning. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name, index, or list of input columns. | — | -| `output` | No | string, array | Name or list of output columns. If omitted, each input column is overwritten. | `null` | -| `case` | No | string; one of: lower, upper, title, sentence | Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts. | `"lower"` | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | +
-## Verified examples +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Options | | | | | +| `case` | Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts. | string; one of:
  • lower
  • upper
  • title
  • sentence
| `"lower"` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | -### Write uppercase text to a new column +
-Verification: `static` +## Examples ```yaml wrangles: @@ -36,66 +50,57 @@ wrangles: case: upper ``` -#### Input - -```json -[ - { - "product": "Cordless Drill" - }, - { - "product": "bearing seal" - } -] -``` +
-#### Output - -```json -[ - { - "product": "Cordless Drill", - "product_upper": "CORDLESS DRILL" - }, - { - "product": "bearing seal", - "product_upper": "BEARING SEAL" - } -] -``` +
+ +| product | +| --- | +| Cordless Drill | +| bearing seal | + +
-## Access +
+ +| product_upper | +| --- | +| CORDLESS DRILL | +| BEARING SEAL | + +
+ +
+ +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-Use `convert.case` to normalize capitalization while keeping the source column -or writing the result to a new column. +
-## Behavior +Technical details -- Supported modes are `lower`, `upper`, `title`, and `sentence`. -- Omitting `output` overwrites the input column. -- Input and output lists must have equal lengths. -- Non-string values are passed through unchanged and produce a warning. +| Field | Value | +| --- | --- | +| Recipe key | `convert.case` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.case` | -## Provenance +**Sources** - [WranglesPY convert.case implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Legacy convert.case documentation](https://wrangles.io/python/recipes/wrangles/convert#case) -## Registry metadata - -- Registry ID: `12ff4120-3613-4801-8653-99c793477fbc` -- Namespace: `convert` -- Recipe key: `convert.case` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.case` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/data-type.md b/wrangles-docs/registry-docs/convert/data-type.md index a5979f0..1c288f8 100644 --- a/wrangles-docs/registry-docs/convert/data-type.md +++ b/wrangles-docs/registry-docs/convert/data-type.md @@ -3,31 +3,48 @@ title: "Convert Data Type" description: "Convert values to strings, numbers, booleans, or datetimes." sidebar_label: "Convert Data Type" slug: "/convert/data-type" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Convert Data Type Convert values to strings, numbers, booleans, or datetimes. -> Pilot Registry entry. Runtime contract status: `verified`. +Use `convert.data_type` when a recipe needs consistent Python-compatible +values rather than display-only formatting. -## Parameters +## Behavior -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name, index, or list of input columns. | — | -| `output` | No | string, array | Name or list of output columns. If omitted, each input column is overwritten. | `null` | -| `data_type` | No | string; one of: str, float, int, bool, datetime | Data type to produce. | `"str"` | -| `default` | No | string, number, array, object, boolean, null | Value returned when conversion fails. If omitted, the original value is retained. | `null` | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | +- Supported target types are `str`, `float`, `int`, `bool`, and `datetime`. +- Omitting `data_type` uses the runtime default `str`. +- Omitting `output` overwrites the input column. +- Failed conversions retain the original value unless `default` is supplied. +- Additional undocumented keyword arguments are implementation details and are + not part of the public Registry contract. -## Verified examples +## Parameters -### Convert quantities to integers with a fallback +
-Verification: `static` +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Options | | | | | +| `data_type` | Data type to produce. | string; one of:
  • str
  • float
  • int
  • bool
  • datetime
| `"str"` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Errors | | | | | +| `default` | Value returned when conversion fails. If omitted, the original value is retained. | string, number, array, object, boolean, null | `null` | No | + +
+ +## Examples ```yaml wrangles: @@ -38,68 +55,57 @@ wrangles: default: 0 ``` -#### Input - -```json -[ - { - "quantity": "12" - }, - { - "quantity": "not available" - } -] -``` +
-#### Output - -```json -[ - { - "quantity": "12", - "quantity_integer": 12 - }, - { - "quantity": "not available", - "quantity_integer": 0 - } -] -``` +
+ +| quantity | +| --- | +| 12 | +| not available | + +
-## Access +
+ +| quantity_integer | +| --- | +| 12 | +| 0 | + +
+ +
+ +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-Use `convert.data_type` when a recipe needs consistent Python-compatible -values rather than display-only formatting. +
-## Behavior +Technical details -- Supported target types are `str`, `float`, `int`, `bool`, and `datetime`. -- Omitting `data_type` uses the runtime default `str`. -- Omitting `output` overwrites the input column. -- Failed conversions retain the original value unless `default` is supplied. -- Additional undocumented keyword arguments are implementation details and are - not part of the public Registry contract. +| Field | Value | +| --- | --- | +| Recipe key | `convert.data_type` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.data_type` | -## Provenance +**Sources** - [WranglesPY convert.data_type implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Legacy convert.data_type documentation](https://wrangles.io/python/recipes/wrangles/convert#data-type) -## Registry metadata - -- Registry ID: `52384f01-7164-404f-8615-063e7677a588` -- Namespace: `convert` -- Recipe key: `convert.data_type` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.data_type` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/fraction-to-decimal.md b/wrangles-docs/registry-docs/convert/fraction-to-decimal.md index 302c0a9..832f523 100644 --- a/wrangles-docs/registry-docs/convert/fraction-to-decimal.md +++ b/wrangles-docs/registry-docs/convert/fraction-to-decimal.md @@ -3,46 +3,36 @@ title: "Fraction to Decimal" description: "Convert fractions to decimals." sidebar_label: "Fraction to Decimal" slug: "/convert/fraction-to-decimal" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Fraction to Decimal Convert fractions to decimals. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `decimals` | No | integer | Number of decimals to round fraction. | `4` | -| `output` | No | string, array, null | Name of the output colum. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Convert fractions to decimals. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output colum. | string, array, null | `null` | No | +| Formatting | | | | | +| `decimals` | Number of decimals to round fraction. | integer | `4` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Converting a Column of Fractions to Decimals +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| fractions | | --- | @@ -65,9 +53,7 @@ wrangles:
-
- -##### Output Sample +
| decimals | | --- | @@ -78,17 +64,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.fraction_to_decimal` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.fraction_to_decimal` | + +**Sources** - [WranglesPY convert.fraction_to_decimal implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Existing convert.fraction_to_decimal Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/fraction-to-decimal.md) -## Registry metadata - -- Registry ID: `7c55752e-70ab-4809-8298-c59436127457` -- Namespace: `convert` -- Recipe key: `convert.fraction_to_decimal` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.fraction_to_decimal` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/from-json.md b/wrangles-docs/registry-docs/convert/from-json.md index 18c1619..4cdbf40 100644 --- a/wrangles-docs/registry-docs/convert/from-json.md +++ b/wrangles-docs/registry-docs/convert/from-json.md @@ -3,30 +3,46 @@ title: "Convert From JSON" description: "Parse JSON text into lists, objects, scalars, booleans, or null values." sidebar_label: "Convert From JSON" slug: "/convert/from-json" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Convert From JSON Parse JSON text into lists, objects, scalars, booleans, or null values. -> Pilot Registry entry. Runtime contract status: `verified`. +Use `convert.from_json` when a column contains JSON text that later wrangles +need to treat as structured values. + +## Behavior + +- Each non-fallback value is parsed with Python's JSON parser. +- Omitting `output` overwrites the input column. +- Multiple input columns may share one fallback or use one fallback per input. +- Invalid JSON raises an error unless a non-null fallback is supplied. +- Permissive Python-literal or YAML-like parsing is outside this wrangle's + contract. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name, index, or list of columns containing valid JSON text. | — | -| `output` | No | string, array | Name or list of output columns. If omitted, each input column is overwritten. | `null` | -| `default` | No | string, number, array, object, boolean, null | Value returned for empty or invalid JSON. A list may supply one fallback per input column. | `null` | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | +
-## Verified examples +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of columns containing valid JSON text. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Errors | | | | | +| `default` | Value returned for empty or invalid JSON. A list may supply one fallback per input column. | string, number, array, object, boolean, null | `null` | No | -### Parse JSON objects and lists +
-Verification: `static` +## Examples ```yaml wrangles: @@ -35,74 +51,57 @@ wrangles: output: attributes ``` -#### Input - -```json -[ - { - "attributes_json": "{\"material\":\"steel\",\"voltage\":18}" - }, - { - "attributes_json": "[\"corded\",\"variable speed\"]" - } -] -``` +
-#### Output - -```json -[ - { - "attributes_json": "{\"material\":\"steel\",\"voltage\":18}", - "attributes": { - "material": "steel", - "voltage": 18 - } - }, - { - "attributes_json": "[\"corded\",\"variable speed\"]", - "attributes": [ - "corded", - "variable speed" - ] - } -] -``` +
+ +| attributes_json | +| --- | +| {"material":"steel","voltage":18} | +| ["corded","variable speed"] | + +
-## Access +
+ +| attributes | +| --- | +| {"material":"steel","voltage":18} | +| ["corded","variable speed"] | + +
+ +
+ +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-Use `convert.from_json` when a column contains JSON text that later wrangles -need to treat as structured values. +
-## Behavior +Technical details -- Each non-fallback value is parsed with Python's JSON parser. -- Omitting `output` overwrites the input column. -- Multiple input columns may share one fallback or use one fallback per input. -- Invalid JSON raises an error unless a non-null fallback is supplied. -- Permissive Python-literal or YAML-like parsing is outside this wrangle's - contract. +| Field | Value | +| --- | --- | +| Recipe key | `convert.from_json` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.from_json` | -## Provenance +**Sources** - [WranglesPY convert.from_json implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Legacy convert.from_json documentation](https://wrangles.io/python/recipes/wrangles/convert#from-json) -## Registry metadata - -- Registry ID: `e370dfcf-b0fe-4c48-8a52-6f34c47e7978` -- Namespace: `convert` -- Recipe key: `convert.from_json` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.from_json` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/from-yaml.md b/wrangles-docs/registry-docs/convert/from-yaml.md index 92af3e0..9844b44 100644 --- a/wrangles-docs/registry-docs/convert/from-yaml.md +++ b/wrangles-docs/registry-docs/convert/from-yaml.md @@ -3,46 +3,36 @@ title: "From YAML" description: "Convert a YAML representation into an object." sidebar_label: "From YAML" slug: "/convert/from-yaml" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # From YAML Convert a YAML representation into an object. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. If omitted, the input column will be overwritten. | `null` | -| `default` | No | string, array, object, number, boolean, null | Value to return if the row is empty or fails to be parsed as YAML. If input is a list, default may also be a list - either a single value to apply to all columns, or one value per input column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Convert a YAML representation into an object. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Value to return if the row is empty or fails to be parsed as YAML. If input is a list, default may also be a list - either a single value to apply to all columns, or one value per input column. | string, array, object, number, boolean, null | `null` | No | -## Migrated examples -#### Convert YAML To An Object +
-##### Recipe +## Examples ```yaml wrangles: @@ -52,9 +42,7 @@ wrangles:
-
- -##### Input Sample +
```yaml Product Specs: @@ -65,9 +53,7 @@ Product Specs:
-
- -##### Output Sample +
```python { @@ -83,17 +69,35 @@ Product Specs:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.from_yaml` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.from_yaml` | + +**Sources** - [WranglesPY convert.from_yaml implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Existing convert.from_yaml Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/from-yaml.md) -## Registry metadata - -- Registry ID: `b1c20004-5f80-41c2-84d2-2d4601a033b5` -- Namespace: `convert` -- Recipe key: `convert.from_yaml` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.from_yaml` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/to-json.md b/wrangles-docs/registry-docs/convert/to-json.md index 44f69a5..5fadcf6 100644 --- a/wrangles-docs/registry-docs/convert/to-json.md +++ b/wrangles-docs/registry-docs/convert/to-json.md @@ -3,48 +3,38 @@ title: "To JSON" description: "Convert an object to a JSON representation." sidebar_label: "To JSON" slug: "/convert/to-json" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # To JSON Convert an object to a JSON representation. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. If omitted, the input column will be overwritten. | `null` | -| `ensure_ascii` | No | boolean | If true, non-ASCII characters will be escaped. Default is false. | `false` | -| `indent` | No | string, integer | If indent is a non-negative integer or string, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, negative, or "" will only insert newlines. None (the default) selects the most compact representation. Using a positive integer indent indents that many spaces per level. If indent is a string (such as '\t'), that string is used to indent each level. | — | -| `sort_keys` | No | boolean | If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Convert an object to a JSON representation. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Formatting | | | | | +| `ensure_ascii` | If true, non-ASCII characters will be escaped. Default is false. | boolean | `false` | No | +| `indent` | If indent is a non-negative integer or string, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, negative, or "" will only insert newlines. None (the default) selects the most compact representation. Using a positive integer indent indents that many spaces per level. If indent is a string (such as '\t'), that string is used to indent each level. | string, integer | — | No | +| `sort_keys` | If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key. | boolean | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Convert Text to JSON +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| column | | --- | @@ -66,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| new column | | --- | @@ -79,17 +65,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.to_json` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.to_json` | + +**Sources** - [WranglesPY convert.to_json implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Existing convert.to_json Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/to-json.md) -## Registry metadata - -- Registry ID: `94e54eb7-2b8c-4047-89d0-fb5d16baf396` -- Namespace: `convert` -- Recipe key: `convert.to_json` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.to_json` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/convert/to-yaml.md b/wrangles-docs/registry-docs/convert/to-yaml.md index 53cd376..5c9216b 100644 --- a/wrangles-docs/registry-docs/convert/to-yaml.md +++ b/wrangles-docs/registry-docs/convert/to-yaml.md @@ -3,48 +3,38 @@ title: "To YAML" description: "Convert an object to a YAML representation." sidebar_label: "To YAML" slug: "/convert/to-yaml" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # To YAML Convert an object to a YAML representation. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. If omitted, the input column will be overwritten. | `null` | -| `sort_keys` | No | boolean | If sort_keys is true (default: False), then the output of dictionaries will be sorted by key. | `false` | -| `allow_unicode` | No | boolean | Allow Unicode value accepted by the runtime. | `true` | -| `indent` | No | integer | Specify the number of spaces for indentation to specify nested elements. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Convert an object to a YAML representation. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Formatting | | | | | +| `sort_keys` | If sort_keys is true (default: False), then the output of dictionaries will be sorted by key. | boolean | `false` | No | +| `allow_unicode` | Allow Unicode value accepted by the runtime. | boolean | `true` | No | +| `indent` | Specify the number of spaces for indentation to specify nested elements. | integer | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Convert a Dictionary to YAML +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
```python { @@ -71,9 +59,7 @@ wrangles:
-
- -##### Output Sample +
```yaml Product Specs: @@ -86,17 +72,35 @@ Product Specs:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.to_yaml` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.to_yaml` | + +**Sources** - [WranglesPY convert.to_yaml implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) - [Existing convert.to_yaml Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/to-yaml.md) -## Registry metadata - -- Registry ID: `4cd6252f-ce47-4a9d-8272-3d87e875b72a` -- Namespace: `convert` -- Recipe key: `convert.to_yaml` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.convert.to_yaml` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/copy.md b/wrangles-docs/registry-docs/copy.md index 082b204..086bd4d 100644 --- a/wrangles-docs/registry-docs/copy.md +++ b/wrangles-docs/registry-docs/copy.md @@ -3,45 +3,34 @@ title: "Copy" description: "Make a copy of a column or a list of columns." sidebar_label: "Copy" slug: "/copy" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Copy Make a copy of a column or a list of columns. -> Pilot Registry entry. Runtime contract status: `verified`. +Create a copy of columns in a dataframe. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | No | string, integer, array, null | Name of the input columns or columns. | `null` | -| `output` | No | string, array, null | Name of the output columns or columns. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Create a copy of columns in a dataframe. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input columns or columns. | string, integer, array, null | `null` | No | +| `output` | Name of the output columns or columns. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Copying a Column With Input and Output +
-##### Recipe +## Examples ```yaml wrangles: @@ -52,9 +41,7 @@ wrangles:
-
- -##### Input Sample +
| Product Data | | --- | @@ -63,9 +50,7 @@ wrangles:
-
- -##### Output Sample +
| Product Data | Product Data (copy) | | --- | --- | @@ -76,9 +61,9 @@ wrangles:
-#### Copying a Column Express as a Dictionary -##### Recipe + + ```yaml wrangles: @@ -88,9 +73,7 @@ wrangles:
-
- -##### Input Sample +
| Product Data | | --- | @@ -99,9 +82,7 @@ wrangles:
-
- -##### Output Sample +
| Product Data | Product Data (copy) | | --- | --- | @@ -112,17 +93,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `copy` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.copy` | + +**Sources** - [WranglesPY copy implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing copy Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/copy.md) -## Registry metadata - -- Registry ID: `b002fbdc-92c0-4347-889d-0f4bfeec99fa` -- Namespace: root-level runtime key -- Recipe key: `copy` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.copy` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/bins.md b/wrangles-docs/registry-docs/create/bins.md index 480064a..6c1b19d 100644 --- a/wrangles-docs/registry-docs/create/bins.md +++ b/wrangles-docs/registry-docs/create/bins.md @@ -3,47 +3,37 @@ title: "Bins" description: "Create a column that groups data into bins." sidebar_label: "Bins" slug: "/create/bins" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Bins Create a column that groups data into bins. -> Pilot Registry entry. Runtime contract status: `verified`. +Creates a column that segments and sorts data values into bins. `bins` can be an integer or a list. When `bins` is an integer, the input data is split equally into that number of bins. When `bins` is a list, the input data is split based on the list boundaries. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of input column. | — | -| `output` | Yes | string, array | Name of new column. | — | -| `bins` | Yes | integer, array | Defines the number of equal-width bins in the range. | — | -| `labels` | No | string, array, null | Labels for the returned bins. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Creates a column that segments and sorts data values into bins. `bins` can be an integer or a list. When `bins` is an integer, the input data is split equally into that number of bins. When `bins` is a list, the input data is split based on the list boundaries. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of input column. | string, integer, array | — | Yes | +| `output` | Name of new column. | string, array | — | Yes | +| Options | | | | | +| `bins` | Defines the number of equal-width bins in the range. | integer, array | — | Yes | +| `labels` | Labels for the returned bins. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating Bins With an Integer +
-##### Recipe +## Examples ```yaml wrangles: @@ -59,34 +49,37 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Data | +| --- | +| 1 | +| 7 | +| 5 | +| 4 | +| 6 | +| 3 |
-
- -##### Output Sample +
-| Data | Category | -| --- | --- | -| 1 | Bad | -| 7 | Good | -| 5 | Medium | -| 4 | Medium | -| 6 | Good | -| 3 | Bad | +| Category | +| --- | +| Bad | +| Good | +| Medium | +| Medium | +| Good | +| Bad |
-#### Creating Bins With a List -##### Recipe + + ```yaml wrangles: @@ -110,9 +103,7 @@ wrangles:
-
- -##### Input Sample +
| Grade | Student | | --- | --- | @@ -123,32 +114,48 @@ wrangles:
-
- -##### Output Sample +
-| Grade | Student | Letter Grade | -| --- | --- | --- | -| 64 | Charles | D | -| 92 | Sabrina | A | -| 76 | Edward | C | -| 84 | Wendy | B | +| Letter Grade | +| --- | +| D | +| A | +| C | +| B |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.bins` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.bins` | + +**Sources** - [WranglesPY create.bins implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.bins Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/bins.md) -## Registry metadata - -- Registry ID: `232e3ba0-4735-4934-88aa-0163181abb3f` -- Namespace: `create` -- Recipe key: `create.bins` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.bins` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/column.md b/wrangles-docs/registry-docs/create/column.md index ec47fc3..8e5cb45 100644 --- a/wrangles-docs/registry-docs/create/column.md +++ b/wrangles-docs/registry-docs/create/column.md @@ -3,47 +3,37 @@ title: "Column" description: "Create column(s) with a user defined value. Defaults to None (empty)." sidebar_label: "Column" slug: "/create/column" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Column Create column(s) with a user defined value. Defaults to None (empty). -> Pilot Registry entry. Runtime contract status: `verified`. +Create column(s) with a user defined value. Defaults to `None` (empty). If you need to copy an existing column, use the copy wrangle instead. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `output` | Yes | string, array | Name or list of names of new columns or column_name: value pairs. | — | -| `value` | No | string, number, object, array, boolean, null | (Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string. | `null` | -| `value_if_exists` | No | string; one of: existing, coalesce, new | Determines behaviour when the output column already exists. existing (default): leave the column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value. | `"existing"` | -| `coalesce_value` | No | string; one of: existing, new | Only used when value_if_exists is coalesce. Determines which side is preferred when both the existing and new values are non-empty. existing (default): keep the existing value, only fill empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null. | `"existing"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +
-| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Create column(s) with a user defined value. Defaults to `None` (empty). If you need to copy an existing column, use the copy wrangle instead. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns or column_name: value pairs. | string, array | — | Yes | +| Options | | | | | +| `value` | (Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string. | string, number, object, array, boolean, null | `null` | No | +| `value_if_exists` | Determines behaviour when the output column already exists. existing (default): leave the column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value. | string; one of:
  • existing
  • coalesce
  • new
| `"existing"` | No | +| `coalesce_value` | Only used when value_if_exists is coalesce. Determines which side is preferred when both the existing and new values are non-empty. existing (default): keep the existing value, only fill empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null. | string; one of:
  • existing
  • new
| `"existing"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating a New Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| column | | --- | @@ -67,23 +55,21 @@ wrangles:
-
+
-##### Output Sample - -| column | New Column | -| --- | --- | -| 1 | | -| 2 | new value | -| 3 | new value | +| New Column | +| --- | +| | +| new value | +| new value |
-#### Creating Multiple Columns -##### Recipe + + ```yaml wrangles: @@ -96,9 +82,7 @@ wrangles:
-
- -##### Input Sample +
| column | | --- | @@ -108,23 +92,21 @@ wrangles:
-
+
-##### Output Sample - -| column | New Column 1 | New Column 2 | New Column 3 | -| --- | --- | --- | --- | -| 1 | new value 1 | new value 2 | new value 1 | -| 2 | new value 1 | new value 2 | new value 1 | -| 3 | new value 1 | new value 2 | new value 1 | +| New Column 1 | New Column 2 | New Column 3 | +| --- | --- | --- | +| new value 1 | new value 2 | new value 1 | +| new value 1 | new value 2 | new value 1 | +| new value 1 | new value 2 | new value 1 |
-#### Creating Columns That Consist of Lists -##### Recipe + + ```yaml wrangles: @@ -138,9 +120,7 @@ wrangles:
-
- -##### Input Sample +
| column | | --- | @@ -150,15 +130,13 @@ wrangles:
-
- -##### Output Sample +
-| column | New Column | -| --- | --- | -| 1 | [4, 5, 6] | -| 2 | [4, 5, 6] | -| 3 | [4, 5, 6] | +| New Column | +| --- | +| [4, 5, 6] | +| [4, 5, 6] | +| [4, 5, 6] |
@@ -166,17 +144,35 @@ wrangles: Columns of empty lists can also be created by passing an empty list (`[]`) as the column value. -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.column` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.column` | + +**Sources** - [WranglesPY create.column implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.column Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/column.md) -## Registry metadata - -- Registry ID: `5a18e2c8-ec7c-45f5-88fd-bb5c358a8b40` -- Namespace: `create` -- Recipe key: `create.column` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.column` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/embeddings.md b/wrangles-docs/registry-docs/create/embeddings.md index 4a65d70..32a595c 100644 --- a/wrangles-docs/registry-docs/create/embeddings.md +++ b/wrangles-docs/registry-docs/create/embeddings.md @@ -3,55 +3,49 @@ title: "Embeddings" description: "Create an embedding based on text input." sidebar_label: "Embeddings" slug: "/create/embeddings" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Embeddings Create an embedding based on text input. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string | The column of text to create the embeddings for. | — | -| `api_key` | Yes | string | The API key. | — | -| `output` | No | string, null | The output column the embeddings will be saved as. | `null` | -| `batch_size` | No | integer | The number of rows to submit per individual request. | `100` | -| `threads` | No | integer | The number of requests to submit in parallel. Each request contains the number of rows set as batch_size. | `10` | -| `output_type` | No | string; one of: numpy array, python list | Output the embeddings as a numpy array or a python list Default - python list. | `"python list"` | -| `model` | No | string | The specific model to use to generate the embeddings. | `"text-embedding-3-small"` | -| `retries` | No | integer | The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting. | `0` | -| `url` | No | string | The endpoint to send embedding requests to. Defaults to the standard endpoint for the resolved provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format. | `"https://api.openai.com/v1/embeddings"` | -| `precision` | No | string; one of: float16, float32 | The precision of the embeddings. Default is float32. This should be used with output_type numpy array. | `"float32"` | -| `provider` | No | string, null; one of: openai, jina | Controls the request/response format for the embedding API. When omitted, inferred from url (jina.ai → jina, otherwise openai). Setting provider also sets the default url for that provider, so you only need one of provider or url for standard endpoints. Use both together only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy). | `null` | -| `task` | No | string, null; one of: retrieval.query, retrieval.passage, text-matching, classification, separation | The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Create an embedding based on text input. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The column of text to create the embeddings for. | string | — | Yes | +| `output` | The output column the embeddings will be saved as. | string, null | `null` | No | +| Options | | | | | +| `task` | The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter. | string, null; one of:
  • retrieval.query
  • retrieval.passage
  • text-matching
  • classification
  • separation
| `null` | No | +| Formatting | | | | | +| `output_type` | Output the embeddings as a numpy array or a python list Default - python list. | string; one of:
  • numpy array
  • python list
| `"python list"` | No | +| `precision` | The precision of the embeddings. Default is float32. This should be used with output_type numpy array. | string; one of:
  • float16
  • float32
| `"float32"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `batch_size` | The number of rows to submit per individual request. | integer | `100` | No | +| `threads` | The number of requests to submit in parallel. Each request contains the number of rows set as batch_size. | integer | `10` | No | +| Errors | | | | | +| `retries` | The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting. | integer | `0` | No | +| Details | | | | | +| `api_key` | The API key. | string | — | Yes | +| `model` | The specific model to use to generate the embeddings. | string | `"text-embedding-3-small"` | No | +| `url` | The endpoint to send embedding requests to. Defaults to the standard endpoint for the resolved provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format. | string | `"https://api.openai.com/v1/embeddings"` | No | +| `provider` | Controls the request/response format for the embedding API. When omitted, inferred from url (jina.ai → jina, otherwise openai). Setting provider also sets the default url for that provider, so you only need one of provider or url for standard endpoints. Use both together only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy). | string, null; one of:
  • openai
  • jina
| `null` | No | -## Migrated examples -#### Creating Embeddings +
-##### Recipe +## Examples ```yaml wrangles: @@ -63,9 +57,7 @@ wrangles:
-
- -##### Input Sample +
| my_column | | --- | @@ -74,30 +66,46 @@ wrangles:
-
- -##### Output Sample +
-| my_column | embeddings | -| --- | --- | -| angle grinder | [0.010793785, -0.010007165, 0.0028609, -0.0139...] | -| jig saw | [-0.008975127, 0.009314879, -0.024150735, -0.0...] | +| embeddings | +| --- | +| [0.010793785, -0.010007165, 0.0028609, -0.0139...] | +| [-0.008975127, 0.009314879, -0.024150735, -0.0...] |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.embeddings` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.embeddings` | + +**Sources** - [WranglesPY create.embeddings implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.embeddings Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/embeddings.md) -## Registry metadata - -- Registry ID: `e3518afd-a819-40ec-8b49-eb25690220c1` -- Namespace: `create` -- Recipe key: `create.embeddings` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.embeddings` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/guid.md b/wrangles-docs/registry-docs/create/guid.md index f2b6872..0a4b651 100644 --- a/wrangles-docs/registry-docs/create/guid.md +++ b/wrangles-docs/registry-docs/create/guid.md @@ -3,44 +3,33 @@ title: "GUID" description: "Create column(s) with a GUID." sidebar_label: "GUID" slug: "/create/guid" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # GUID Create column(s) with a GUID. -> Pilot Registry entry. Runtime contract status: `verified`. +Create a column with a GUID (Globally Unique Identifier). ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `output` | Yes | string, array | Name or list of names of new columns. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Create a column with a GUID (Globally Unique Identifier). +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating a New GUID Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,17 +44,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| GUID Column | | --- | @@ -76,17 +61,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.guid` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.guid` | + +**Sources** - [WranglesPY create.guid implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.guid Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/guid.md) -## Registry metadata - -- Registry ID: `cf3aaab2-3d5b-4c9b-826e-7af7510521c9` -- Namespace: `create` -- Recipe key: `create.guid` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.guid` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/hash.md b/wrangles-docs/registry-docs/create/hash.md index f241baf..3281a7b 100644 --- a/wrangles-docs/registry-docs/create/hash.md +++ b/wrangles-docs/registry-docs/create/hash.md @@ -3,46 +3,36 @@ title: "Hash" description: "Create a hash of a column." sidebar_label: "Hash" slug: "/create/hash" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Hash Create a hash of a column. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of input column. | — | -| `output` | Yes | string, array | Name of new column. | — | -| `method` | No | string; one of: md5, sha1, sha256, sha512 | The method to use to hash the input (Default: md5). | `"md5"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Create a hash of a column. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of input column. | string, integer, array | — | Yes | +| `output` | Name of new column. | string, array | — | Yes | +| Options | | | | | +| `method` | The method to use to hash the input (Default: md5). | string; one of:
  • md5
  • sha1
  • sha256
  • sha512
| `"md5"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating a New Hash Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Description | | --- | @@ -65,9 +53,7 @@ wrangles:
-
- -##### Output Sample +
| New Column | | --- | @@ -78,17 +64,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.hash` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.hash` | + +**Sources** - [WranglesPY create.hash implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.hash Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/hash.md) -## Registry metadata - -- Registry ID: `11e8fc13-00d2-4779-8d87-6288b07de7e7` -- Namespace: `create` -- Recipe key: `create.hash` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.hash` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/index.md b/wrangles-docs/registry-docs/create/index.md index 7640c5c..e012850 100644 --- a/wrangles-docs/registry-docs/create/index.md +++ b/wrangles-docs/registry-docs/create/index.md @@ -3,47 +3,37 @@ title: "Index" description: "Create column(s) with an incremental index. e.g. 1,2,3..." sidebar_label: "Index" slug: "/create/index" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Index Create column(s) with an incremental index. e.g. 1,2,3... -> Pilot Registry entry. Runtime contract status: `verified`. +Create a new incremental index. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `output` | Yes | string, array | Name or list of names of new columns. | — | -| `start` | No | integer | (Optional; default 1) Starting number for the index. | `1` | -| `step` | No | integer | (Optional; default 1) Step between successive rows. | `1` | -| `by` | No | string, array, null | Optional. Cluster the created indexes by one or more columns. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Create a new incremental index. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| `by` | Optional. Cluster the created indexes by one or more columns. | string, array, null | `null` | No | +| Options | | | | | +| `start` | (Optional; default 1) Starting number for the index. | integer | `1` | No | +| `step` | (Optional; default 1) Step between successive rows. | integer | `1` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating a New Index Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,17 +45,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| New Index | | --- | @@ -76,17 +62,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.index` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.index` | + +**Sources** - [WranglesPY create.index implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.index Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/index.md) -## Registry metadata - -- Registry ID: `0a4909ca-6e14-4da7-8a70-a7fd106d6944` -- Namespace: `create` -- Recipe key: `create.index` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.index` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/jinja.md b/wrangles-docs/registry-docs/create/jinja.md index d666439..0a74f46 100644 --- a/wrangles-docs/registry-docs/create/jinja.md +++ b/wrangles-docs/registry-docs/create/jinja.md @@ -3,50 +3,40 @@ title: "Jinja" description: "Output text using a jinja template." sidebar_label: "Jinja" slug: "/create/jinja" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Jinja Output text using a jinja template. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `template` | Yes | object | A dictionary which defines the template/location as well as the form which the template is input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_). | — | -| `output` | Yes | array | Name of the column to be output to. | — | -| `input` | No | string, null | Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Makes use of a Jinja template to create a description, title, or summary based on your data. :::info Jinja templates do not allow variables with spaces. This wrangle automatically replaces spaces in column headers with underscores, so use underscores instead of spaces when referencing columns in the template. ::: -## Migrated examples -#### Creating a Jinja Description +## Parameters + +
-##### Recipe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name of the column to be output to. | array | — | Yes | +| `input` | Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys. | string, null | `null` | No | +| Options | | | | | +| `template` | A dictionary which defines the template/location as well as the form which the template is input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_). | object | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +## Examples ```yaml wrangles: @@ -59,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| Size | Brand | Item Type | | --- | --- | --- | @@ -70,14 +58,12 @@ wrangles:
-
- -##### Output Sample +
-| Size | Brand | Item Type | Description | -| --- | --- | --- | --- | -| 10mm | SKF | ball bearing | This is a SKF ball bearing that is 10mm | -| 15mm | Timken | bearing seal | This is a Timken bearing seal that is 15mm | +| Description | +| --- | +| This is a SKF ball bearing that is 10mm | +| This is a Timken bearing seal that is 15mm |
@@ -85,17 +71,35 @@ wrangles: Using `|` in YAML denotes a multi-line string that preserves line breaks. Use `>` for a multi-line string when line breaks should not be preserved. -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.jinja` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.jinja` | + +**Sources** - [WranglesPY create.jinja implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.jinja Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/jinja.md) -## Registry metadata - -- Registry ID: `10fc6709-16d4-4eab-8f56-6cb5d170ea66` -- Namespace: `create` -- Recipe key: `create.jinja` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.jinja` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/create/uuid.md b/wrangles-docs/registry-docs/create/uuid.md index 186706e..40de29e 100644 --- a/wrangles-docs/registry-docs/create/uuid.md +++ b/wrangles-docs/registry-docs/create/uuid.md @@ -3,44 +3,33 @@ title: "UUID" description: "Create column(s) with a UUID." sidebar_label: "UUID" slug: "/create/uuid" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # UUID Create column(s) with a UUID. -> Pilot Registry entry. Runtime contract status: `verified`. +Create a column with a UUID (Universally Unique Identifier). ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `output` | Yes | string, array | Name or list of names of new columns. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Create a column with a UUID (Universally Unique Identifier). +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Creating a New GUID Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,17 +44,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| GUID Column | | --- | @@ -76,17 +61,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.uuid` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.uuid` | + +**Sources** - [WranglesPY create.uuid implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) - [Existing create.uuid Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/uuid.md) -## Registry metadata - -- Registry ID: `c376f3ff-2283-4c4c-8d7f-70db6f53ed19` -- Namespace: `create` -- Recipe key: `create.uuid` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.create.uuid` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/date-calculator.md b/wrangles-docs/registry-docs/date-calculator.md index 5e85c56..026278e 100644 --- a/wrangles-docs/registry-docs/date-calculator.md +++ b/wrangles-docs/registry-docs/date-calculator.md @@ -3,48 +3,38 @@ title: "Date Calculator" description: "Add or Subtract time from a date." sidebar_label: "Date Calculator" slug: "/date-calculator" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Date Calculator Add or Subtract time from a date. -> Pilot Registry entry. Runtime contract status: `verified`. +Add or subtract time from a date. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string | Name of the dates column. | — | -| `operation` | No | string; one of: add, subtract | Date operation. | `"add"` | -| `output` | No | string, null | Name of the output column of dates. | `null` | -| `time_unit` | No | string, null; one of: years, months, weeks, days, hours, minutes, seconds, milliseconds | Time unit for operation. | `null` | -| `time_value` | No | number, null | Time unit value for operation. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Add or subtract time from a date. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the dates column. | string | — | Yes | +| `output` | Name of the output column of dates. | string, null | `null` | No | +| Options | | | | | +| `operation` | Date operation. | string; one of:
  • add
  • subtract
| `"add"` | No | +| `time_unit` | Time unit for operation. | string, null; one of:
  • years
  • months
  • weeks
  • days
  • hours
  • minutes
  • seconds
  • milliseconds
| `null` | No | +| `time_value` | Time unit value for operation. | number, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Calculating a Future Date +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,9 +48,7 @@ wrangles:
-
- -##### Input Sample +
| Date | | --- | @@ -68,9 +56,7 @@ wrangles:
-
- -##### Output Sample +
| New Date | | --- | @@ -80,17 +66,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `date_calculator` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `date` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.date_calculator` | + +**Sources** - [WranglesPY date_calculator implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing date_calculator Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/date/_sources/date-calculator.md) -## Registry metadata - -- Registry ID: `19cfeb4f-02af-4ab2-895c-0ff2bb5cce19` -- Namespace: root-level runtime key -- Recipe key: `date_calculator` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.date_calculator` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/drop.md b/wrangles-docs/registry-docs/drop.md index 3d9ea65..a4dcef7 100644 --- a/wrangles-docs/registry-docs/drop.md +++ b/wrangles-docs/registry-docs/drop.md @@ -3,42 +3,31 @@ title: "Drop" description: "Drop (Delete) selected column(s)." sidebar_label: "Drop" slug: "/drop" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Drop Drop (Delete) selected column(s). -> Pilot Registry entry. Runtime contract status: `verified`. +Drop (Delete) selected column(s) ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `columns` | Yes | string, array | Name of the column(s) to drop. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Drop (Delete) selected column(s) +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | Name of the column(s) to drop. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | -## Migrated examples -#### Dropping a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -49,9 +38,7 @@ wrangles:
-
- -##### Input Sample +
| Material | Product Data | | --- | --- | @@ -60,9 +47,7 @@ wrangles:
-
- -##### Output Sample +
| Product Data | | --- | @@ -73,17 +58,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `drop` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.drop` | + +**Sources** - [WranglesPY drop implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing drop Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/drop.md) -## Registry metadata - -- Registry ID: `363bffbf-397e-4975-8382-e9efa5e9eed6` -- Namespace: root-level runtime key -- Recipe key: `drop` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.drop` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/explode.md b/wrangles-docs/registry-docs/explode.md index 344865d..08a90ef 100644 --- a/wrangles-docs/registry-docs/explode.md +++ b/wrangles-docs/registry-docs/explode.md @@ -3,46 +3,37 @@ title: "Explode" description: "Explode a column of lists into rows." sidebar_label: "Explode" slug: "/explode" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Explode Explode a column of lists into rows. -> Pilot Registry entry. Runtime contract status: `verified`. +Explode a column of lists into rows ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length. | — | -| `reset_index` | No | boolean | Reset the index after exploding. Default True. | `true` | -| `drop_empty` | No | boolean | If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Explode a column of lists into rows +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length. | string, integer, array | — | Yes | +| Options | | | | | +| `drop_empty` | If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False. | boolean | `false` | No | +| Formatting | | | | | +| `reset_index` | Reset the index after exploding. Default True. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Exploding a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -52,9 +43,7 @@ wrangles:
-
- -##### Input Sample +
| Products | Manufacturer | | --- | --- | @@ -64,9 +53,7 @@ wrangles:
-
- -##### Output Sample +
| Products | Manufacturer | | --- | --- | @@ -81,17 +68,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `explode` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.explode` | + +**Sources** - [WranglesPY explode implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing explode Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/explode.md) -## Registry metadata - -- Registry ID: `4e4b13ac-8d50-4b2c-85c8-2c31de1e817d` -- Namespace: root-level runtime key -- Recipe key: `explode` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.explode` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/address.md b/wrangles-docs/registry-docs/extract/address.md index a90e2eb..b3def87 100644 --- a/wrangles-docs/registry-docs/extract/address.md +++ b/wrangles-docs/registry-docs/extract/address.md @@ -3,48 +3,39 @@ title: "Address" description: "Extract parts of addresses. Requires WrangleWorks Account." sidebar_label: "Address" slug: "/extract/address" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Address Extract parts of addresses. Requires WrangleWorks Account. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract geographical information from unstructured text such as streets, cities, or countries. Requires WrangleWorks Account. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | Yes | string, array | Name of the output column. | — | -| `dataType` | Yes | string; one of: streets, cities, regions, countries | Specific part of the address to extract. | — | -| `output_format` | No | string, null; one of: list, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Extract geographical information from unstructured text such as streets, cities, or countries. Requires WrangleWorks Account. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `dataType` | Specific part of the address to extract. | string; one of:
  • streets
  • cities
  • regions
  • countries
| — | Yes | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Street Name +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,38 +48,55 @@ wrangles:
-
+
-##### Input Sample +| Location | +| --- | +| 221 B Baker St., London, England, United Kingdom | +| London SW1A 1AA, London, England, United Kingdom | -_No sample available._ +
+ +
+ +| Street | +| --- | +| ['221 B Baker St.'] | +| |
-
+
+ +
-##### Output Sample +Access -| Street | Location | +| Requirement | Value | | --- | --- | -| ['221 B Baker St.'] | 221 B Baker St., London, England, United Kingdom | -| | London SW1A 1AA, London, England, United Kingdom | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | -
+ -
+
+ +Technical details -## Provenance +| Field | Value | +| --- | --- | +| Recipe key | `extract.address` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.address` | + +**Sources** - [WranglesPY extract.address implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.address Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/address.md) -## Registry metadata - -- Registry ID: `44153f95-4581-4cee-898e-b3b4714045fd` -- Namespace: `extract` -- Recipe key: `extract.address` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.address` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/ai.md b/wrangles-docs/registry-docs/extract/ai.md index 8f625e5..fff609b 100644 --- a/wrangles-docs/registry-docs/extract/ai.md +++ b/wrangles-docs/registry-docs/extract/ai.md @@ -3,70 +3,64 @@ title: "AI" description: "Extract structured data from each input row using an AI model. Define the desired fields with output, or reuse a saved definition with model_id." sidebar_label: "AI" slug: "/extract/ai" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # AI Extract structured data from each input row using an AI model. Define the desired fields with output, or reuse a saved definition with model_id. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `api_key` | Yes | string | OpenAI API key used for this wrangle, normally supplied through a recipe variable. | — | -| `input` | No | array, null | Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied. | `null` | -| `output` | No | string, array, object, null | Desired extraction. Use an object keyed by output column name for structured fields, a string for one prompted value, or an array of field names/definitions. Each field may use the schema options below. | `null` | -| `model_id` | No | string, null | ID of a saved extract.ai definition. Use it instead of defining an output schema. When output is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields. | `null` | -| `record_examples` | No | array, object, null | Whole-record examples. Each example has a separate input value or record and the complete expected output record. Optional name and notes provide model-visible context. Use {name: ..., notes: ..., input: ..., output: ...}. Omitted nullable output fields are completed with null. Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field. | `null` | -| `output_format` | No | string, null; one of: dictionary, columns, concatenate | How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char. | `null` | -| `char` | No | string | Separator used only when output_format is concatenate. Defaults to comma-space. | `", "` | -| `web_search` | No | boolean | Enable OpenAI Responses web search; the model decides when searching helps. When true, every row also receives web_search_sources: a deduplicated list of {title, url} objects in source order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false. | `false` | -| `instructions` | No | string, array, null | Additional guidance applied to every input row. Use this for decision rules, evidence priorities, normalization requirements, or other behavior that applies to the complete extraction. | `null` | -| `model` | No | string | OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model. | — | -| `threads` | No | integer | Maximum number of row-level requests sent in parallel. The configured default is 32. | — | -| `timeout` | No | number | Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner. | — | -| `retries` | No | integer | Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline. | — | -| `url` | No | string | Override the endpoint for the selected protocol. A chat/completions URL selects the legacy protocol only when protocol is omitted; new recipes should use the configured Responses endpoint. | — | -| `provider` | No | string; one of: openai | AI service provider. Currently only OpenAI is supported. | — | -| `protocol` | No | string; one of: responses, chat_completions | OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions. | — | -| `deadline` | No | number | Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15. | — | -| `store` | No | boolean | Whether OpenAI may store Responses API results. Defaults to false. | — | -| `cache` | No | boolean | Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required. | — | -| `cache_ttl` | No | number | Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together. | — | -| `strict` | No | boolean | Require OpenAI structured-output strict mode. Defaults to true. Definitions with dynamic dictionary keys automatically switch to non-strict provider mode and are still validated locally. | — | -| `reasoning` | No | object | Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies. | — | -| `verbosity` | No | string; one of: low, medium, high | Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Use AI to extract meaningful structured data. `extract.ai` can be used recipe-first, where the output schema is defined in the recipe, or model-first, where a saved extract.ai model is called by `model_id`. :::info For saved extract.ai models, this is the preferred calling pattern compared with using `extract.custom`. ::: -## Migrated examples -#### Making Use of Output Parameters +## Parameters + +
-##### Recipe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied. | array, null | `null` | No | +| `output` | Desired extraction. Use an object keyed by output column name for structured fields, a string for one prompted value, or an array of field names/definitions. Each field may use the schema options below. | string, array, object, null | `null` | No | +| Options | | | | | +| `record_examples` | Whole-record examples. Each example has a separate input value or record and the complete expected output record. Optional name and notes provide model-visible context. Use {name: ..., notes: ..., input: ..., output: ...}. Omitted nullable output fields are completed with null. Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field. | array, object, null | `null` | No | +| `web_search` | Enable OpenAI Responses web search; the model decides when searching helps. When true, every row also receives web_search_sources: a deduplicated list of {title, url} objects in source order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false. | boolean | `false` | No | +| `instructions` | Additional guidance applied to every input row. Use this for decision rules, evidence priorities, normalization requirements, or other behavior that applies to the complete extraction. | string, array, null | `null` | No | +| Formatting | | | | | +| `output_format` | How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char. | string, null; one of:
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Separator used only when output_format is concatenate. Defaults to comma-space. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Maximum number of row-level requests sent in parallel. The configured default is 32. | integer | — | No | +| `timeout` | Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner. | number | — | No | +| `deadline` | Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15. | number | — | No | +| Errors | | | | | +| `retries` | Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline. | integer | — | No | +| Details | | | | | +| `api_key` | OpenAI API key used for this wrangle, normally supplied through a recipe variable. | string | — | Yes | +| `model_id` | ID of a saved extract.ai definition. Use it instead of defining an output schema. When output is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields. | string, null | `null` | No | +| `model` | OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model. | string | — | No | +| `url` | Override the endpoint for the selected protocol. A chat/completions URL selects the legacy protocol only when protocol is omitted; new recipes should use the configured Responses endpoint. | string | — | No | +| `provider` | AI service provider. Currently only OpenAI is supported. | string; one of:
  • openai
| — | No | +| `protocol` | OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions. | string; one of:
  • responses
  • chat_completions
| — | No | +| `store` | Whether OpenAI may store Responses API results. Defaults to false. | boolean | — | No | +| `cache` | Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required. | boolean | — | No | +| `cache_ttl` | Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together. | number | — | No | +| `strict` | Require OpenAI structured-output strict mode. Defaults to true. Definitions with dynamic dictionary keys automatically switch to non-strict provider mode and are still validated locally. | boolean | — | No | +| `reasoning` | Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies. | object | — | No | +| `verbosity` | Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models. | string; one of:
  • low
  • medium
  • high
| — | No | + +
+ +## Examples ```yaml wrangles: @@ -91,9 +85,7 @@ wrangles:
-
- -##### Input Sample +
| Product Specs | | --- | @@ -102,9 +94,7 @@ wrangles:
-
- -##### Output Sample +
| Blade Diameter | Max. RPM | | --- | --- | @@ -115,9 +105,9 @@ wrangles:
-#### Description Only -##### Recipe + + ```yaml wrangles: @@ -131,9 +121,7 @@ wrangles:
-
- -##### Input Sample +
| Product Specs | | --- | @@ -142,9 +130,7 @@ wrangles:
-
- -##### Output Sample +
| Blade Diameter | Max. RPM | | --- | --- | @@ -155,9 +141,9 @@ wrangles:
-#### Model Based Column Output by Name -##### Recipe + + ```yaml wrangles: @@ -171,9 +157,7 @@ wrangles:
-
- -##### Input Sample +
| Items | | --- | @@ -182,9 +166,7 @@ wrangles:
-
- -##### Output Sample +
| Colors | Sizes | | --- | --- | @@ -195,17 +177,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.ai` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.ai` | + +**Sources** - [WranglesPY extract.ai implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.ai Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/ai.md) -## Registry metadata - -- Registry ID: `d9f89b00-fda3-4f4c-826c-6417b9390607` -- Namespace: `extract` -- Recipe key: `extract.ai` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.ai` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/attributes.md b/wrangles-docs/registry-docs/extract/attributes.md index 00bdf97..6a1f944 100644 --- a/wrangles-docs/registry-docs/extract/attributes.md +++ b/wrangles-docs/registry-docs/extract/attributes.md @@ -3,52 +3,43 @@ title: "Attributes" description: "Extract numeric attributes from the input such as weights or lengths. Requires WrangleWorks Account." sidebar_label: "Attributes" slug: "/extract/attributes" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Attributes Extract numeric attributes from the input such as weights or lengths. Requires WrangleWorks Account. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract numeric attributes from unstructured text such as lengths, voltages, weights, or temperatures. Requires WrangleWorks Account. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | Yes | string, array | Name of the output column. | — | -| `responseContent` | No | string; one of: span, object | Span - returns the text found. object - returns an object with the value and unit. | `"span"` | -| `attribute_type` | No | string, null; one of: angle, area, capacitance, charge, current, data transfer rate, electrical conductance, electrical resistance, energy, force, frequency, inductance, instance frequency, length, luminous flux, weight, power, pressure, speed, velocity, temperature, time, voltage, volume, volumetric flow | Request only a specific type of attribute. | `null` | -| `desired_unit` | No | string, null | Convert the extracted unit to the desired unit. | `null` | -| `bound` | No | string; one of: min, mid, max | When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid. | `"mid"` | -| `first_element` | No | boolean | Get the first element from results. | `false` | -| `output_format` | No | string, null; one of: list, dictionary, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +
-| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Extract numeric attributes from unstructured text such as lengths, voltages, weights, or temperatures. Requires WrangleWorks Account. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `responseContent` | Span - returns the text found. object - returns an object with the value and unit. | string; one of:
  • span
  • object
| `"span"` | No | +| `attribute_type` | Request only a specific type of attribute. | string, null; one of:
  • angle
  • area
  • capacitance
  • charge
  • current
  • data transfer rate
  • electrical conductance
  • electrical resistance
  • energy
  • force
  • frequency
  • inductance
  • instance frequency
  • length
  • luminous flux
  • weight
  • power
  • pressure
  • speed
  • velocity
  • temperature
  • time
  • voltage
  • volume
  • volumetric flow
| `null` | No | +| `desired_unit` | Convert the extracted unit to the desired unit. | string, null | `null` | No | +| `bound` | When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid. | string; one of:
  • min
  • mid
  • max
| `"mid"` | No | +| Formatting | | | | | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting All Attributes +
-##### Recipe +## Examples ```yaml wrangles: @@ -60,29 +51,27 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Tools (input) | +| --- | +| hammer 5kg, 0.5m |
-
+
-##### Output Sample - -| Tools (input) | Attributes (span )(output) | -| --- | --- | -| hammer 5kg, 0.5m | \{'length': ['0.5m'], 'mass': ['5kg']\} | +| Attributes (span )(output) | +| --- | +| \{'length': ['0.5m'], 'mass': ['5kg']\} |
-#### Extracting All Attributes -##### Recipe + + ```yaml wrangles: @@ -94,29 +83,27 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Tools (input) | +| --- | +| hammer 5kg, 0.5m |
-
- -##### Output Sample +
-| Tools (input) | Attributes (Object) (output) | -| --- | --- | -| hammer 5kg, 0.5m | \{'length': [\{'unit': 'metre', 'value': 0.5\}], 'mass': [\{'unit': 'kilogram', 'value': 5.0\}]\} | +| Attributes (Object) (output) | +| --- | +| \{'length': [\{'unit': 'metre', 'value': 0.5\}], 'mass': [\{'unit': 'kilogram', 'value': 5.0\}]\} |
-#### Extracting Specific Attributes -##### Recipe + + ```yaml wrangles: @@ -129,17 +116,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
_No sample available._ @@ -147,17 +130,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.attributes` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.attributes` | + +**Sources** - [WranglesPY extract.attributes implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.attributes Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/attributes.md) -## Registry metadata - -- Registry ID: `03ccedef-c938-41f1-8980-280f1a91542e` -- Namespace: `extract` -- Recipe key: `extract.attributes` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.attributes` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/brackets.md b/wrangles-docs/registry-docs/extract/brackets.md index f30b554..791da62 100644 --- a/wrangles-docs/registry-docs/extract/brackets.md +++ b/wrangles-docs/registry-docs/extract/brackets.md @@ -3,49 +3,40 @@ title: "Brackets" description: "Extract text properties in brackets from the input." sidebar_label: "Brackets" slug: "/extract/brackets" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Brackets Extract text properties in brackets from the input. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract text in brackets from the input. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | Yes | string, array | Name of the output columns. | — | -| `find` | No | string, array | (Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets. | `"all"` | -| `include_brackets` | No | boolean | (Optional) Include the brackets in the output. | `false` | -| `output_format` | No | string, null; one of: list, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Extract text in brackets from the input. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, array | — | Yes | +| Options | | | | | +| `find` | (Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets. | string, array | `"all"` | No | +| `include_brackets` | (Optional) Include the brackets in the output. | boolean | `false` | No | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Data Within Brackets +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,40 +47,59 @@ wrangles:
-
+
-##### Input Sample +| Data | +| --- | +| `{Hello}` | +| `[Wrangles]` | +| `(!)` | +| `` | -_No sample available._ +
+ +
+ +| Output | +| --- | +| Hello | +| Wrangles | +| ! | +| ! |
-
+
+ +
-##### Output Sample +Access -| Data | Output | +| Requirement | Value | | --- | --- | -| `{Hello}` | Hello | -| `[Wrangles]` | Wrangles | -| `(!)` | ! | -| `` | ! | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -
+ -
+
+ +Technical details -## Provenance +| Field | Value | +| --- | --- | +| Recipe key | `extract.brackets` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.brackets` | + +**Sources** - [WranglesPY extract.brackets implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.brackets Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/brackets.md) -## Registry metadata - -- Registry ID: `9b4c15fa-2aaa-40c8-8834-6e835760bee5` -- Namespace: `extract` -- Recipe key: `extract.brackets` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.brackets` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/codes.md b/wrangles-docs/registry-docs/extract/codes.md index f60d3fb..c42d2e3 100644 --- a/wrangles-docs/registry-docs/extract/codes.md +++ b/wrangles-docs/registry-docs/extract/codes.md @@ -3,55 +3,46 @@ title: "Codes" description: "Extract alphanumeric codes from the input. Requires WrangleWorks Account." sidebar_label: "Codes" slug: "/extract/codes" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Codes Extract alphanumeric codes from the input. Requires WrangleWorks Account. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract alphanumeric codes from unstructured text. Can be performed on one column or multiple columns. Requires WrangleWorks Account. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `output` | Yes | string, array | Name or list of output columns. | — | -| `first_element` | No | boolean | Get the first element from results. | `false` | -| `output_format` | No | string, null; one of: list, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `min_length` | No | integer, string | Minimum length of allowed results. | — | -| `max_length` | No | integer, string | Maximum length of allowed results. | — | -| `strategy` | No | string; one of: lenient, balanced, strict | Controls filtering of likely false positives such as measurements. Lenient skips this filter; balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict. | — | -| `sort_order` | No | string; one of: input, longest, shortest | Default is input order. Also allows longest or shortest. | — | -| `disallowed_patterns` | No | string | A pattern or JSON array of regex patterns to not include in the found codes. | — | -| `include_multi_part_tokens` | No | boolean | Whether to include multi-part tokens that have a space. Default True. | — | -| `extract_raw` | No | boolean | Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +
-| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Extract alphanumeric codes from unstructured text. Can be performed on one column or multiple columns. Requires WrangleWorks Account. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array | — | Yes | +| Options | | | | | +| `min_length` | Minimum length of allowed results. | integer, string | — | No | +| `max_length` | Maximum length of allowed results. | integer, string | — | No | +| `strategy` | Controls filtering of likely false positives such as measurements. Lenient skips this filter; balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict. | string; one of:
  • lenient
  • balanced
  • strict
| — | No | +| `disallowed_patterns` | A pattern or JSON array of regex patterns to not include in the found codes. | string | — | No | +| `include_multi_part_tokens` | Whether to include multi-part tokens that have a space. Default True. | boolean | — | No | +| `extract_raw` | Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False. | boolean | — | No | +| Formatting | | | | | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| `sort_order` | Default is input order. Also allows longest or shortest. | string; one of:
  • input
  • longest
  • shortest
| — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Codes From Single Column +
-##### Recipe +## Examples ```yaml # One column input @@ -63,29 +54,27 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Secret | +| --- | +| to gain access use Z1ON0101 |
-
+
-##### Output Sample - -| Secret | Code Extract | -| --- | --- | -| to gain access use Z1ON0101 | ['Z1ON0101'] | +| Code Extract | +| --- | +| ['Z1ON0101'] |
-#### Extracting Codes From Multiple Columns -##### Recipe + + ```yaml # Multi column input @@ -99,37 +88,53 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| code1 | code2 | +| --- | --- | +| code CH465517080-1 | code CH465517080-2 |
-
+
-##### Output Sample - -| Codes | code1 | code2 | -| --- | --- | --- | -| ['CH465517080-1', 'CH465517080-2'] | code CH465517080-1 | code CH465517080-2 | +| Codes | +| --- | +| ['CH465517080-1', 'CH465517080-2'] |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.codes` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.codes` | + +**Sources** - [WranglesPY extract.codes implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.codes Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/codes.md) -## Registry metadata - -- Registry ID: `da591387-0fca-4842-8bcb-d19f561f0292` -- Namespace: `extract` -- Recipe key: `extract.codes` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.codes` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/custom.md b/wrangles-docs/registry-docs/extract/custom.md index 1c62782..2c24ba8 100644 --- a/wrangles-docs/registry-docs/extract/custom.md +++ b/wrangles-docs/registry-docs/extract/custom.md @@ -3,59 +3,51 @@ title: "Custom" description: "Extract data from the input using a DIY or bespoke extraction wrangle. Requires WrangleWorks Account and Subscription." sidebar_label: "Custom" slug: "/extract/custom" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Custom Extract data from the input using a DIY or bespoke extraction wrangle. Requires WrangleWorks Account and Subscription. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `model_id` | Yes | string, array | The ID of the wrangle to use. | — | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `use_labels` | No | boolean | Use Labels in the extract output {label: value}. | `false` | -| `first_element` | No | boolean | Get the first element from results. | `false` | -| `case_sensitive` | No | boolean | Allows the wrangle to be case sensitive if set to True, default is False. | `false` | -| `extract_raw` | No | boolean | Extract the raw data from the wrangle. | `false` | -| `use_spellcheck` | No | boolean | Use spellcheck to also find minor mispellings compared to the reference data. | `false` | -| `include_empty_labels` | No | boolean | Include labels with no found values in the output when using use_labels=True. | `true` | -| `sort` | No | string; one of: training_order, input_order, longest, shortest, alphabetical, reverse_alphabetical, ascending, descending | Sort the results. | `"training_order"` | -| `output_format` | No | string, null; one of: list, dictionary, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Extract data from the input using a DIY or bespoke extraction wrangle. Can be performed on one column or multiple columns. Requires WrangleWorks Account and Subscription. :::info Non-regex pattern matching extracts whole-word matches separated by word boundaries. Word boundaries include anything that is not a letter, number, or underscore. ::: -## Migrated examples -#### Extracting Wood Types From Single Column +## Parameters -##### Recipe +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| `extract_raw` | Extract the raw data from the wrangle. | boolean | `false` | No | +| `use_spellcheck` | Use spellcheck to also find minor mispellings compared to the reference data. | boolean | `false` | No | +| `sort` | Sort the results. | string; one of:
  • training_order
  • input_order
  • longest
  • shortest
  • alphabetical
  • reverse_alphabetical
  • ascending
  • descending
| `"training_order"` | No | +| Formatting | | | | | +| `use_labels` | Use Labels in the extract output {label: value}. | boolean | `false` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `include_empty_labels` | Include labels with no found values in the output when using use_labels=True. | boolean | `true` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use. | string, array | — | Yes | + +
+ +## Examples ```yaml # One column input @@ -68,30 +60,29 @@ wrangles:
-
- -##### Input Sample +
-_No sample available._ +| Product | +| --- | +| Dining Oakwood Chair | +| Living Room Teakwood Frame Mirror |
-
- -##### Output Sample +
-| Product | Wood Types | -| --- | --- | -| Dining Oakwood Chair | Oakwood | -| Living Room Teakwood Frame Mirror | Teakwood | +| Wood Types | +| --- | +| Oakwood | +| Teakwood |
-#### Extracting Wood Types From Multiple Columns -##### Recipe + + ```yaml # Multi column input @@ -106,29 +97,27 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Part 1 of 2 | Part 2 of 2 | +| --- | --- | +| Dining Acacia Wood Table | Imitation Wood Table Chairs |
-
+
-##### Output Sample - -| Wood Types | Part 1 of 2 | Part 2 of 2 | -| --- | --- | --- | -| ['Acacia Wood', 'Imitation Wood'] | Dining Acacia Wood Table | Imitation Wood Table Chairs | +| Wood Types | +| --- | +| ['Acacia Wood', 'Imitation Wood'] |
-#### Using Multiple Extract Models -##### Recipe + + ```yaml # Multiple Models @@ -147,17 +136,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| Product | Item Type | Wood Types | | --- | --- | --- | @@ -168,17 +153,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.custom` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.custom` | + +**Sources** - [WranglesPY extract.custom implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.custom Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/custom.md) -## Registry metadata - -- Registry ID: `e8e96b76-86bf-41dc-8d16-825dcff9688b` -- Namespace: `extract` -- Recipe key: `extract.custom` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.custom` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/date-properties.md b/wrangles-docs/registry-docs/extract/date-properties.md index ad87f6a..7c329a3 100644 --- a/wrangles-docs/registry-docs/extract/date-properties.md +++ b/wrangles-docs/registry-docs/extract/date-properties.md @@ -3,46 +3,36 @@ title: "Date Properties" description: "Extract date properties from a date (day, month, year, etc...)." sidebar_label: "Date Properties" slug: "/extract/date-properties" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Date Properties Extract date properties from a date (day, month, year, etc...). -> Pilot Registry entry. Runtime contract status: `verified`. +Extract date properties from a date, such as day, month, year, weekday, or quarter. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `property` | Yes | string; one of: day, day_of_year, month, month_name, weekday, week_day_name, week_year, quarter | Property to extract from date. | — | -| `output` | No | string, null | Name of the output columns. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Extract date properties from a date, such as day, month, year, weekday, or quarter. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, null | `null` | No | +| Options | | | | | +| `property` | Property to extract from date. | string; one of:
  • day
  • day_of_year
  • month
  • month_name
  • weekday
  • week_day_name
  • week_year
  • quarter
| — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Month From Date +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,37 +44,53 @@ wrangles:
-
+
-##### Input Sample +| Date | +| --- | +| 1992-08-13 00:00:00 | -_No sample available._ +
+ +
+ +| Output | +| --- | +| August |
-
+
+ +
-##### Output Sample +Access -| Date | Output | +| Requirement | Value | | --- | --- | -| 1992-08-13 00:00:00 | August | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -
+ -
+
+ +Technical details -## Provenance +| Field | Value | +| --- | --- | +| Recipe key | `extract.date_properties` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.date_properties` | + +**Sources** - [WranglesPY extract.date_properties implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.date_properties Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/date-properties.md) -## Registry metadata - -- Registry ID: `a346de62-93cd-44ba-8d30-a6305629c6d7` -- Namespace: `extract` -- Recipe key: `extract.date_properties` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.date_properties` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/date-range.md b/wrangles-docs/registry-docs/extract/date-range.md index 6185d27..ceb97cd 100644 --- a/wrangles-docs/registry-docs/extract/date-range.md +++ b/wrangles-docs/registry-docs/extract/date-range.md @@ -3,47 +3,37 @@ title: "Date Range" description: "Extract date range frequency from two dates." sidebar_label: "Date Range" slug: "/extract/date-range" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Date Range Extract date range frequency from two dates. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `start_time` | Yes | string | Name of the start date column. | — | -| `end_time` | Yes | string | Name of the end date column. | — | -| `output` | Yes | string | Name of the output column. | — | -| `range` | No | string; one of: business days, days, weeks, months, semi months, business month ends, month starts, semi month starts, business month starts, quarters, quarter starts, years, business hours, hours, minutes, seconds, milliseconds | Type of frequency to count. | `"day"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Extract date range frequency from two dates. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `start_time` | Name of the start date column. | string | — | Yes | +| `end_time` | Name of the end date column. | string | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `range` | Type of frequency to count. | string; one of:
  • business days
  • days
  • weeks
  • months
  • semi months
  • business month ends
  • month starts
  • semi month starts
  • business month starts
  • quarters
  • quarter starts
  • years
  • business hours
  • hours
  • minutes
  • seconds
  • milliseconds
| `"day"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Number of Months From Range +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,17 +46,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| End | Start | Output | | --- | --- | --- | @@ -76,17 +62,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.date_range` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.date_range` | + +**Sources** - [WranglesPY extract.date_range implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.date_range Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/date-range.md) -## Registry metadata - -- Registry ID: `9cebfa6f-a524-4aec-84a7-02d77b792843` -- Namespace: `extract` -- Recipe key: `extract.date_range` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.date_range` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/html.md b/wrangles-docs/registry-docs/extract/html.md index 4459cea..23a4966 100644 --- a/wrangles-docs/registry-docs/extract/html.md +++ b/wrangles-docs/registry-docs/extract/html.md @@ -3,48 +3,39 @@ title: "HTML" description: "Extract elements from strings containing html. Requires WrangleWorks Account." sidebar_label: "HTML" slug: "/extract/html" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # HTML Extract elements from strings containing html. Requires WrangleWorks Account. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract text and links from HTML elements. Requires WrangleWorks Account. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `data_type` | Yes | string; one of: text, links | The type of data to extract. | — | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `output_format` | No | string, null; one of: list, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Extract text and links from HTML elements. Requires WrangleWorks Account. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `data_type` | The type of data to extract. | string; one of:
  • text
  • links
| — | Yes | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Text From HTML +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,29 +47,27 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| HTML | +| --- | +| ` |
-
- -##### Output Sample +
-| HTML | Text | -| --- | --- | -| ` | | +| Text | +| --- | +| |
-#### Extracting Links From HTML -##### Recipe + + ```yaml wrangles: @@ -90,37 +79,53 @@ wrangles:
-
+
-##### Input Sample +| HTML | +| --- | +| ` | + +
-_No sample available._ +
+ +| Links | +| --- | +| | + +
-
+
-##### Output Sample +Access -| HTML | Links | +| Requirement | Value | | --- | --- | -| ` | | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | -
+ -
+
-## Provenance +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.html` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.html` | + +**Sources** - [WranglesPY extract.html implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.html Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/html.md) -## Registry metadata - -- Registry ID: `728fc87a-a20d-4efa-833a-612e0b5eadc3` -- Namespace: `extract` -- Recipe key: `extract.html` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.html` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/properties.md b/wrangles-docs/registry-docs/extract/properties.md index 46bfa27..f7f2b72 100644 --- a/wrangles-docs/registry-docs/extract/properties.md +++ b/wrangles-docs/registry-docs/extract/properties.md @@ -3,50 +3,41 @@ title: "Properties" description: "Extract text properties from the input. Requires WrangleWorks Account." sidebar_label: "Properties" slug: "/extract/properties" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Properties Extract text properties from the input. Requires WrangleWorks Account. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract categorical properties from unstructured text, such as colours or materials. Requires WrangleWorks Account. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | Yes | string, array | Name of the output columns. | — | -| `property_type` | No | string, null; one of: Colours, Materials, Shapes, Standards | The specific type of properties to extract. | `null` | -| `return_data_type` | No | string; one of: list, string | Legacy format option. Prefer output_format. | `"list"` | -| `first_element` | No | boolean | Get the first element from results. | `false` | -| `output_format` | No | string, null; one of: list, dictionary, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Extract categorical properties from unstructured text, such as colours or materials. Requires WrangleWorks Account. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, array | — | Yes | +| Options | | | | | +| `property_type` | The specific type of properties to extract. | string, null; one of:
  • Colours
  • Materials
  • Shapes
  • Standards
| `null` | No | +| Formatting | | | | | +| `return_data_type` | Legacy format option. Prefer output_format. | string; one of:
  • list
  • string
| `"list"` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting All Properties +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,17 +48,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| Products | Product Properties | | --- | --- | @@ -78,9 +65,9 @@ _No sample available._
-#### Extracting Colour -##### Recipe + + ```yaml wrangles: @@ -92,17 +79,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
| Products | Product Colours | | --- | --- | @@ -113,17 +96,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.properties` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.properties` | + +**Sources** - [WranglesPY extract.properties implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.properties Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/properties.md) -## Registry metadata - -- Registry ID: `23bbdb86-fd13-4b78-8fff-a7a76769ab63` -- Namespace: `extract` -- Recipe key: `extract.properties` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.properties` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/extract/regex.md b/wrangles-docs/registry-docs/extract/regex.md index fb9d977..0527db2 100644 --- a/wrangles-docs/registry-docs/extract/regex.md +++ b/wrangles-docs/registry-docs/extract/regex.md @@ -3,50 +3,41 @@ title: "Regex" description: "Extract matches or specific capture groups using regex." sidebar_label: "Regex" slug: "/extract/regex" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Regex Extract matches or specific capture groups using regex. -> Pilot Registry entry. Runtime contract status: `verified`. +Extract single values, matches, or specific capture groups using regex. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column(s). | — | -| `find` | Yes | string | Pattern to find using regex. | — | -| `output` | Yes | string, array | Name of the output column(s). | — | -| `output_pattern` | No | string, null | Specifies the format to output matches and specific capture groups using backreferences (e.g., `\1`, `\2`). Default is to return entire matches. **Example**: For a regex pattern `r'(\d+)\s(\w+)'` and `output_pattern = '\2 \1'`, with input `'120 volt'`, the output would be `'volt 120'`. | `null` | -| `first_element` | No | boolean | Get the first element from results. | `false` | -| `output_format` | No | string, null; one of: list, columns, concatenate | Format of the extract output. | `null` | -| `char` | No | string | Character to use when output_format is concatenate. | `", "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Extract single values, matches, or specific capture groups using regex. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array | — | Yes | +| Options | | | | | +| `find` | Pattern to find using regex. | string | — | Yes | +| Formatting | | | | | +| `output_pattern` | Specifies the format to output matches and specific capture groups using backreferences (e.g., `\1`, `\2`). Default is to return entire matches. **Example**: For a regex pattern `r'(\d+)\s(\w+)'` and `output_pattern = '\2 \1'`, with input `'120 volt'`, the output would be `'volt 120'`. | string, null | `null` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Extracting Number of Months From Range +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,30 +49,29 @@ wrangles:
-
+
-##### Input Sample - -_No sample available._ +| Product | +| --- | +| 3.4 gpm water pump | +| 2gpm water pump |
-
- -##### Output Sample +
-| GPM | Product | -| --- | --- | -| 3.4 gpm | 3.4 gpm water pump | -| 2gpm | 2gpm water pump | +| GPM | +| --- | +| 3.4 gpm | +| 2gpm |
-#### Implementing output_pattern -##### Recipe + + ```yaml wrangles: @@ -94,38 +84,55 @@ wrangles:
-
+
-##### Input Sample +| Product | +| --- | +| 3.4 gpm water pump for 5.5 gallon tank | +| 2gpm water pump for 2 gal tank | + +
-_No sample available._ +
+ +| GPM | +| --- | +| 3.4 Gallons Per Minute | +| 2 Gallons Per Minute | + +
-
+
-##### Output Sample +Access -| GPM | Product | +| Requirement | Value | | --- | --- | -| 3.4 Gallons Per Minute | 3.4 gpm water pump for 5.5 gallon tank | -| 2 Gallons Per Minute | 2gpm water pump for 2 gal tank | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -
+ -
+
-## Provenance +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.regex` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.regex` | + +**Sources** - [WranglesPY extract.regex implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) - [Existing extract.regex Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/regex.md) -## Registry metadata - -- Registry ID: `9aa0253a-4b70-4737-832c-964e15967289` -- Namespace: `extract` -- Recipe key: `extract.regex` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.extract.regex` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/filter.md b/wrangles-docs/registry-docs/filter.md index 63379d8..336a37a 100644 --- a/wrangles-docs/registry-docs/filter.md +++ b/wrangles-docs/registry-docs/filter.md @@ -3,56 +3,46 @@ title: "Filter" description: "Filter the dataframe based on the contents. If multiple filters are specified, all must be correct. For complex filters, use the where parameter." sidebar_label: "Filter" slug: "/filter" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Filter Filter the dataframe based on the contents. If multiple filters are specified, all must be correct. For complex filters, use the where parameter. -> Pilot Registry entry. Runtime contract status: `verified`. +Filter the dataframe based on the contents. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | No | string, integer, array | Name of the column to filter on. If multiple are provided, all must match the criteria. | `[]` | -| `equal` | No | string, array, null | Select rows where the values equal a given value. | `null` | -| `not_equal` | No | string, array, null | Select rows where the values do not equal a given value. | `null` | -| `is_in` | No | string, array, null | Select rows where the values are in a given list. | `null` | -| `not_in` | No | string, array, null | Select rows where the values are not in a given list. | `null` | -| `greater_than` | No | integer, number, null | Select rows where the values are greater than a specified value. Does include the value itself. | `null` | -| `greater_than_equal_to` | No | integer, number, null | Select rows where the values are greater than a specified value. Does include the value itself. | `null` | -| `less_than` | No | integer, number, null | Select rows where the values are less than a specified value. Does not include the value itself. | `null` | -| `less_than_equal_to` | No | integer, number, null | Select rows where the values are less than a specified value. Does include the value itself. | `null` | -| `between` | No | array, null | Value or list of values to filter that are in between two parameter values. | `null` | -| `contains` | No | string, null | Select rows where the input contains the value. Allows regular expressions. | `null` | -| `not_contains` | No | string, null | Select rows where the input does not contain the value. Allows regular expressions. | `null` | -| `is_null` | No | boolean, null | If true, select all rows where the value is NULL. If false, where is not NULL. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Filter the dataframe based on the contents. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to filter on. If multiple are provided, all must match the criteria. | string, integer, array | `[]` | No | +| Options | | | | | +| `equal` | Select rows where the values equal a given value. | string, array, null | `null` | No | +| `not_equal` | Select rows where the values do not equal a given value. | string, array, null | `null` | No | +| `is_in` | Select rows where the values are in a given list. | string, array, null | `null` | No | +| `not_in` | Select rows where the values are not in a given list. | string, array, null | `null` | No | +| `greater_than` | Select rows where the values are greater than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `greater_than_equal_to` | Select rows where the values are greater than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `less_than` | Select rows where the values are less than a specified value. Does not include the value itself. | integer, number, null | `null` | No | +| `less_than_equal_to` | Select rows where the values are less than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `between` | Value or list of values to filter that are in between two parameter values. | array, null | `null` | No | +| `contains` | Select rows where the input contains the value. Allows regular expressions. | string, null | `null` | No | +| `not_contains` | Select rows where the input does not contain the value. Allows regular expressions. | string, null | `null` | No | +| `is_null` | If true, select all rows where the value is NULL. If false, where is not NULL. | boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Filtering a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -65,9 +55,7 @@ wrangles:
-
- -##### Input Sample +
| Color | Fruit | | --- | --- | @@ -78,9 +66,7 @@ wrangles:
-
- -##### Output Sample +
| Color | Fruit | | --- | --- | @@ -91,17 +77,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `filter` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.filter` | + +**Sources** - [WranglesPY filter implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing filter Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/filter.md) -## Registry metadata - -- Registry ID: `e3242acf-d204-433f-8373-205b77481131` -- Namespace: root-level runtime key -- Recipe key: `filter` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.filter` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/dates.md b/wrangles-docs/registry-docs/format/dates.md index b97fde7..b633de3 100644 --- a/wrangles-docs/registry-docs/format/dates.md +++ b/wrangles-docs/registry-docs/format/dates.md @@ -3,46 +3,36 @@ title: "Dates" description: "Format a date." sidebar_label: "Dates" slug: "/format/dates" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Dates Format a date. -> Pilot Registry entry. Runtime contract status: `verified`. +Format a date into the desired pattern. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `format` | Yes | string | String pattern to format date. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Format a date into the desired pattern. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Formatting | | | | | +| `format` | String pattern to format date. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Changing The Format of a Date +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Date | | --- | @@ -64,29 +52,45 @@ wrangles:
-
- -##### Output Sample +
-| Date | Output Format | -| --- | --- | -| 6/23/1912 | 1912-06-23 | +| Output Format | +| --- | +| 1912-06-23 |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.dates` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.dates` | + +**Sources** - [WranglesPY format.dates implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.dates Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/dates.md) -## Registry metadata - -- Registry ID: `a3c15135-4f7a-4659-83fd-f657afa603c9` -- Namespace: `format` -- Recipe key: `format.dates` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.dates` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/pad.md b/wrangles-docs/registry-docs/format/pad.md index 2a51ae0..9fc9882 100644 --- a/wrangles-docs/registry-docs/format/pad.md +++ b/wrangles-docs/registry-docs/format/pad.md @@ -3,49 +3,40 @@ title: "Pad" description: "Pad a string to a fixed length." sidebar_label: "Pad" slug: "/format/pad" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Pad Pad a string to a fixed length. -> Pilot Registry entry. Runtime contract status: `verified`. +Pad a string to a fixed length ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `pad_length` | Yes | integer | Length for the output. | — | -| `side` | Yes | string | Side from which to fill resulting string. | — | -| `char` | Yes | string | The character to pad the input with. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `skip_empty` | No | boolean | If true, skip padding for empty or whitespace-only values. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Pad a string to a fixed length +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `side` | Side from which to fill resulting string. | string | — | Yes | +| `skip_empty` | If true, skip padding for empty or whitespace-only values. | boolean | `false` | No | +| Formatting | | | | | +| `pad_length` | Length for the output. | integer | — | Yes | +| `char` | The character to pad the input with. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Adding Dashes to Part Numbers +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| Part Number | | --- | @@ -71,9 +60,7 @@ wrangles:
-
- -##### Output Sample +
| Part Number | | --- | @@ -86,17 +73,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.pad` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.pad` | + +**Sources** - [WranglesPY format.pad implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.pad Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/pad.md) -## Registry metadata - -- Registry ID: `76c19378-38f4-45aa-85d1-3cdf8f8aae29` -- Namespace: `format` -- Recipe key: `format.pad` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.pad` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/prefix.md b/wrangles-docs/registry-docs/format/prefix.md index 63f1afc..530b383 100644 --- a/wrangles-docs/registry-docs/format/prefix.md +++ b/wrangles-docs/registry-docs/format/prefix.md @@ -3,47 +3,37 @@ title: "Prefix" description: "Add a prefix to a column." sidebar_label: "Prefix" slug: "/format/prefix" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Prefix Add a prefix to a column. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `value` | Yes | string, integer, number | Prefix value to add. | — | -| `output` | No | string, array, null | (Optional) Name of the output column. | `null` | -| `skip_empty` | No | boolean | Whether to skip empty values. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Add a prefix to a column. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | (Optional) Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `value` | Prefix value to add. | string, integer, number | — | Yes | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Adding a Prefix to a String +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Data | | --- | @@ -67,31 +55,47 @@ wrangles:
-
- -##### Output Sample +
-| Data | Prefix | -| --- | --- | -| freeze | antifreeze | -| dote | antidote | -| hero | antihero | +| Prefix | +| --- | +| antifreeze | +| antidote | +| antihero |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.prefix` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.prefix` | + +**Sources** - [WranglesPY format.prefix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.prefix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/prefix.md) -## Registry metadata - -- Registry ID: `c12f99b9-2363-4da7-8405-7c73b87906e5` -- Namespace: `format` -- Recipe key: `format.prefix` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.prefix` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/price-breaks.md b/wrangles-docs/registry-docs/format/price-breaks.md index a18ba6b..ca977b7 100644 --- a/wrangles-docs/registry-docs/format/price-breaks.md +++ b/wrangles-docs/registry-docs/format/price-breaks.md @@ -3,56 +3,69 @@ title: "Format Price Breaks" description: "Expand non-empty price-break cells into paired category and value columns." sidebar_label: "Format Price Breaks" slug: "/format/price-breaks" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Format Price Breaks Expand non-empty price-break cells into paired category and value columns. -> Pilot Registry entry. Runtime contract status: `verified`. +## Behavior + +Expand non-empty price-break cells into paired category and value columns. + +This guidance was derived from the callable signature. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | +
+ +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | -| `input` | Yes | array | Name, index, or list of input columns. | — | -| `categoryLabel` | Yes | string | Prefix for output columns that identify the source price-break category. | — | -| `valueLabel` | Yes | string | Prefix for output columns that contain the corresponding price-break value. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | array | — | Yes | +| Formatting | | | | | +| `categoryLabel` | Prefix for output columns that identify the source price-break category. | string | — | Yes | +| `valueLabel` | Prefix for output columns that contain the corresponding price-break value. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
-## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-## Behavior +
-Expand non-empty price-break cells into paired category and value columns. +Technical details -This first-pass guidance is derived from the callable signature. +| Field | Value | +| --- | --- | +| Recipe key | `format.price_breaks` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.price_breaks` | -## Provenance +**Sources** - [WranglesPY format.price_breaks implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: `format` -- Recipe key: `format.price_breaks` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.price_breaks` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/remove-duplicates.md b/wrangles-docs/registry-docs/format/remove-duplicates.md index 52f2fd1..ba4b917 100644 --- a/wrangles-docs/registry-docs/format/remove-duplicates.md +++ b/wrangles-docs/registry-docs/format/remove-duplicates.md @@ -3,46 +3,36 @@ title: "Remove Duplicates" description: "Remove duplicates from a list. Preserves input order." sidebar_label: "Remove Duplicates" slug: "/format/remove-duplicates" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Remove Duplicates Remove duplicates from a list. Preserves input order. -> Pilot Registry entry. Runtime contract status: `verified`. +Remove duplicate values in a list. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `ignore_case` | No | boolean | Ignore case when removing duplicates. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Remove duplicate values in a list. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `ignore_case` | Ignore case when removing duplicates. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Removing Duplicates From a List +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Rank | Attack of the Clones | | --- | --- | @@ -65,30 +53,46 @@ wrangles:
-
+
-##### Output Sample - -| Rank | Commander | Attack of the Clones | -| --- | --- | --- | -| Commander | ['Cody'] | ['Cody', 'Cody', 'Cody'] | -| Captain | | ['Rex', 'Rex', 'Rex'] | +| Commander | +| --- | +| ['Cody'] | +| |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.remove_duplicates` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.remove_duplicates` | + +**Sources** - [WranglesPY format.remove_duplicates implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.remove_duplicates Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/remove-duplicates.md) -## Registry metadata - -- Registry ID: `283b9e78-b2b2-43d0-844f-9842c33120aa` -- Namespace: `format` -- Recipe key: `format.remove_duplicates` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.remove_duplicates` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/significant-figures.md b/wrangles-docs/registry-docs/format/significant-figures.md index a4f4f6d..dca1ff7 100644 --- a/wrangles-docs/registry-docs/format/significant-figures.md +++ b/wrangles-docs/registry-docs/format/significant-figures.md @@ -3,46 +3,36 @@ title: "Significant Figures" description: "Format a value to a specific number of significant figures." sidebar_label: "Significant Figures" slug: "/format/significant-figures" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Significant Figures Format a value to a specific number of significant figures. -> Pilot Registry entry. Runtime contract status: `verified`. +Format a value to a specific number of significant figures ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `significant_figures` | No | integer | Number of significant figures to format to. Default is 3. | `3` | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Format a value to a specific number of significant figures +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Formatting | | | | | +| `significant_figures` | Number of significant figures to format to. Default is 3. | integer | `3` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Rounding to Significant Figures +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Data | | --- | @@ -66,31 +54,47 @@ wrangles:
-
- -##### Output Sample +
-| Data | Data to 2 Figures | -| --- | --- | -| 1.25 | 1.2 | -| 12.3 | 12 | -| 55.6 | 55 | +| Data to 2 Figures | +| --- | +| 1.2 | +| 12 | +| 55 |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.significant_figures` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.significant_figures` | + +**Sources** - [WranglesPY format.significant_figures implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.significant_figures Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/significant-figures.md) -## Registry metadata - -- Registry ID: `a359f72a-5250-4dd8-84f6-8a8173bee0f6` -- Namespace: `format` -- Recipe key: `format.significant_figures` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.significant_figures` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/suffix.md b/wrangles-docs/registry-docs/format/suffix.md index a222525..b407dc8 100644 --- a/wrangles-docs/registry-docs/format/suffix.md +++ b/wrangles-docs/registry-docs/format/suffix.md @@ -3,47 +3,37 @@ title: "Suffix" description: "Add a suffix to a column." sidebar_label: "Suffix" slug: "/format/suffix" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Suffix Add a suffix to a column. -> Pilot Registry entry. Runtime contract status: `verified`. +Add a suffix to a column ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `value` | Yes | string, integer, number, array | Suffix value to add. | — | -| `output` | No | string, null | (Optional) Name of the output column. | `null` | -| `skip_empty` | No | boolean | Whether to skip empty values. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Add a suffix to a column +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | (Optional) Name of the output column. | string, null | `null` | No | +| Options | | | | | +| `value` | Suffix value to add. | string, integer, number, array | — | Yes | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Adding a Suffix to a String +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Data | | --- | @@ -67,31 +55,47 @@ wrangles:
-
- -##### Output Sample +
-| Data | Suffix | -| --- | --- | -| sto | stoic | -| hero | heroic | -| icon | iconic | +| Suffix | +| --- | +| stoic | +| heroic | +| iconic |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.suffix` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.suffix` | + +**Sources** - [WranglesPY format.suffix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.suffix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/suffix.md) -## Registry metadata - -- Registry ID: `8d127060-ba2d-4934-897f-07662e01e40b` -- Namespace: `format` -- Recipe key: `format.suffix` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.suffix` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/format/trim.md b/wrangles-docs/registry-docs/format/trim.md index 9db7a1d..6e92c7b 100644 --- a/wrangles-docs/registry-docs/format/trim.md +++ b/wrangles-docs/registry-docs/format/trim.md @@ -3,49 +3,38 @@ title: "Trim" description: "Remove excess whitespace at the start and end of text." sidebar_label: "Trim" slug: "/format/trim" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Trim Remove excess whitespace at the start and end of text. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Remove excess whitespace at the start and end of text. Can accept multiple columns. :::note Non-string values pass through unaltered. ::: -## Migrated examples -#### Trimming a String +## Parameters + +
-##### Recipe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +## Examples ```yaml wrangles: @@ -57,9 +46,7 @@ wrangles:
-
- -##### Input Sample +
| col1 | | --- | @@ -67,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| col1 | col1 trimmed | | --- | --- | @@ -79,17 +64,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.trim` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.trim` | + +**Sources** - [WranglesPY format.trim implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) - [Existing format.trim Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/trim.md) -## Registry metadata - -- Registry ID: `af16b3c1-c230-4868-8ebe-f574904a0c76` -- Namespace: `format` -- Recipe key: `format.trim` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.format.trim` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/generate/ai.md b/wrangles-docs/registry-docs/generate/ai.md index 95200c1..4d921d7 100644 --- a/wrangles-docs/registry-docs/generate/ai.md +++ b/wrangles-docs/registry-docs/generate/ai.md @@ -3,59 +3,54 @@ title: "Ai" description: "Generate structured AI output for each recipe row." sidebar_label: "Ai" slug: "/generate/ai" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Ai Generate structured AI output for each recipe row. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `api_key` | Yes | string | OpenAI-compatible API key. | — | -| `output` | Yes | string, array, object | Target schema; string/array shorthands are expanded automatically. | — | -| `input` | No | string, array, null | Column(s) to concatenate into the prompt (defaults to all columns). | `null` | -| `model` | No | string | Responses model name (e.g. gpt-5-mini). | `"gpt-5"` | -| `threads` | No | integer | Maximum concurrent requests (default 20). | `20` | -| `timeout` | No | integer | Per-request timeout in seconds. | `90` | -| `retries` | No | integer | Number of retry attempts on failure. | `0` | -| `messages` | No | array, object, null | Optional extra messages forwarded to the inner generate helper. | `null` | -| `url` | No | string | Override for the OpenAI-compatible endpoint. | `"https://api.openai.com/v1/responses"` | -| `strict` | No | boolean | Enforce JSON-schema validation on the response. | `false` | -| `web_search` | No | boolean | Enable DuckDuckGo context lookup per row. | `false` | -| `reasoning` | No | string, object | Responses API reasoning options (forwarded verbatim). | `{"effort":"low"}` | -| `previous_response` | No | boolean | Chain responses by reusing previous_response_id for field-by-field calls. | `false` | -| `summary` | No | boolean | Request summary text to be merged into the output. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +## Parameters -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Target schema; string/array shorthands are expanded automatically. | string, array, object | — | Yes | +| `input` | Column(s) to concatenate into the prompt (defaults to all columns). | string, array, null | `null` | No | +| Options | | | | | +| `messages` | Optional extra messages forwarded to the inner generate helper. | array, object, null | `null` | No | +| `web_search` | Enable DuckDuckGo context lookup per row. | boolean | `false` | No | +| `summary` | Request summary text to be merged into the output. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Maximum concurrent requests (default 20). | integer | `20` | No | +| `timeout` | Per-request timeout in seconds. | integer | `90` | No | +| Errors | | | | | +| `retries` | Number of retry attempts on failure. | integer | `0` | No | +| Details | | | | | +| `api_key` | OpenAI-compatible API key. | string | — | Yes | +| `model` | Responses model name (e.g. gpt-5-mini). | string | `"gpt-5"` | No | +| `url` | Override for the OpenAI-compatible endpoint. | string | `"https://api.openai.com/v1/responses"` | No | +| `strict` | Enforce JSON-schema validation on the response. | boolean | `false` | No | +| `reasoning` | Responses API reasoning options (forwarded verbatim). | string, object | `{"effort":"low"}` | No | +| `previous_response` | Chain responses by reusing previous_response_id for field-by-field calls. | boolean | `false` | No | -Generate structured AI output for each recipe row. +
-## Migrated examples -#### Generate Structured Product Data +## Examples This template generates a category from a product description. AI-generated values can vary by model and request. -##### Recipe + ```yaml wrangles: @@ -72,9 +67,7 @@ wrangles:
-
- -##### Input Sample +
| Product Description | | --- | @@ -82,13 +75,11 @@ wrangles:
-
- -##### Output Sample +
-| Product Description | Category | -| --- | --- | -| Stainless steel insulated water bottle, 750 ml | Drinkware | +| Category | +| --- | +| Drinkware |
@@ -96,17 +87,35 @@ wrangles: _Template based on the documented parameters; no published source example is currently available._ -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `generate.ai` | +| Lifecycle status | active | +| Namespace | `generate` | +| Documentation group | `generate` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.generate.ai` | + +**Sources** - [WranglesPY generate.ai implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/generate.py) - [Existing generate.ai Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/ai.md) -## Registry metadata - -- Registry ID: `12c31c6c-cade-484d-84ba-7f302bf6af52` -- Namespace: `generate` -- Recipe key: `generate.ai` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.generate.ai` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/huggingface.md b/wrangles-docs/registry-docs/huggingface.md index d31ed6f..8e3c5a4 100644 --- a/wrangles-docs/registry-docs/huggingface.md +++ b/wrangles-docs/registry-docs/huggingface.md @@ -3,50 +3,43 @@ title: "Huggingface" description: "Use a model from huggingface." sidebar_label: "Huggingface" slug: "/huggingface" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Huggingface Use a model from huggingface. -> Pilot Registry entry. Runtime contract status: `verified`. +Use a model from huggingface ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `api_token` | Yes | string | Huggingface API Token. | — | -| `model` | Yes | string | Name of the model to use. e.g. facebook/bart-large-cnn. | — | -| `output` | No | string, array, null | Name of the output column. If not provided, will overwrite the input column. | `null` | -| `parameters` | No | object, null | Optionally, provide additional parameters to define the model behaviour. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If not provided, will overwrite the input column. | string, array, null | `null` | No | +| Options | | | | | +| `parameters` | Optionally, provide additional parameters to define the model behaviour. | object, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `api_token` | Huggingface API Token. | string | — | Yes | +| `model` | Name of the model to use. e.g. facebook/bart-large-cnn. | string | — | Yes | -Use a model from huggingface +
-## Migrated examples -#### Summarize Product Descriptions +## Examples This template uses a Hugging Face summarization model. The exact response shape and text depend on the selected model. -##### Recipe + ```yaml wrangles: @@ -61,9 +54,7 @@ wrangles:
-
- -##### Input Sample +
| Product Description | | --- | @@ -71,13 +62,11 @@ wrangles:
-
- -##### Output Sample +
-| Product Description | Summary | -| --- | --- | -| A cordless drill with two batteries, a charger, and a compact carrying case. | Cordless drill kit with batteries, charger, and case. | +| Summary | +| --- | +| Cordless drill kit with batteries, charger, and case. |
@@ -85,17 +74,35 @@ wrangles: _Template based on the documented parameters; no published source example is currently available._ -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `huggingface` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `ai` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.huggingface` | + +**Sources** - [WranglesPY huggingface implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing huggingface Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/huggingface.md) -## Registry metadata - -- Registry ID: `ae12cf20-4934-428f-84a9-a6898cb7ffe0` -- Namespace: root-level runtime key -- Recipe key: `huggingface` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.huggingface` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/index.md b/wrangles-docs/registry-docs/index.md index 1f2b0db..924a06d 100644 --- a/wrangles-docs/registry-docs/index.md +++ b/wrangles-docs/registry-docs/index.md @@ -1,112 +1,33 @@ --- -title: Wrangles Registry Pilot -description: Pilot of the versioned Wrangles recipe knowledge registry. +title: Wrangles Registry +description: Pre-production version of the Wrangles recipe knowledge registry. slug: / +registry_entry: true --- -# Wrangles Registry Pilot +# Wrangles Registry -This preview contains the first Registry records compiled from the new -Markdown contract. These pages are not yet the production replacement for the -existing wrangle reference. +These pages are compiled from the Registry Markdown contract. The Registry is +pre-production until the migration and production-readiness criteria in its +contract are complete. Wrangles are grouped using their Registry namespace; +root-level compatibility keys remain in their existing documentation group. -| Wrangle | Description | Runtime contract | -| --- | --- | --- | -| [`accordion`](/wrangles/accordion) | Apply a series of wrangles to column(s) containing lists. The wrangles will be applied to each element in the list and the results will be returned back as a list. | `verified` | -| [`batch`](/wrangles/batch) | Split the data into batches for executing a list of wrangles. Use this in situations such as where the intermediate data is too large to fit in memory. | `verified` | -| [`classify`](/wrangles/classify) | Run classify wrangles on the specified columns. Requires WrangleWorks Account and Subscription. | `verified` | -| [`clean_whitespaces`](/wrangles/clean-whitespaces) | Condense multiple spaces to a single space and convert special space characters to a standard space. | `verified` | -| [`compare.lists`](/wrangles/compare/lists) | Compare multiple lists and return the intersection, difference, or union. | `verified` | -| [`compare.text`](/wrangles/compare/text) | Compare two strings and return the intersection or difference, use overlap to find the matching characters between the two strings, or use similarity to get a numeric similarity score. | `verified` | -| [`compute.case_when`](/wrangles/compute/case-when) | Assign values to a column based on conditional logic. | `verified` | -| [`compute.score_search_results`](/wrangles/compute/score-search-results) | Scores and filters search results based on progressive partial/exact matching. Can return dictionaries or a parallel list of formatted strings. | `verified` | -| [`concurrent`](/wrangles/concurrent) | Run multiple wrangles concurrently rather than sequentially. Wrangles must specify output columns to be used concurrently. When using concurrent, Wrangles may not complete in a predictable order and it is not recommended to update overlapping columns with different wrangles. | `verified` | -| [`convert.case`](/wrangles/convert/case) | Change the letter case of text values. | `verified` | -| [`convert.data_type`](/wrangles/convert/data-type) | Convert values to strings, numbers, booleans, or datetimes. | `verified` | -| [`convert.fraction_to_decimal`](/wrangles/convert/fraction-to-decimal) | Convert fractions to decimals. | `verified` | -| [`convert.from_json`](/wrangles/convert/from-json) | Parse JSON text into lists, objects, scalars, booleans, or null values. | `verified` | -| [`convert.from_yaml`](/wrangles/convert/from-yaml) | Convert a YAML representation into an object. | `verified` | -| [`convert.to_json`](/wrangles/convert/to-json) | Convert an object to a JSON representation. | `verified` | -| [`convert.to_yaml`](/wrangles/convert/to-yaml) | Convert an object to a YAML representation. | `verified` | -| [`copy`](/wrangles/copy) | Make a copy of a column or a list of columns. | `verified` | -| [`create.bins`](/wrangles/create/bins) | Create a column that groups data into bins. | `verified` | -| [`create.column`](/wrangles/create/column) | Create column(s) with a user defined value. Defaults to None (empty). | `verified` | -| [`create.embeddings`](/wrangles/create/embeddings) | Create an embedding based on text input. | `verified` | -| [`create.guid`](/wrangles/create/guid) | Create column(s) with a GUID. | `verified` | -| [`create.hash`](/wrangles/create/hash) | Create a hash of a column. | `verified` | -| [`create.index`](/wrangles/create/index) | Create column(s) with an incremental index. e.g. 1,2,3... | `verified` | -| [`create.jinja`](/wrangles/create/jinja) | Output text using a jinja template. | `verified` | -| [`create.uuid`](/wrangles/create/uuid) | Create column(s) with a UUID. | `verified` | -| [`date_calculator`](/wrangles/date-calculator) | Add or Subtract time from a date. | `verified` | -| [`drop`](/wrangles/drop) | Drop (Delete) selected column(s). | `verified` | -| [`explode`](/wrangles/explode) | Explode a column of lists into rows. | `verified` | -| [`extract.address`](/wrangles/extract/address) | Extract parts of addresses. Requires WrangleWorks Account. | `verified` | -| [`extract.ai`](/wrangles/extract/ai) | Extract structured data from each input row using an AI model. Define the desired fields with output, or reuse a saved definition with model_id. | `verified` | -| [`extract.attributes`](/wrangles/extract/attributes) | Extract numeric attributes from the input such as weights or lengths. Requires WrangleWorks Account. | `verified` | -| [`extract.brackets`](/wrangles/extract/brackets) | Extract text properties in brackets from the input. | `verified` | -| [`extract.codes`](/wrangles/extract/codes) | Extract alphanumeric codes from the input. Requires WrangleWorks Account. | `verified` | -| [`extract.custom`](/wrangles/extract/custom) | Extract data from the input using a DIY or bespoke extraction wrangle. Requires WrangleWorks Account and Subscription. | `verified` | -| [`extract.date_properties`](/wrangles/extract/date-properties) | Extract date properties from a date (day, month, year, etc...). | `verified` | -| [`extract.date_range`](/wrangles/extract/date-range) | Extract date range frequency from two dates. | `verified` | -| [`extract.html`](/wrangles/extract/html) | Extract elements from strings containing html. Requires WrangleWorks Account. | `verified` | -| [`extract.properties`](/wrangles/extract/properties) | Extract text properties from the input. Requires WrangleWorks Account. | `verified` | -| [`extract.regex`](/wrangles/extract/regex) | Extract matches or specific capture groups using regex. | `verified` | -| [`filter`](/wrangles/filter) | Filter the dataframe based on the contents. If multiple filters are specified, all must be correct. For complex filters, use the where parameter. | `verified` | -| [`format.dates`](/wrangles/format/dates) | Format a date. | `verified` | -| [`format.pad`](/wrangles/format/pad) | Pad a string to a fixed length. | `verified` | -| [`format.prefix`](/wrangles/format/prefix) | Add a prefix to a column. | `verified` | -| [`format.price_breaks`](/wrangles/format/price-breaks) | Expand non-empty price-break cells into paired category and value columns. | `verified` | -| [`format.remove_duplicates`](/wrangles/format/remove-duplicates) | Remove duplicates from a list. Preserves input order. | `verified` | -| [`format.significant_figures`](/wrangles/format/significant-figures) | Format a value to a specific number of significant figures. | `verified` | -| [`format.suffix`](/wrangles/format/suffix) | Add a suffix to a column. | `verified` | -| [`format.trim`](/wrangles/format/trim) | Remove excess whitespace at the start and end of text. | `verified` | -| [`generate.ai`](/wrangles/generate/ai) | Generate structured AI output for each recipe row. | `verified` | -| [`huggingface`](/wrangles/huggingface) | Use a model from huggingface. | `verified` | -| [`log`](/wrangles/log) | Log the current status of the dataframe. | `verified` | -| [`lookup`](/wrangles/lookup) | Lookup values from a saved lookup wrangle. | `verified` | -| [`math`](/wrangles/math) | Apply a mathematical calculation. | `verified` | -| [`maths`](/wrangles/maths) | Deprecated alias for `math`; evaluate an expression and write its result to an output column. | `verified` | -| [`matrix`](/wrangles/matrix) | Apply a matrix of wrangles to the dataframe. This will run the wrangles for each combination of the variables. | `verified` | -| [`merge.coalesce`](/wrangles/merge/coalesce) | Take the first non-empty value from a series of columns or lists. | `verified` | -| [`merge.concatenate`](/wrangles/merge/concatenate) | Concatenate a list of columns or a list within a single column. | `verified` | -| [`merge.dictionaries`](/wrangles/merge/dictionaries) | Take dictionaries in multiple columns and merge them to a single dictionary. | `verified` | -| [`merge.key_value_pairs`](/wrangles/merge/key-value-pairs) | Create a dictionary from keys and values in paired columns e.g. COLUMN_NAME_1, COLUMN_VALUE_1, COLUMN_NAME_2, COLUMN_VALUE_2 ... | `verified` | -| [`merge.lists`](/wrangles/merge/lists) | Take lists in multiple columns and merge them to a single list. | `verified` | -| [`merge.to_dict`](/wrangles/merge/to-dict) | Take multiple columns and merge them to a dictionary (aka object) using the column headers as keys. | `verified` | -| [`merge.to_list`](/wrangles/merge/to-list) | Take multiple columns and merge them to a list. | `verified` | -| [`python`](/wrangles/python) | Apply a simple single-line python command. For more complex python use a custom function. Note, this evaluates the python command - be especially cautious including variables from untrusted sources within the command string. The python command will be evaluated once for each row and the result returned. Reference column values by using their name. Non-alphanumeric characters within column names are replaced by underscores (_) Additionally, all columns are available as a dict named kwargs. Additional parameters set for the wrangle will also be available to the command. | `verified` | -| [`recipe`](/wrangles/recipe) | Run another recipe as a wrangle against the current dataframe. | `verified` | -| [`reindex`](/wrangles/reindex) | Changes the row labels and column labels of a DataFrame. | `verified` | -| [`remove_words`](/wrangles/remove-words) | Remove all the elements that occur in one list from another. | `verified` | -| [`rename`](/wrangles/rename) | Rename a column or list of columns. | `verified` | -| [`replace`](/wrangles/replace) | Quick find and replace for simple values. Can use regex if 'input' in params and isinstance(params['input'], list):in the find field. | `verified` | -| [`round`](/wrangles/round) | Round column(s) to the specified decimals. | `verified` | -| [`search.find_links`](/wrangles/search/find-links) | Perform web searches to find links. Returns structured search results with titles, links, snippets, and optional pricing. | `verified` | -| [`search.retrieve_link_content`](/wrangles/search/retrieve-link-content) | Retrieves targeted content from web pages using LLM URL extraction. Can optionally output a second column containing a clean, human-readable text summary of the retrieved data. | `verified` | -| [`select.columns`](/wrangles/select/columns) | Select columns from the dataframe. | `verified` | -| [`select.dictionary_element`](/wrangles/select/dictionary-element) | Select one or more element of a dictionary. | `verified` | -| [`select.element`](/wrangles/select/element) | Select elements of lists or dicts using python syntax like col[0]['key']. | `verified` | -| [`select.group_by`](/wrangles/select/group-by) | Group and aggregate the data. | `verified` | -| [`select.head`](/wrangles/select/head) | Return the first n rows. | `verified` | -| [`select.highest_confidence`](/wrangles/select/highest-confidence) | Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form [<<value>>, <<confidence_score>>]. | `verified` | -| [`select.left`](/wrangles/select/left) | Return characters from the left of text. Strings shorter than the length defined will be unaffected. | `verified` | -| [`select.length`](/wrangles/select/length) | Calculate the lengths of data in a column. The length depends on the data type e.g. text will be the length of the text, lists will be the number of elements in the list. | `verified` | -| [`select.list_element`](/wrangles/select/list-element) | Select a numbered element of a list (zero indexed). | `verified` | -| [`select.right`](/wrangles/select/right) | Return characters from the right of text. Strings shorter than the length defined will be unaffected. | `verified` | -| [`select.sample`](/wrangles/select/sample) | Return a random sample of the rows. | `verified` | -| [`select.substring`](/wrangles/select/substring) | Return characters from the middle of text. | `verified` | -| [`select.tail`](/wrangles/select/tail) | Return the last n rows. | `verified` | -| [`select.threshold`](/wrangles/select/threshold) | Select the first option if it exceeds a given threshold, else the second option. | `verified` | -| [`similarity`](/wrangles/similarity) | Calculate the cosine similarity of two vectors. | `verified` | -| [`sort`](/wrangles/sort) | Sort the data. | `verified` | -| [`split.dictionary`](/wrangles/split/dictionary) | Split one or more dictionaries into columns. The dictionary keys will be returned as the new column headers. If the dictionaries contain overlapping values, the last value will be returned. | `verified` | -| [`split.list`](/wrangles/split/list) | Split a list in a single column to multiple columns. | `verified` | -| [`split.text`](/wrangles/split/text) | Split a string to multiple columns or a list. | `verified` | -| [`split.tokenize`](/wrangles/split/tokenize) | Split text into tokens. A variety of methods are available. The default method is to split on spaces. | `verified` | -| [`sql`](/wrangles/sql) | Apply a SQL command to the current dataframe. Only SELECT statements are supported - the result will be the output. | `verified` | -| [`standardize`](/wrangles/standardize) | Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. | `verified` | -| [`standardize.clean`](/wrangles/standardize/clean) | Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. | `verified` | -| [`standardize.custom`](/wrangles/standardize/custom) | Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. | `verified` | -| [`translate`](/wrangles/translate) | Translate the input to a different language. Requires WrangleWorks Account and DeepL API Key (A free account for up to 500,000 characters per month is available). | `verified` | -| [`transpose`](/wrangles/transpose) | Transpose the DataFrame (swap columns to rows). | `verified` | -| [`try`](/wrangles/try) | Try a list of wrangles and catch any errors that occur. | `verified` | +| Namespace or group | Wrangles | Description | +| --- | ---: | --- | +| [Convert](/wrangles/namespaces/convert) | 7 | Convert wrangles and compatibility entries. | +| [Merge](/wrangles/namespaces/merge) | 7 | Merge wrangles and compatibility entries. | +| [Split](/wrangles/namespaces/split) | 5 | Split wrangles and compatibility entries. | +| [Select](/wrangles/namespaces/select) | 17 | Select wrangles and compatibility entries. | +| [Format](/wrangles/namespaces/format) | 14 | Format wrangles and compatibility entries. | +| [Create](/wrangles/namespaces/create) | 8 | Create wrangles and compatibility entries. | +| [Extract](/wrangles/namespaces/extract) | 11 | Extract wrangles and compatibility entries. | +| [Compare](/wrangles/namespaces/compare) | 3 | Compare wrangles and compatibility entries. | +| [Compute](/wrangles/namespaces/compute) | 6 | Compute wrangles and compatibility entries. | +| [Search](/wrangles/namespaces/search) | 2 | Search wrangles and compatibility entries. | +| [AI](/wrangles/namespaces/ai) | 2 | AI wrangles and compatibility entries. | +| [Generate](/wrangles/namespaces/generate) | 1 | Generate wrangles and compatibility entries. | +| [Lookup](/wrangles/namespaces/lookup) | 1 | Lookup wrangles and compatibility entries. | +| [Standardize](/wrangles/namespaces/standardize) | 2 | Standardize wrangles and compatibility entries. | +| [Transform](/wrangles/namespaces/transform) | 4 | Transform wrangles and compatibility entries. | +| [Utility](/wrangles/namespaces/utility) | 7 | Utility wrangles and compatibility entries. | +| [Date](/wrangles/namespaces/date) | 1 | Date wrangles and compatibility entries. | diff --git a/wrangles-docs/registry-docs/log.md b/wrangles-docs/registry-docs/log.md index 895f2db..69f6b5b 100644 --- a/wrangles-docs/registry-docs/log.md +++ b/wrangles-docs/registry-docs/log.md @@ -3,49 +3,39 @@ title: "Log" description: "Log the current status of the dataframe." sidebar_label: "Log" slug: "/log" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Log Log the current status of the dataframe. -> Pilot Registry entry. Runtime contract status: `verified`. +Print the current status of the dataframe. Only a sample of rows is logged. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `columns` | No | array, null | (Optional, default all columns) List of specific columns to log. | `null` | -| `write` | No | array, null | (Optional) Allows for an intermediate output to a file/dataframe/database etc. | `null` | -| `warning` | No | string, null | Log a warning to the console. | `null` | -| `info` | No | string, null | Log info to the console. | `null` | -| `log_data` | No | boolean, null | Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise. | `null` | -| `error` | No | string | Log an error to the console. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Print the current status of the dataframe. Only a sample of rows is logged. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | (Optional, default all columns) List of specific columns to log. | array, null | `null` | No | +| Options | | | | | +| `write` | (Optional) Allows for an intermediate output to a file/dataframe/database etc. | array, null | `null` | No | +| `warning` | Log a warning to the console. | string, null | `null` | No | +| `info` | Log info to the console. | string, null | `null` | No | +| `log_data` | Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise. | boolean, null | `null` | No | +| `error` | Log an error to the console. | string | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Logging All Columns to Terminal +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,17 +44,13 @@ wrangles:
-
- -##### Input Sample +
_Logging side-effect; no tabular input sample._
-
- -##### Output Sample +
_Logs a sample of rows or status information to the configured destination._ @@ -72,9 +58,9 @@ _Logs a sample of rows or status information to the configured destination._
-#### Logging Specific Columns to Terminal -##### Recipe + + ```yaml wrangles: @@ -86,17 +72,13 @@ wrangles:
-
- -##### Input Sample +
_Logging side-effect; no tabular input sample._
-
- -##### Output Sample +
_Logs a sample of rows or status information to the configured destination._ @@ -104,9 +86,9 @@ _Logs a sample of rows or status information to the configured destination._
-#### Logging to a File -##### Recipe + + ```yaml wrangles: @@ -121,17 +103,13 @@ wrangles:
-
- -##### Input Sample +
_Logging side-effect; no tabular input sample._
-
- -##### Output Sample +
_Logs a sample of rows or status information to the configured destination._ @@ -139,7 +117,7 @@ _Logs a sample of rows or status information to the configured destination._
-#### Native Variables + | Variable | Function | | --- | --- | @@ -148,17 +126,35 @@ _Logs a sample of rows or status information to the configured destination._ | `${df}` | Current dataframe. | | `${row_count}` | Number of visible rows processed in each batch. | -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `log` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.log` | + +**Sources** - [WranglesPY log implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing log Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/log.md) -## Registry metadata - -- Registry ID: `6177808e-aa2d-4d0b-8385-858b16948a5d` -- Namespace: root-level runtime key -- Recipe key: `log` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.log` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/lookup.md b/wrangles-docs/registry-docs/lookup.md index a6138ba..ee4cc99 100644 --- a/wrangles-docs/registry-docs/lookup.md +++ b/wrangles-docs/registry-docs/lookup.md @@ -3,48 +3,39 @@ title: "Lookup" description: "Lookup values from a saved lookup wrangle." sidebar_label: "Lookup" slug: "/lookup" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Lookup Lookup values from a saved lookup wrangle. -> Pilot Registry entry. Runtime contract status: `verified`. +Look up data from a saved Lookup Wrangle. Data is output as a dictionary if an output is not specified or the output does not match any columns in the lookup. If specific lookup columns are named in the output, they will be output as individual columns. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string | Name of the column(s) to lookup. | — | -| `output` | No | string, array, null | Name of the output column(s). When n is provided and the output list length equals n, each output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. "Top *" with n: 3 becomes "Top 1", "Top 2", "Top 3". | `null` | -| `model_id` | No | string, null | The model_id to use lookup against. | `null` | -| `lookup_mode` | No | string; one of: by_row, by_matrix, by_dataframe | How to perform lookups. 'by_row' (default): lookup each row individually. 'by_dataframe': lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation. | `"by_row"` | -| `n` | No | integer, null | Number of matches to return per input value. When the output list length equals n, each output column receives the corresponding match. Otherwise all n matches are stored as a list in each output column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Look up data from a saved Lookup Wrangle. Data is output as a dictionary if an output is not specified or the output does not match any columns in the lookup. If specific lookup columns are named in the output, they will be output as individual columns. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to lookup. | string | — | Yes | +| `output` | Name of the output column(s). When n is provided and the output list length equals n, each output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. "Top *" with n: 3 becomes "Top 1", "Top 2", "Top 3". | string, array, null | `null` | No | +| Options | | | | | +| `lookup_mode` | How to perform lookups. 'by_row' (default): lookup each row individually. 'by_dataframe': lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation. | string; one of:
  • by_row
  • by_matrix
  • by_dataframe
| `"by_row"` | No | +| `n` | Number of matches to return per input value. When the output list length equals n, each output column receives the corresponding match. Otherwise all n matches are stored as a list in each output column. | integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The model_id to use lookup against. | string, null | `null` | No | -## Migrated examples -#### State Example +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,9 +48,7 @@ wrangles:
-
- -##### Input Sample +
| State | | --- | @@ -69,9 +58,7 @@ wrangles:
-
- -##### Output Sample +
| Abbreviation | | --- | @@ -83,17 +70,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `lookup` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `lookup` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.lookup` | + +**Sources** - [WranglesPY lookup implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing lookup Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/lookup/_sources/lookup.md) -## Registry metadata - -- Registry ID: `b3339193-d1cc-4c89-8ed6-901efa6d81be` -- Namespace: root-level runtime key -- Recipe key: `lookup` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.lookup` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/math.md b/wrangles-docs/registry-docs/math.md index 48c9b62..de11eaf 100644 --- a/wrangles-docs/registry-docs/math.md +++ b/wrangles-docs/registry-docs/math.md @@ -3,49 +3,38 @@ title: "Math" description: "Apply a mathematical calculation." sidebar_label: "Math" slug: "/math" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Math Apply a mathematical calculation. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string | The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_). | — | -| `output` | Yes | string | The column to output the results to. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Apply mathematical calculations to columns. Also called as `maths`. :::info Spaces within column headers are replaced with underscores automatically. Account for this when writing expressions in `input`. ::: -## Migrated examples -#### Square Root Example +## Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_). | string | — | Yes | +| `output` | The column to output the results to. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Values | | --- | @@ -68,31 +55,47 @@ wrangles:
-
- -##### Output Sample +
-| Values | Square Root | -| --- | --- | -| 4 | 2 | -| 9 | 3 | -| 16 | 4 | +| Square Root | +| --- | +| 2 | +| 3 | +| 4 |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `math` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.math` | + +**Sources** - [WranglesPY math implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing math Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/math.md) -## Registry metadata - -- Registry ID: `494b11ad-00c1-4748-8b93-6bec982f4fec` -- Namespace: root-level runtime key -- Recipe key: `math` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.math` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/maths.md b/wrangles-docs/registry-docs/maths.md index 82079a3..65c6549 100644 --- a/wrangles-docs/registry-docs/maths.md +++ b/wrangles-docs/registry-docs/maths.md @@ -1,57 +1,74 @@ --- -title: "Maths" +title: "Maths (Deprecated)" description: "Deprecated alias for `math`; evaluate an expression and write its result to an output column." -sidebar_label: "Maths" +sidebar_label: "Maths (Deprecated)" slug: "/maths" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- -# Maths +# Maths (Deprecated) + +:::warning Deprecated +This compatibility wrangle remains available for existing recipes. Use [`math`](/wrangles/namespaces/compute#math) for new recipes. +::: + +Deprecated alias for `math`; evaluate an expression and write its result to an output column. + +## Behavior Deprecated alias for `math`; evaluate an expression and write its result to an output column. -> Pilot Registry entry. Runtime contract status: `verified`. +This guidance was derived from the callable signature. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | +
+ +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | -| `input` | Yes | string | Name, index, or list of input columns. | — | -| `output` | Yes | string | Name or list of output columns. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string | — | Yes | +| `output` | Name or list of output columns. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Verified examples +
-_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access + +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-## Behavior +
-Deprecated alias for `math`; evaluate an expression and write its result to an output column. +Technical details -This first-pass guidance is derived from the callable signature. +| Field | Value | +| --- | --- | +| Recipe key | `maths` | +| Lifecycle status | deprecated | +| Replaced by | [`math`](/wrangles/namespaces/compute#math) | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.maths` | -## Provenance +**Sources** - [WranglesPY maths implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `maths` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.maths` -- Status: `deprecated` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/matrix.md b/wrangles-docs/registry-docs/matrix.md index 0456ff2..8dde35b 100644 --- a/wrangles-docs/registry-docs/matrix.md +++ b/wrangles-docs/registry-docs/matrix.md @@ -3,50 +3,42 @@ title: "Matrix" description: "Apply a matrix of wrangles to the dataframe. This will run the wrangles for each combination of the variables." sidebar_label: "Matrix" slug: "/matrix" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Matrix Apply a matrix of wrangles to the dataframe. This will run the wrangles for each combination of the variables. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `variables` | Yes | object | A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values. | — | -| `wrangles` | Yes | array | The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables. | — | -| `strategy` | No | string; one of: permutations, loop | Determines how to combine variables when there are multiple. loop (default) iterates over each set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables. | `"loop"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +Apply a matrix of wrangles to the dataframe. Each wrangle runs for the configured combinations of variables, including recipe variables and variables declared by the `variables` parameter. -## Access +See the [Matrix connector](/python/connectors/matrix) for the connector equivalent. -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Apply a matrix of wrangles to the dataframe. Each wrangle runs for the configured combinations of variables, including recipe variables and variables declared by the `variables` parameter. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `strategy` | Determines how to combine variables when there are multiple. loop (default) iterates over each set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables. | string; one of:
  • permutations
  • loop
| `"loop"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `variables` | A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values. | object | — | Yes | +| `wrangles` | The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables. | array | — | Yes | -See the [Matrix connector](/python/connectors/matrix) for the connector equivalent. +
-## Migrated examples -#### Use Hardcoded Variables +## Examples Run a custom function once for each configured variable value. -##### Recipe + ```yaml wrangles: @@ -67,9 +59,7 @@ def test_fn(part_code, value):
-
- -##### Input Sample +
| Part Code | | --- | @@ -80,26 +70,24 @@ def test_fn(part_code, value):
-
+
-##### Output Sample - -| Part Code | Part Code A | Part Code B | Part Code C | -| --- | --- | --- | --- | -| 6202 | 6202A | 6202B | 6202C | -| br549 | br549A | br549B | br549C | -| 554-114 | 554-114A | 554-114B | 554-114C | -| 554-112 | 554-112A | 554-112B | 554-112C | +| Part Code A | Part Code B | Part Code C | +| --- | --- | --- | +| 6202A | 6202B | 6202C | +| br549A | br549B | br549C | +| 554-114A | 554-114B | 554-114C | +| 554-112A | 554-112B | 554-112C |
-#### Use Unique Variables Per Row + This example runs `extract.custom` once for each unique model ID. -##### Recipe + ```yaml wrangles: @@ -118,9 +106,7 @@ wrangles:
-
- -##### Input Sample +
| Description | Model ID | | --- | --- | @@ -130,21 +116,19 @@ wrangles:
-
+
-##### Output Sample - -| Description | Model ID | Extracted Values | -| --- | --- | --- | -| The SKF 6202 bearing is the best bearing in the world | xxxxxxxx-xxxx-xxxx | 6202 | -| The Timken 6102 bearing is indestructible | yyyyyyyy-yyyy-yyyy | indestructible | -| The Milwaukee impact has 1200ft-lbs of torque | zzzzzzzz-zzzz-zzzz | 1200ft-lbs | +| Extracted Values | +| --- | +| 6202 | +| indestructible | +| 1200ft-lbs |
-#### Native Variables + | Variable | Function | | --- | --- | @@ -153,17 +137,35 @@ wrangles: | `${df}` | Current dataframe. | | `${row_count}` | Number of visible rows processed in each batch. | -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `matrix` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.matrix` | + +**Sources** - [WranglesPY matrix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing matrix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/matrix.md) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `matrix` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.matrix` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/coalesce.md b/wrangles-docs/registry-docs/merge/coalesce.md index 6012906..b504ca1 100644 --- a/wrangles-docs/registry-docs/merge/coalesce.md +++ b/wrangles-docs/registry-docs/merge/coalesce.md @@ -3,45 +3,34 @@ title: "Coalesce" description: "Take the first non-empty value from a series of columns or lists." sidebar_label: "Coalesce" slug: "/merge/coalesce" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Coalesce Take the first non-empty value from a series of columns or lists. -> Pilot Registry entry. Runtime contract status: `verified`. +Take the first non-empty value from a series of columns. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns or a single column containing lists. | — | -| `output` | No | string, null | Name of the output columns. This is required if multiple input columns are provided. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Take the first non-empty value from a series of columns. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns or a single column containing lists. | array | — | Yes | +| `output` | Name of the output columns. This is required if multiple input columns are provided. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Coalescing 3 Columns +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -68,31 +55,47 @@ wrangles:
-
+
-##### Output Sample - -| Col1 | Col2 | Col3 | Output Col | -| --- | --- | --- | --- | -| A | B | C | | -| D | E | F | D | -| G | H | I | | +| Output Col | +| --- | +| | +| D | +| |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.coalesce` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.coalesce` | + +**Sources** - [WranglesPY merge.coalesce implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.coalesce Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/coalesce.md) -## Registry metadata - -- Registry ID: `25c1a60d-fa48-4b9a-8c03-0921d5b31049` -- Namespace: `merge` -- Recipe key: `merge.coalesce` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.coalesce` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/concatenate.md b/wrangles-docs/registry-docs/merge/concatenate.md index 691bf11..16d70c3 100644 --- a/wrangles-docs/registry-docs/merge/concatenate.md +++ b/wrangles-docs/registry-docs/merge/concatenate.md @@ -3,47 +3,38 @@ title: "Concatenate" description: "Concatenate a list of columns or a list within a single column." sidebar_label: "Concatenate" slug: "/merge/concatenate" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Concatenate Concatenate a list of columns or a list within a single column. -> Pilot Registry entry. Runtime contract status: `verified`. +If the input is a list of columns, concatenate multiple columns into one as a delimited string. If the input is a single column, concatenate a list within that column into a delimited string. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Either a single column name or list of columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `char` | No | string | (Optional) Character to add between successive values. | `","` | -| `skip_empty` | No | boolean | Whether to skip empty values, defaults to false. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -If the input is a list of columns, concatenate multiple columns into one as a delimited string. If the input is a single column, concatenate a list within that column into a delimited string. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Either a single column name or list of columns. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty values, defaults to false. | boolean | `false` | No | +| Formatting | | | | | +| `char` | (Optional) Character to add between successive values. | string | `","` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Concatenating 3 Columns +
-##### Recipe +## Examples ```yaml # Using concatenate to combine multiple columns @@ -59,9 +50,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -69,9 +58,7 @@ wrangles:
-
- -##### Output Sample +
| Join Col | | --- | @@ -81,9 +68,9 @@ wrangles:
-#### Concatenating a Single Column -##### Recipe + + ```yaml # Using concatenate to join a column that is a list @@ -96,9 +83,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | | --- | @@ -106,9 +91,7 @@ wrangles:
-
- -##### Output Sample +
| Join List | | --- | @@ -118,17 +101,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.concatenate` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.concatenate` | + +**Sources** - [WranglesPY merge.concatenate implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.concatenate Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/concatenate.md) -## Registry metadata - -- Registry ID: `6ff76728-e1f6-4d3d-8946-6aa3b7524b3f` -- Namespace: `merge` -- Recipe key: `merge.concatenate` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.concatenate` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/dictionaries.md b/wrangles-docs/registry-docs/merge/dictionaries.md index edc07bf..42da0ac 100644 --- a/wrangles-docs/registry-docs/merge/dictionaries.md +++ b/wrangles-docs/registry-docs/merge/dictionaries.md @@ -3,50 +3,38 @@ title: "Dictionaries" description: "Take dictionaries in multiple columns and merge them to a single dictionary." sidebar_label: "Dictionaries" slug: "/merge/dictionaries" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Dictionaries Take dictionaries in multiple columns and merge them to a single dictionary. -> Pilot Registry entry. Runtime contract status: `verified`. +:::note +For duplicate keys, the last key in the input list takes precedence in the merged dictionary. +::: ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `skip_empty` | No | boolean | Whether to skip empty dictionaries when merging. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Take dictionaries in multiple columns and merge them to a single dictionary. - -:::note -For duplicate keys, the last key in the input list takes precedence in the merged dictionary. -::: +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty dictionaries when merging. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Using Named Columns +
-##### Recipe +## Examples ```yaml wrangles: @@ -59,9 +47,7 @@ wrangles:
-
- -##### Input Sample +
| Dict 1 | Dict 2 | | --- | --- | @@ -69,9 +55,7 @@ wrangles:
-
- -##### Output Sample +
| Merged | | --- | @@ -81,9 +65,9 @@ wrangles:
-#### Using a Wildcard (*) -##### Recipe + + ```yaml # Using a Wildcard (*) @@ -95,9 +79,7 @@ wrangles:
-
- -##### Input Sample +
| Dict 1 | Dict 2 | | --- | --- | @@ -105,9 +87,7 @@ wrangles:
-
- -##### Output Sample +
| Merged | | --- | @@ -117,9 +97,9 @@ wrangles:
-#### Using a Wildcard (*) With Not Columns -##### Recipe + + ```yaml # Using a Wildcard (*) @@ -137,7 +117,7 @@ Note the extra dash in front of `Dict 2` excludes that column from the wildcard
-##### Input Sample + | Dict 1 | Dict 2 | Dict 3 | | --- | --- | --- | @@ -147,7 +127,7 @@ Note the extra dash in front of `Dict 2` excludes that column from the wildcard
-##### Output Sample + | Merged | | --- | @@ -157,17 +137,35 @@ Note the extra dash in front of `Dict 2` excludes that column from the wildcard
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.dictionaries` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.dictionaries` | + +**Sources** - [WranglesPY merge.dictionaries implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.dictionaries Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/dictionaries.md) -## Registry metadata - -- Registry ID: `93e27737-e966-4ba9-8777-4e96724ebfc4` -- Namespace: `merge` -- Recipe key: `merge.dictionaries` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.dictionaries` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/key-value-pairs.md b/wrangles-docs/registry-docs/merge/key-value-pairs.md index 775b301..193d668 100644 --- a/wrangles-docs/registry-docs/merge/key-value-pairs.md +++ b/wrangles-docs/registry-docs/merge/key-value-pairs.md @@ -3,46 +3,36 @@ title: "Key Value Pairs" description: "Create a dictionary from keys and values in paired columns e.g. COLUMN_NAME_1, COLUMN_VALUE_1, COLUMN_NAME_2, COLUMN_VALUE_2 ..." sidebar_label: "Key Value Pairs" slug: "/merge/key-value-pairs" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Key Value Pairs Create a dictionary from keys and values in paired columns e.g. COLUMN_NAME_1, COLUMN_VALUE_1, COLUMN_NAME_2, COLUMN_VALUE_2 ... -> Pilot Registry entry. Runtime contract status: `verified`. +Create a dictionary from keys and values in paired columns. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | object | Matched pairs of key and value columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `skip_empty` | No | boolean | Whether to skip empty keys or values when creating the dictionary. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Create a dictionary from keys and values in paired columns. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Matched pairs of key and value columns. | object | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty keys or values when creating the dictionary. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Using Named Columns +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Letter | Number | | --- | --- | @@ -66,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| Pairs | | --- | @@ -80,9 +66,9 @@ wrangles:
-#### Using a wildcard (*) -##### Recipe + + ```yaml # Using a Wildcard (*) @@ -95,9 +81,7 @@ wrangles:
-
- -##### Input Sample +
| key 1 | key 2 | value 1 | value 2 | | --- | --- | --- | --- | @@ -107,9 +91,7 @@ wrangles:
-
- -##### Output Sample +
| Object | | --- | @@ -121,17 +103,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.key_value_pairs` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.key_value_pairs` | + +**Sources** - [WranglesPY merge.key_value_pairs implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.key_value_pairs Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/key-value-pairs.md) -## Registry metadata - -- Registry ID: `8a2cd37c-8ef7-4b05-8264-36512f5dd837` -- Namespace: `merge` -- Recipe key: `merge.key_value_pairs` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.key_value_pairs` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/lists.md b/wrangles-docs/registry-docs/merge/lists.md index 11adfec..142eacd 100644 --- a/wrangles-docs/registry-docs/merge/lists.md +++ b/wrangles-docs/registry-docs/merge/lists.md @@ -3,48 +3,38 @@ title: "Lists" description: "Take lists in multiple columns and merge them to a single list." sidebar_label: "Lists" slug: "/merge/lists" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Lists Take lists in multiple columns and merge them to a single list. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `remove_duplicates` | No | boolean | Whether to remove duplicates from the created list. | `false` | -| `ignore_case` | No | boolean | Ignore case when removing duplicates. | `false` | -| `include_empty` | No | boolean | Whether to include empty values in the created list. | `true` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Take lists in multiple columns and merge them to a single list. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `remove_duplicates` | Whether to remove duplicates from the created list. | boolean | `false` | No | +| `ignore_case` | Ignore case when removing duplicates. | boolean | `false` | No | +| `include_empty` | Whether to include empty values in the created list. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Merging Two Lists +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,9 +48,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -68,9 +56,7 @@ wrangles:
-
- -##### Output Sample +
| Combined Col | | --- | @@ -80,17 +66,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.lists` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.lists` | + +**Sources** - [WranglesPY merge.lists implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.lists Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/lists.md) -## Registry metadata - -- Registry ID: `d9978f00-b3d4-4583-884e-a53b98a43e9a` -- Namespace: `merge` -- Recipe key: `merge.lists` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.lists` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/to-dict.md b/wrangles-docs/registry-docs/merge/to-dict.md index 5ec8452..8082cae 100644 --- a/wrangles-docs/registry-docs/merge/to-dict.md +++ b/wrangles-docs/registry-docs/merge/to-dict.md @@ -3,46 +3,36 @@ title: "To Dict" description: "Take multiple columns and merge them to a dictionary (aka object) using the column headers as keys." sidebar_label: "To Dict" slug: "/merge/to-dict" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # To Dict Take multiple columns and merge them to a dictionary (aka object) using the column headers as keys. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `include_empty` | No | boolean | Whether to include empty columns in the created dictionary. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Take multiple columns and merge them to a dictionary (aka object) using the column headers as keys. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `include_empty` | Whether to include empty columns in the created dictionary. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Merging Two Columns Into a Dictionary +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -65,9 +53,7 @@ wrangles:
-
- -##### Output Sample +
| Dict Col | | --- | @@ -77,17 +63,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.to_dict` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.to_dict` | + +**Sources** - [WranglesPY merge.to_dict implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.to_dict Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/to-dict.md) -## Registry metadata - -- Registry ID: `9b869210-0d89-403b-8409-7cecdb5f9c7c` -- Namespace: `merge` -- Recipe key: `merge.to_dict` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.to_dict` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/merge/to-list.md b/wrangles-docs/registry-docs/merge/to-list.md index 169d847..c18aa64 100644 --- a/wrangles-docs/registry-docs/merge/to-list.md +++ b/wrangles-docs/registry-docs/merge/to-list.md @@ -3,46 +3,36 @@ title: "To List" description: "Take multiple columns and merge them to a list." sidebar_label: "To List" slug: "/merge/to-list" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # To List Take multiple columns and merge them to a list. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of input columns. | — | -| `output` | Yes | string | Name of the output column. | — | -| `include_empty` | No | boolean | Whether to include empty columns in the created list. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Take multiple columns and merge them to a list. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `include_empty` | Whether to include empty columns in the created list. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Merging Multiple Columns to a Single List +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +46,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -66,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| List Col | | --- | @@ -78,17 +64,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.to_list` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.to_list` | + +**Sources** - [WranglesPY merge.to_list implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) - [Existing merge.to_list Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/to-list.md) -## Registry metadata - -- Registry ID: `d5300fe7-c8a4-4a41-8f12-f2c1698678cc` -- Namespace: `merge` -- Recipe key: `merge.to_list` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.merge.to_list` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/namespaces/ai.md b/wrangles-docs/registry-docs/namespaces/ai.md new file mode 100644 index 0000000..56a8fcd --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/ai.md @@ -0,0 +1,207 @@ +--- +title: "AI Wrangles" +description: "AI wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "AI" +slug: "/namespaces/ai" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# AI Wrangles + +AI wrangles, with recipe examples, parameters, and behavior. + +## Classify {#classify} + +Run classify wrangles on the specified columns. Requires WrangleWorks Account and Subscription. + +Run a custom classification wrangle on the specified column or columns. A classification wrangle must be trained first. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Formatting | | | | | +| `include_confidence` | For models that support it, include the confidence level in the output. | boolean | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | ID of the classification model to be used. | string | — | Yes | + +
+ +### Examples + +```yaml +wrangles: + - classify: + input: Products + output: Category + model_id: ${model_id} + where: Products = Milk +``` + +
+ +
+ +| Products | +| --- | +| Rice | +| Milk | + +
+ +
+ +| Category | +| --- | +| | +| Dairy | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `classify` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `ai` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.classify` | + +**Sources** + +- [WranglesPY classify implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing classify Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/classify.md) + +
+ + +--- + +## Huggingface {#huggingface} + +Use a model from huggingface. + +Use a model from huggingface + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If not provided, will overwrite the input column. | string, array, null | `null` | No | +| Options | | | | | +| `parameters` | Optionally, provide additional parameters to define the model behaviour. | object, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `api_token` | Huggingface API Token. | string | — | Yes | +| `model` | Name of the model to use. e.g. facebook/bart-large-cnn. | string | — | Yes | + +
+ +### Examples + +This template uses a Hugging Face summarization model. The exact response shape and text depend on the selected model. + + + +```yaml +wrangles: + - huggingface: + input: + - Product Description + output: + - Summary + api_token: Your Hugging Face API token + model: facebook/bart-large-cnn +``` + +
+ +
+ +| Product Description | +| --- | +| A cordless drill with two batteries, a charger, and a compact carrying case. | + +
+ +
+ +| Summary | +| --- | +| Cordless drill kit with batteries, charger, and case. | + +
+ +
+ +_Template based on the documented parameters; no published source example is currently available._ + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `huggingface` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `ai` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.huggingface` | + +**Sources** + +- [WranglesPY huggingface implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing huggingface Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/huggingface.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/compare.md b/wrangles-docs/registry-docs/namespaces/compare.md new file mode 100644 index 0000000..2160857 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/compare.md @@ -0,0 +1,475 @@ +--- +title: "Compare Wrangles" +description: "Compare wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Compare" +slug: "/namespaces/compare" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Compare Wrangles + +Compare wrangles, with recipe examples, parameters, and behavior. + +## Lists {#lists} + +Compare multiple lists and return the intersection, difference, or union. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns containing lists to compare. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `method` | Type of comparison to perform. | string; one of:
  • intersection
  • difference
  • union
| `"intersection"` | No | +| `remove_duplicates` | Remove duplicates from the result. | boolean | `false` | No | +| `ignore_case` | Ignore case when comparing string items. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - compare.lists: + input: + - col1 + - col2 + output: Difference + method: difference +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| [A, B, C] | [C, D, E] | +| [H, I, J, K] | [H, I, J] | +| [X, Y, Z] | [1, 2, 3] | + +
+ +
+ +| Difference | +| --- | +| [A, B] | +| [K] | +| [X, Y, Z] | + +
+ +
+ + + + + +```yaml +wrangles: + - compare.lists: + input: + - col1 + - col2 + output: Intersection + method: intersection +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| [A, B, C] | [C, D, E] | +| [H, I, J, K] | [H, I, J] | +| [X, Y, Z] | [1, 2, 3] | + +
+ +
+ +| Intersection | +| --- | +| [C] | +| [H, I, J] | +| [] | + +
+ +
+ + + + + +```yaml +wrangles: + - compare.lists: + input: + - col1 + - col2 + output: Union + method: union +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| [A, B, C] | [C, D, E] | +| [H, I, J, K] | [H, I, J] | +| [X, Y, Z] | [1, 2, 3] | + +
+ +
+ +| Union | +| --- | +| [A, B, C, D, E] | +| [H, I, J, K] | +| [X, Y, Z, 1, 2, 3] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compare.lists` | +| Lifecycle status | active | +| Namespace | `compare` | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compare.lists` | + +**Sources** + +- [WranglesPY compare.lists implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compare.py) +- [Existing compare.lists Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/lists.md) + +
+ + +--- + +## Text {#text} + +Compare two strings and return the intersection or difference, use overlap to find the matching characters between the two strings, or use similarity to get a numeric similarity score. + +Compare two strings and return the intersection or difference, or use overlap to find the matching characters between the two strings. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The columns to compare. First column is the base column. | array | — | Yes | +| `output` | The column to output the results to. Must be a list of two column names [mask_column, ratio_column] when method is overlap and include_ratio is true; otherwise a single column name. | string, array | — | Yes | +| Options | | | | | +| `method` | The type of comparison to perform (difference, intersection, overlap, similarity). | string; one of:
  • difference
  • intersection
  • overlap
  • similarity
| `"difference"` | No | +| `char` | Character to split strings on for difference and intersection. Defaults to a space. | string | `" "` | No | +| `exact_match` | Value to use for exact matches when using overlap. | string, null | `null` | No | +| `empty_a` | Value to use when input A is empty when using overlap. | string, null | `null` | No | +| `empty_b` | Value to use when input B is empty when using overlap. | string, null | `null` | No | +| `all_empty` | Value to use when both inputs are empty when using overlap. | string, null | `null` | No | +| `case_sensitive` | Whether the comparison is case sensitive. Defaults to true. | boolean | `false` | No | +| `metric` | Metric value accepted by the runtime. | string | `"token_sort"` | No | +| Formatting | | | | | +| `non_match_char` | Character to use for non-matching characters when using overlap. | string | `"*"` | No | +| `include_ratio` | Include the ratio of matching characters when using overlap. | boolean | `false` | No | +| `decimal_places` | Number of decimal places to round the overlap ratio to. | integer | `3` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - compare.text: + input: + - col1 + - col2 + output: Difference + method: difference +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| Large Oak Wood White Marble Top Bookshelf | Large Pine Wood Black Marble Bottom Bookshelf | +| Medium Oak Wood White Marble Top Coffee Table | Medium Maple Wood Orange Steel Top Coffee Table | +| Small Oak Wood White Marble Top Console Table | Small Normal Wood Blue Plastic Top Console Table | + +
+ +
+ +| Difference | +| --- | +| Pine Black Bottom | +| Maple Orange Steel | +| Normal Blue Plastic | + +
+ +
+ + + + + +```yaml +wrangles: + - compare.text: + input: + - col1 + - col2 + output: Intersection + method: intersection +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| Large Oak Wood White Marble Top Bookshelf | Large Pine Wood Black Marble Bottom Bookshelf | +| Medium Oak Wood White Marble Top Coffee Table | Medium Maple Wood Orange Steel Top Coffee Table | +| Small Oak Wood White Marble Top Console Table | Small Normal Wood Blue Plastic Top Console Table | + +
+ +
+ +| Intersection | +| --- | +| Large Wood Marble Bookshelf | +| Medium Wood Top Coffee Table | +| Small Wood Top Console Table | + +
+ +
+ + + + + +```yaml +wrangles: + - compare.text: + input: + - Part Code1 + - Part Code2 + output: Overlap + method: overlap +``` + +
+ +
+ +| Part Code1 | Part Code2 | +| --- | --- | +| SKF6202 | TMKN6202 | +| X06-02-000 | X06-81-000 | +| 7100E15-V-230/3 | 7100E15-V-120/1 | + +
+ +
+ +| Overlap | +| --- | +| ****6202 | +| X06-**-000 | +| 7100E15-V-\**0/\* | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compare.text` | +| Lifecycle status | active | +| Namespace | `compare` | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compare.text` | + +**Sources** + +- [WranglesPY compare.text implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compare.py) +- [Existing compare.text Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/text.md) + +
+ + +--- + +## Similarity {#similarity} + +Calculate the cosine similarity of two vectors. + +Calculate the similarity of two vectors. + +:::info +Similarity only works on vectors. To produce vectors from a column of strings, use `create.embeddings` first. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Two columns of vectors to compare the similarity of. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `method` | The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons. | string; one of:
  • cosine
  • adjusted cosine
  • euclidean
| `"cosine"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.embeddings: + input: col1 + api_key: ${my_key} + output: col1 embeddings + + - create.embeddings: + input: col2 + api_key: ${my_key} + output: col2 embeddings + + - similarity: + input: + - col1 embeddings + - col2 embeddings + output: similarity + method: adjusted cosine +``` + +
+ +
+ +| col1 | col2 | +| --- | --- | +| SKF | Timken | +| Ball Bearing | Roller Bearing | + +
+ +
+ +| col1 embeddings | col2 embeddings | similarity | +| --- | --- | --- | +| [1, 2, 3, 4] | [4, 3, 2, 1] | 0.158931 | +| [5, 6, 7, 8] | [5, 6, 7, 9] | 0.942437 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `similarity` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.similarity` | + +**Sources** + +- [WranglesPY similarity implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing similarity Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/similarity.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/compute.md b/wrangles-docs/registry-docs/namespaces/compute.md new file mode 100644 index 0000000..17091cc --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/compute.md @@ -0,0 +1,661 @@ +--- +title: "Compute Wrangles" +description: "Compute wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Compute" +slug: "/namespaces/compute" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Compute Wrangles + +Compute wrangles, with recipe examples, parameters, and behavior. + +## Case When {#case-when} + +Assign values to a column based on conditional logic. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `cases` | List of conditions and corresponding values. | array | — | Yes | +| `default` | Value to assign if no conditions are met. Default None. | string, number, integer, boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - compute.case_when: + output: Letter Grade + cases: + - condition: Grade > 89 + value: 'A' + - condition: 90 > Grade > 79 + value: 'B' + - condition: 80 > Grade > 69 + value: 'C' + - condition: 70 > Grade + value: 'F' +``` + +
+ +
+ +| Student | Grade | +| --- | --- | +| Billy | 62 | +| Sarah | 91 | +| Timmy | 88 | +| Tammy | 74 | + +
+ +
+ +| Letter Grade | +| --- | +| F | +| A | +| B | +| C | + +
+ +
+ + + +Conditions can be combined so multiple criteria must be met before assigning a value. + + + +```yaml +wrangles: + - compute.case_when: + output: Letter Grade + cases: + - condition: (Grade > .89) & (Attendance == 'Good') + value: 'A' + - condition: (.90 > Grade > .79) & (Attendance == 'Good') + value: 'B' + - condition: (.80 > Grade > .69) & (Attendance == 'Good') + value: 'C' + - condition: (.70 > Grade) or (Attendance == 'Poor') + value: 'F' +``` + +
+ +
+ +| Student | Grade | Attendance | +| --- | --- | --- | +| Billy | 62 | Poor | +| Sarah | 91 | Poor | +| Timmy | 88 | Good | +| Tammy | 74 | Good | + +
+ +
+ +| Letter Grade | +| --- | +| F | +| F | +| B | +| C | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compute.case_when` | +| Lifecycle status | active | +| Namespace | `compute` | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compute.case_when` | + +**Sources** + +- [WranglesPY compute.case_when implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compute.py) +- [Existing compute.case_when Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/case-when.md) + +
+ + +--- + +## Score Search Results {#score-search-results} + +Scores and filters search results based on progressive partial/exact matching. Can return dictionaries or a parallel list of formatted strings. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)]. | array | — | Yes | +| `output` | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | string, array | — | Yes | +| Options | | | | | +| `must_match_part_code` | If true, filters out results that don't satisfy the allowed match types. | boolean | `true` | No | +| `allow_mpn_exact` | Treat exact MPN matches as valid part code matches. | boolean | `true` | No | +| `allow_mpn_partial` | Treat partial MPN matches as valid part code matches. | boolean | `true` | No | +| `allow_other_exact` | Treat exact other part code matches as valid part code matches. | boolean | `true` | No | +| `allow_other_partial` | Treat partial other part code matches as valid part code matches. | boolean | `true` | No | +| `blacklist_keywords` | Comma-separated list or array of keywords to filter out URLs containing them. | string | `""` | No | +| `mpn_exact_score` | Mpn Exact Score value accepted by the runtime. | number | `8` | No | +| `mpn_partial_base` | Mpn Partial Base value accepted by the runtime. | number | `4` | No | +| `part_code_exact_score` | Part Code Exact Score value accepted by the runtime. | number | `6` | No | +| `part_code_partial_base` | Part Code Partial Base value accepted by the runtime. | number | `2` | No | +| `supplier_exact_score` | Supplier Exact Score value accepted by the runtime. | number | `3` | No | +| `supplier_partial_base` | Supplier Partial Base value accepted by the runtime. | number | `1` | No | +| `context_match_base` | Context Match Base value accepted by the runtime. | number | `2` | No | +| `fuzzy_match_threshold` | Fuzzy Match Threshold value accepted by the runtime. | number | `0.8` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +This template scores search results using supplier and part-code context. The fields added to each result dictionary depend on the scoring configuration. + + + +```yaml +wrangles: + - compute.score_search_results: + input: + - Search Results + - Suppliers + - Part Codes + output: + - Scored Results + allow_other_exact: true + must_match_part_code: true +``` + +
+ +
+ +| Search Results | Suppliers | Part Codes | +| --- | --- | --- | +| `[{"title": "SKF 6202 bearing", "link": "https://example.com/6202"}]` | `["SKF"]` | `["6202"]` | + +
+ +
+ +| Scored Results | +| --- | +| Ranked result dictionaries | + +
+ +
+ +_Template based on the documented parameters; no published source example is currently available._ + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `compute.score_search_results` | +| Lifecycle status | active | +| Namespace | `compute` | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.compute.score_search_results` | + +**Sources** + +- [WranglesPY compute.score_search_results implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/compute.py) +- [Existing compute.score_search_results Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/score-search-results.md) + +
+ + +--- + +## Math {#math} + +Apply a mathematical calculation. + +Apply mathematical calculations to columns. Also called as `maths`. + +:::info +Spaces within column headers are replaced with underscores automatically. Account for this when writing expressions in `input`. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_). | string | — | Yes | +| `output` | The column to output the results to. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - math: + input: sqrt(Values) + output: Square Root +``` + +
+ +
+ +| Values | +| --- | +| 4 | +| 9 | +| 16 | + +
+ +
+ +| Square Root | +| --- | +| 2 | +| 3 | +| 4 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `math` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.math` | + +**Sources** + +- [WranglesPY math implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing math Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/math.md) + +
+ + +--- + +## Python {#python} + +Apply a simple single-line python command. For more complex python use a custom function. Note, this evaluates the python command - be especially cautious including variables from untrusted sources within the command string. The python command will be evaluated once for each row and the result returned. Reference column values by using their name. Non-alphanumeric characters within column names are replaced by underscores (_) Additionally, all columns are available as a dict named kwargs. Additional parameters set for the wrangle will also be available to the command. + +The Python wrangle executes simple Python commands inline within a recipe. Row values are referenced by column name and commands are evaluated once per row. Spaces within column names are replaced by underscores (`_`). All columns are also available as a dictionary named `kwargs`. For more complex Python, use custom functions. + +:::caution +This wrangle evaluates the Python command. Be cautious when including variables from untrusted sources in the command string. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of output column(s). To output multiple columns, return a list of the corresponding length. | string, array | — | Yes | +| `input` | Name or list of input column(s) to filter the data available to the command. Useful in conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available. | string, integer, array, null | `null` | No | +| Options | | | | | +| `command` | Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_). | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `except` | Value to return for the row if an exception occurs during the evaluation. If not provided, an exception will be raised as normal. If multiple output columns are specified, this must match the length. | string, array, number, integer, boolean, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - python: + output: result + command: My_Column.upper() +``` + +
+ +
+ +| My Column | +| --- | +| example text | + +
+ +
+ +| result | +| --- | +| EXAMPLE TEXT | + +
+ +
+ + + +The Python wrangle supports parameters so values of unknown origin can be injected safely. + + + +```yaml +wrangles: + - python: + output: sliced + command: input_column[:i] + i: ${var} +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +_No sample available._ + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `python` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.python` | + +**Sources** + +- [WranglesPY python implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing python Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/python.md) + +
+ + +--- + +## SQL {#sql} + +Apply a SQL command to the current dataframe. Only SELECT statements are supported - the result will be the output. + +Apply a SQL command to the current dataframe. Only `SELECT` statements are supported; the result becomes the output. The current table is called `df`. + +:::info +SQL does not currently work with objects. If your table contains objects, use `convert.to_json` before using SQL. SQL is not compatible with `where` filtering. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `command` | SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect. | string | — | Yes | +| `params` | Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name). | array, object, null | `null` | No | +| `preserve_data_types` | Preserve Data Types value accepted by the runtime. | boolean | `true` | No | +| Formatting | | | | | +| `preserve_index` | Preserve Index value accepted by the runtime. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - sql: + command: | + SELECT header1, header2 + FROM df + WHERE header1 >= 2 +``` + +
+ +
+ +| header1 | header2 | header3 | +| --- | --- | --- | +| 1 | a | x | +| 2 | b | y | +| 3 | c | z | + +
+ +
+ +| header1 | header2 | +| --- | --- | +| 2 | b | +| 3 | c | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `sql` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.sql` | + +**Sources** + +- [WranglesPY sql implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing sql Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/sql.md) + +
+ + +--- + +## Maths (Deprecated) {#maths} + +:::warning Deprecated +This compatibility wrangle remains available for existing recipes. Use [`math`](/wrangles/namespaces/compute#math) for new recipes. +::: + +Deprecated alias for `math`; evaluate an expression and write its result to an output column. + +### Behavior + +Deprecated alias for `math`; evaluate an expression and write its result to an output column. + +This guidance was derived from the callable signature. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string | — | Yes | +| `output` | Name or list of output columns. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ + + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `maths` | +| Lifecycle status | deprecated | +| Replaced by | [`math`](/wrangles/namespaces/compute#math) | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.maths` | + +**Sources** + +- [WranglesPY maths implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) + +
diff --git a/wrangles-docs/registry-docs/namespaces/convert.md b/wrangles-docs/registry-docs/namespaces/convert.md new file mode 100644 index 0000000..cd8e28c --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/convert.md @@ -0,0 +1,698 @@ +--- +title: "Convert Wrangles" +description: "Convert wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Convert" +slug: "/namespaces/convert" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Convert Wrangles + +Convert wrangles, with recipe examples, parameters, and behavior. + +## Convert Case {#case} + +Change the letter case of text values. + +Use `convert.case` to normalize capitalization while keeping the source column +or writing the result to a new column. + +### Behavior + +- Supported modes are `lower`, `upper`, `title`, and `sentence`. +- Omitting `output` overwrites the input column. +- Input and output lists must have equal lengths. +- Non-string values are passed through unchanged and produce a warning. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Options | | | | | +| `case` | Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts. | string; one of:
  • lower
  • upper
  • title
  • sentence
| `"lower"` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.case: + input: product + output: product_upper + case: upper +``` + +
+ +
+ +| product | +| --- | +| Cordless Drill | +| bearing seal | + +
+ +
+ +| product_upper | +| --- | +| CORDLESS DRILL | +| BEARING SEAL | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.case` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.case` | + +**Sources** + +- [WranglesPY convert.case implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Legacy convert.case documentation](https://wrangles.io/python/recipes/wrangles/convert#case) + +
+ + +--- + +## Convert Data Type {#data-type} + +Convert values to strings, numbers, booleans, or datetimes. + +Use `convert.data_type` when a recipe needs consistent Python-compatible +values rather than display-only formatting. + +### Behavior + +- Supported target types are `str`, `float`, `int`, `bool`, and `datetime`. +- Omitting `data_type` uses the runtime default `str`. +- Omitting `output` overwrites the input column. +- Failed conversions retain the original value unless `default` is supplied. +- Additional undocumented keyword arguments are implementation details and are + not part of the public Registry contract. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Options | | | | | +| `data_type` | Data type to produce. | string; one of:
  • str
  • float
  • int
  • bool
  • datetime
| `"str"` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Errors | | | | | +| `default` | Value returned when conversion fails. If omitted, the original value is retained. | string, number, array, object, boolean, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.data_type: + input: quantity + output: quantity_integer + data_type: int + default: 0 +``` + +
+ +
+ +| quantity | +| --- | +| 12 | +| not available | + +
+ +
+ +| quantity_integer | +| --- | +| 12 | +| 0 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.data_type` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.data_type` | + +**Sources** + +- [WranglesPY convert.data_type implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Legacy convert.data_type documentation](https://wrangles.io/python/recipes/wrangles/convert#data-type) + +
+ + +--- + +## Fraction to Decimal {#fraction-to-decimal} + +Convert fractions to decimals. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output colum. | string, array, null | `null` | No | +| Formatting | | | | | +| `decimals` | Number of decimals to round fraction. | integer | `4` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.fraction_to_decimal: + input: fractions + output: decimals + decimals: 3 +``` + +
+ +
+ +| fractions | +| --- | +| 3/32 | +| 25/64 | + +
+ +
+ +| decimals | +| --- | +| 0.094 | +| 0.391 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.fraction_to_decimal` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.fraction_to_decimal` | + +**Sources** + +- [WranglesPY convert.fraction_to_decimal implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Existing convert.fraction_to_decimal Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/fraction-to-decimal.md) + +
+ + +--- + +## Convert From JSON {#from-json} + +Parse JSON text into lists, objects, scalars, booleans, or null values. + +Use `convert.from_json` when a column contains JSON text that later wrangles +need to treat as structured values. + +### Behavior + +- Each non-fallback value is parsed with Python's JSON parser. +- Omitting `output` overwrites the input column. +- Multiple input columns may share one fallback or use one fallback per input. +- Invalid JSON raises an error unless a non-null fallback is supplied. +- Permissive Python-literal or YAML-like parsing is outside this wrangle's + contract. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of columns containing valid JSON text. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. If omitted, each input column is overwritten. | string, array | `null` | No | +| Conditions | | | | | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Errors | | | | | +| `default` | Value returned for empty or invalid JSON. A list may supply one fallback per input column. | string, number, array, object, boolean, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.from_json: + input: attributes_json + output: attributes +``` + +
+ +
+ +| attributes_json | +| --- | +| {"material":"steel","voltage":18} | +| ["corded","variable speed"] | + +
+ +
+ +| attributes | +| --- | +| {"material":"steel","voltage":18} | +| ["corded","variable speed"] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.from_json` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.from_json` | + +**Sources** + +- [WranglesPY convert.from_json implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Legacy convert.from_json documentation](https://wrangles.io/python/recipes/wrangles/convert#from-json) + +
+ + +--- + +## From YAML {#from-yaml} + +Convert a YAML representation into an object. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Value to return if the row is empty or fails to be parsed as YAML. If input is a list, default may also be a list - either a single value to apply to all columns, or one value per input column. | string, array, object, number, boolean, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.from_yaml: + input: column 1 +``` + +
+ +
+ +```yaml +Product Specs: + length: 6 inch + voltage: 24V + weight: 3lb +``` + +
+ +
+ +```python +{ + 'Product Specs': { + 'length': '6 inch', + 'voltage': '24V', + 'weight': '3lb' + } +} +``` + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.from_yaml` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.from_yaml` | + +**Sources** + +- [WranglesPY convert.from_yaml implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Existing convert.from_yaml Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/from-yaml.md) + +
+ + +--- + +## To JSON {#to-json} + +Convert an object to a JSON representation. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Formatting | | | | | +| `ensure_ascii` | If true, non-ASCII characters will be escaped. Default is false. | boolean | `false` | No | +| `indent` | If indent is a non-negative integer or string, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, negative, or "" will only insert newlines. None (the default) selects the most compact representation. Using a positive integer indent indents that many spaces per level. If indent is a string (such as '\t'), that string is used to indent each level. | string, integer | — | No | +| `sort_keys` | If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key. | boolean | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.to_json: + input: column + output: new column +``` + +
+ +
+ +| column | +| --- | +| `['a', 'python', 'list']` | +| `{'python': 'dict'}` | + +
+ +
+ +| new column | +| --- | +| `["a","python","list"]` | +| `{"python":"dict"}` | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.to_json` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.to_json` | + +**Sources** + +- [WranglesPY convert.to_json implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Existing convert.to_json Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/to-json.md) + +
+ + +--- + +## To YAML {#to-yaml} + +Convert an object to a YAML representation. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be overwritten. | string, array, null | `null` | No | +| Formatting | | | | | +| `sort_keys` | If sort_keys is true (default: False), then the output of dictionaries will be sorted by key. | boolean | `false` | No | +| `allow_unicode` | Allow Unicode value accepted by the runtime. | boolean | `true` | No | +| `indent` | Specify the number of spaces for indentation to specify nested elements. | integer | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - convert.to_yaml: + input: column 1 + indent: 2 +``` + +
+ +
+ +```python +{ + 'Product Specs': { + 'length': '6 inch', + 'voltage': '24V', + 'weight': '3lb' + } +} +``` + +
+ +
+ +```yaml +Product Specs: + length: 6 inch + voltage: 24V + weight: 3lb +``` + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `convert.to_yaml` | +| Lifecycle status | active | +| Namespace | `convert` | +| Documentation group | `convert` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.convert.to_yaml` | + +**Sources** + +- [WranglesPY convert.to_yaml implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/convert.py) +- [Existing convert.to_yaml Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/convert/_sources/to-yaml.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/create.md b/wrangles-docs/registry-docs/namespaces/create.md new file mode 100644 index 0000000..ba2393d --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/create.md @@ -0,0 +1,901 @@ +--- +title: "Create Wrangles" +description: "Create wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Create" +slug: "/namespaces/create" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Create Wrangles + +Create wrangles, with recipe examples, parameters, and behavior. + +## Bins {#bins} + +Create a column that groups data into bins. + +Creates a column that segments and sorts data values into bins. `bins` can be an integer or a list. When `bins` is an integer, the input data is split equally into that number of bins. When `bins` is a list, the input data is split based on the list boundaries. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of input column. | string, integer, array | — | Yes | +| `output` | Name of new column. | string, array | — | Yes | +| Options | | | | | +| `bins` | Defines the number of equal-width bins in the range. | integer, array | — | Yes | +| `labels` | Labels for the returned bins. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.bins: + input: Data + output: Category + bins: 3 + labels: + - Bad + - Medium + - Good +``` + +
+ +
+ +| Data | +| --- | +| 1 | +| 7 | +| 5 | +| 4 | +| 6 | +| 3 | + +
+ +
+ +| Category | +| --- | +| Bad | +| Good | +| Medium | +| Medium | +| Good | +| Bad | + +
+ +
+ + + + + +```yaml +wrangles: + - create.bins: + input: Grades + output: Letter Grade + bins: + - 0 + - 60 + - 70 + - 80 + - 90 + - 100 + labels: + - F + - D + - C + - B + - A +``` + +
+ +
+ +| Grade | Student | +| --- | --- | +| 64 | Charles | +| 92 | Sabrina | +| 76 | Edward | +| 84 | Wendy | + +
+ +
+ +| Letter Grade | +| --- | +| D | +| A | +| C | +| B | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.bins` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.bins` | + +**Sources** + +- [WranglesPY create.bins implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.bins Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/bins.md) + +
+ + +--- + +## Column {#column} + +Create column(s) with a user defined value. Defaults to None (empty). + +Create column(s) with a user defined value. Defaults to `None` (empty). If you need to copy an existing column, use the copy wrangle instead. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns or column_name: value pairs. | string, array | — | Yes | +| Options | | | | | +| `value` | (Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string. | string, number, object, array, boolean, null | `null` | No | +| `value_if_exists` | Determines behaviour when the output column already exists. existing (default): leave the column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value. | string; one of:
  • existing
  • coalesce
  • new
| `"existing"` | No | +| `coalesce_value` | Only used when value_if_exists is coalesce. Determines which side is preferred when both the existing and new values are non-empty. existing (default): keep the existing value, only fill empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null. | string; one of:
  • existing
  • new
| `"existing"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.column: + output: New Column + value: new value # Optional, otherwise empty + where: column > 1 +``` + +
+ +
+ +| column | +| --- | +| 1 | +| 2 | +| 3 | + +
+ +
+ +| New Column | +| --- | +| | +| new value | +| new value | + +
+ +
+ + + + + +```yaml +wrangles: + - create.column: + output: + - New Column 1: new value 1 # Optional, otherwise empty + - New Column 2: new value 2 + - New Column 3: new value 1 +``` + +
+ +
+ +| column | +| --- | +| 1 | +| 2 | +| 3 | + +
+ +
+ +| New Column 1 | New Column 2 | New Column 3 | +| --- | --- | --- | +| new value 1 | new value 2 | new value 1 | +| new value 1 | new value 2 | new value 1 | +| new value 1 | new value 2 | new value 1 | + +
+ +
+ + + + + +```yaml +wrangles: + - create.column: + output: + - New Column: + - 4 + - 5 + - 6 +``` + +
+ +
+ +| column | +| --- | +| 1 | +| 2 | +| 3 | + +
+ +
+ +| New Column | +| --- | +| [4, 5, 6] | +| [4, 5, 6] | +| [4, 5, 6] | + +
+ +
+ +Columns of empty lists can also be created by passing an empty list (`[]`) as the column value. + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.column` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.column` | + +**Sources** + +- [WranglesPY create.column implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.column Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/column.md) + +
+ + +--- + +## Embeddings {#embeddings} + +Create an embedding based on text input. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The column of text to create the embeddings for. | string | — | Yes | +| `output` | The output column the embeddings will be saved as. | string, null | `null` | No | +| Options | | | | | +| `task` | The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter. | string, null; one of:
  • retrieval.query
  • retrieval.passage
  • text-matching
  • classification
  • separation
| `null` | No | +| Formatting | | | | | +| `output_type` | Output the embeddings as a numpy array or a python list Default - python list. | string; one of:
  • numpy array
  • python list
| `"python list"` | No | +| `precision` | The precision of the embeddings. Default is float32. This should be used with output_type numpy array. | string; one of:
  • float16
  • float32
| `"float32"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `batch_size` | The number of rows to submit per individual request. | integer | `100` | No | +| `threads` | The number of requests to submit in parallel. Each request contains the number of rows set as batch_size. | integer | `10` | No | +| Errors | | | | | +| `retries` | The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting. | integer | `0` | No | +| Details | | | | | +| `api_key` | The API key. | string | — | Yes | +| `model` | The specific model to use to generate the embeddings. | string | `"text-embedding-3-small"` | No | +| `url` | The endpoint to send embedding requests to. Defaults to the standard endpoint for the resolved provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format. | string | `"https://api.openai.com/v1/embeddings"` | No | +| `provider` | Controls the request/response format for the embedding API. When omitted, inferred from url (jina.ai → jina, otherwise openai). Setting provider also sets the default url for that provider, so you only need one of provider or url for standard endpoints. Use both together only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy). | string, null; one of:
  • openai
  • jina
| `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - create.embeddings: + input: my_column + api_key: ${my_key} + output: embeddings +``` + +
+ +
+ +| my_column | +| --- | +| angle grinder | +| jig saw | + +
+ +
+ +| embeddings | +| --- | +| [0.010793785, -0.010007165, 0.0028609, -0.0139...] | +| [-0.008975127, 0.009314879, -0.024150735, -0.0...] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.embeddings` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.embeddings` | + +**Sources** + +- [WranglesPY create.embeddings implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.embeddings Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/embeddings.md) + +
+ + +--- + +## GUID {#guid} + +Create column(s) with a GUID. + +Create a column with a GUID (Globally Unique Identifier). + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.guid: + output: GUID Column + + # OR + + - create.uuid: + output: GUID Column +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| GUID Column | +| --- | +| 9a13b4dd-1993-4bd2-8974-3f9a0f006d86 | +| 1528d918-ecd7-457c-b44d-3d30705696d3 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.guid` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.guid` | + +**Sources** + +- [WranglesPY create.guid implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.guid Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/guid.md) + +
+ + +--- + +## Hash {#hash} + +Create a hash of a column. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of input column. | string, integer, array | — | Yes | +| `output` | Name of new column. | string, array | — | Yes | +| Options | | | | | +| `method` | The method to use to hash the input (Default: md5). | string; one of:
  • md5
  • sha1
  • sha256
  • sha512
| `"md5"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.hash: + input: Description + output: hash + method: md5 +``` + +
+ +
+ +| Description | +| --- | +| The wrench is blue | +| The hammer is yellow | + +
+ +
+ +| New Column | +| --- | +| ce114e4501d2f4e2dcea3e17b546f339 | +| a54d88e06612d820bc3be72877c74f257b561b19 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.hash` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.hash` | + +**Sources** + +- [WranglesPY create.hash implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.hash Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/hash.md) + +
+ + +--- + +## Index {#index} + +Create column(s) with an incremental index. e.g. 1,2,3... + +Create a new incremental index. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| `by` | Optional. Cluster the created indexes by one or more columns. | string, array, null | `null` | No | +| Options | | | | | +| `start` | (Optional; default 1) Starting number for the index. | integer | `1` | No | +| `step` | (Optional; default 1) Step between successive rows. | integer | `1` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.index: + output: New Index + start: 1 # optional + step: 1 # optional +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| New Index | +| --- | +| 1 | +| 2 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.index` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.index` | + +**Sources** + +- [WranglesPY create.index implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.index Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/index.md) + +
+ + +--- + +## Jinja {#jinja} + +Output text using a jinja template. + +Makes use of a Jinja template to create a description, title, or summary based on your data. + +:::info +Jinja templates do not allow variables with spaces. This wrangle automatically replaces spaces in column headers with underscores, so use underscores instead of spaces when referencing columns in the template. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name of the column to be output to. | array | — | Yes | +| `input` | Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys. | string, null | `null` | No | +| Options | | | | | +| `template` | A dictionary which defines the template/location as well as the form which the template is input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_). | object | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.jinja: + output: Description + template: + string: | + This is a {{ Brand }} {{ Item_Type }} that is {{ Size }} +``` + +
+ +
+ +| Size | Brand | Item Type | +| --- | --- | --- | +| 10mm | SKF | ball bearing | +| 15mm | Timken | bearing seal | + +
+ +
+ +| Description | +| --- | +| This is a SKF ball bearing that is 10mm | +| This is a Timken bearing seal that is 15mm | + +
+ +
+ +Using `|` in YAML denotes a multi-line string that preserves line breaks. Use `>` for a multi-line string when line breaks should not be preserved. + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.jinja` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.jinja` | + +**Sources** + +- [WranglesPY create.jinja implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.jinja Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/jinja.md) + +
+ + +--- + +## UUID {#uuid} + +Create column(s) with a UUID. + +Create a column with a UUID (Universally Unique Identifier). + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of names of new columns. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - create.guid: + output: GUID Column + + # OR + + - create.uuid: + output: GUID Column +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| GUID Column | +| --- | +| 9a13b4dd-1993-4bd2-8974-3f9a0f006d86 | +| 1528d918-ecd7-457c-b44d-3d30705696d3 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `create.uuid` | +| Lifecycle status | active | +| Namespace | `create` | +| Documentation group | `create` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.create.uuid` | + +**Sources** + +- [WranglesPY create.uuid implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/create.py) +- [Existing create.uuid Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/create/_sources/uuid.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/date.md b/wrangles-docs/registry-docs/namespaces/date.md new file mode 100644 index 0000000..9fc85cd --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/date.md @@ -0,0 +1,104 @@ +--- +title: "Date Wrangles" +description: "Date wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Date" +slug: "/namespaces/date" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Date Wrangles + +Date wrangles, with recipe examples, parameters, and behavior. + +## Date Calculator {#date-calculator} + +Add or Subtract time from a date. + +Add or subtract time from a date. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the dates column. | string | — | Yes | +| `output` | Name of the output column of dates. | string, null | `null` | No | +| Options | | | | | +| `operation` | Date operation. | string; one of:
  • add
  • subtract
| `"add"` | No | +| `time_unit` | Time unit for operation. | string, null; one of:
  • years
  • months
  • weeks
  • days
  • hours
  • minutes
  • seconds
  • milliseconds
| `null` | No | +| `time_value` | Time unit value for operation. | number, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - date_calculator: + input: Date + output: New Date + operation: subtract # Optional default is addition + time_unit: days + time_value: 1 +``` + +
+ +
+ +| Date | +| --- | +| 2022-12-26 | + +
+ +
+ +| New Date | +| --- | +| 2022-12-25 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `date_calculator` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `date` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.date_calculator` | + +**Sources** + +- [WranglesPY date_calculator implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing date_calculator Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/date/_sources/date-calculator.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/extract.md b/wrangles-docs/registry-docs/namespaces/extract.md new file mode 100644 index 0000000..b18acfe --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/extract.md @@ -0,0 +1,1444 @@ +--- +title: "Extract Wrangles" +description: "Extract wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Extract" +slug: "/namespaces/extract" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Extract Wrangles + +Extract wrangles, with recipe examples, parameters, and behavior. + +## Address {#address} + +Extract parts of addresses. Requires WrangleWorks Account. + +Extract geographical information from unstructured text such as streets, cities, or countries. Requires WrangleWorks Account. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `dataType` | Specific part of the address to extract. | string; one of:
  • streets
  • cities
  • regions
  • countries
| — | Yes | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.address: + input: Location + output: Street + dataType: streets + where: SUBSTRING(Location, 1, 3) = '221' +``` + +
+ +
+ +| Location | +| --- | +| 221 B Baker St., London, England, United Kingdom | +| London SW1A 1AA, London, England, United Kingdom | + +
+ +
+ +| Street | +| --- | +| ['221 B Baker St.'] | +| | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.address` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.address` | + +**Sources** + +- [WranglesPY extract.address implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.address Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/address.md) + +
+ + +--- + +## AI {#ai} + +Extract structured data from each input row using an AI model. Define the desired fields with output, or reuse a saved definition with model_id. + +Use AI to extract meaningful structured data. `extract.ai` can be used recipe-first, where the output schema is defined in the recipe, or model-first, where a saved extract.ai model is called by `model_id`. + +:::info +For saved extract.ai models, this is the preferred calling pattern compared with using `extract.custom`. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied. | array, null | `null` | No | +| `output` | Desired extraction. Use an object keyed by output column name for structured fields, a string for one prompted value, or an array of field names/definitions. Each field may use the schema options below. | string, array, object, null | `null` | No | +| Options | | | | | +| `record_examples` | Whole-record examples. Each example has a separate input value or record and the complete expected output record. Optional name and notes provide model-visible context. Use {name: ..., notes: ..., input: ..., output: ...}. Omitted nullable output fields are completed with null. Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field. | array, object, null | `null` | No | +| `web_search` | Enable OpenAI Responses web search; the model decides when searching helps. When true, every row also receives web_search_sources: a deduplicated list of {title, url} objects in source order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false. | boolean | `false` | No | +| `instructions` | Additional guidance applied to every input row. Use this for decision rules, evidence priorities, normalization requirements, or other behavior that applies to the complete extraction. | string, array, null | `null` | No | +| Formatting | | | | | +| `output_format` | How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char. | string, null; one of:
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Separator used only when output_format is concatenate. Defaults to comma-space. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Maximum number of row-level requests sent in parallel. The configured default is 32. | integer | — | No | +| `timeout` | Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner. | number | — | No | +| `deadline` | Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15. | number | — | No | +| Errors | | | | | +| `retries` | Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline. | integer | — | No | +| Details | | | | | +| `api_key` | OpenAI API key used for this wrangle, normally supplied through a recipe variable. | string | — | Yes | +| `model_id` | ID of a saved extract.ai definition. Use it instead of defining an output schema. When output is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields. | string, null | `null` | No | +| `model` | OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model. | string | — | No | +| `url` | Override the endpoint for the selected protocol. A chat/completions URL selects the legacy protocol only when protocol is omitted; new recipes should use the configured Responses endpoint. | string | — | No | +| `provider` | AI service provider. Currently only OpenAI is supported. | string; one of:
  • openai
| — | No | +| `protocol` | OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions. | string; one of:
  • responses
  • chat_completions
| — | No | +| `store` | Whether OpenAI may store Responses API results. Defaults to false. | boolean | — | No | +| `cache` | Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required. | boolean | — | No | +| `cache_ttl` | Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together. | number | — | No | +| `strict` | Require OpenAI structured-output strict mode. Defaults to true. Definitions with dynamic dictionary keys automatically switch to non-strict provider mode and are still validated locally. | boolean | — | No | +| `reasoning` | Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies. | object | — | No | +| `verbosity` | Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models. | string; one of:
  • low
  • medium
  • high
| — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.ai: + api_key: Your OpenAI api key + input: Product Specs + output: + Blade Diameter: + type: number + description: The diameter of the blade used, reported in inches. + default: N/A + examples: + - 4.5" + - 8 inch + Max. RPM: + type: number + description: The maximum rotations per minute (rpm). + default: 3600 + examples: + - 3600 max. rpm +``` + +
+ +
+ +| Product Specs | +| --- | +| 18V Cordless 4.5in angle grinder | +| 120V 12in chop saw 3600 max. rpm | + +
+ +
+ +| Blade Diameter | Max. RPM | +| --- | --- | +| 4.5 inches | | +| 12 inches | 3600 | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.ai: + api_key: Your OpenAI api key + input: Product Specs + output: + Blade Diameter: The diameter of the blade used, reported in inches. + Max. RPM: The maximum rotations per minute (rpm). +``` + +
+ +
+ +| Product Specs | +| --- | +| 18V Cordless 4.5in angle grinder | +| 120V 12in chop saw 3600 max. rpm | + +
+ +
+ +| Blade Diameter | Max. RPM | +| --- | --- | +| 4.5 inches | | +| 12 inches | 3600 | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.ai: + api_key: Your OpenAI api key + model_id: xxxx-xxxx-xxxxxxxx + output: + - Colors + - Sizes +``` + +
+ +
+ +| Items | +| --- | +| Large yellow square | +| Medium orange triangle | + +
+ +
+ +| Colors | Sizes | +| --- | --- | +| [yellow] | Large | +| [orange] | Medium | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.ai` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.ai` | + +**Sources** + +- [WranglesPY extract.ai implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.ai Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/ai.md) + +
+ + +--- + +## Attributes {#attributes} + +Extract numeric attributes from the input such as weights or lengths. Requires WrangleWorks Account. + +Extract numeric attributes from unstructured text such as lengths, voltages, weights, or temperatures. Requires WrangleWorks Account. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `responseContent` | Span - returns the text found. object - returns an object with the value and unit. | string; one of:
  • span
  • object
| `"span"` | No | +| `attribute_type` | Request only a specific type of attribute. | string, null; one of:
  • angle
  • area
  • capacitance
  • charge
  • current
  • data transfer rate
  • electrical conductance
  • electrical resistance
  • energy
  • force
  • frequency
  • inductance
  • instance frequency
  • length
  • luminous flux
  • weight
  • power
  • pressure
  • speed
  • velocity
  • temperature
  • time
  • voltage
  • volume
  • volumetric flow
| `null` | No | +| `desired_unit` | Convert the extracted unit to the desired unit. | string, null | `null` | No | +| `bound` | When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid. | string; one of:
  • min
  • mid
  • max
| `"mid"` | No | +| Formatting | | | | | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.attributes: + input: tools + output: attributes + responseContent: span +``` + +
+ +
+ +| Tools (input) | +| --- | +| hammer 5kg, 0.5m | + +
+ +
+ +| Attributes (span )(output) | +| --- | +| \{'length': ['0.5m'], 'mass': ['5kg']\} | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.attributes: + input: tools + output: attributes + responseContent: object +``` + +
+ +
+ +| Tools (input) | +| --- | +| hammer 5kg, 0.5m | + +
+ +
+ +| Attributes (Object) (output) | +| --- | +| \{'length': [\{'unit': 'metre', 'value': 0.5\}], 'mass': [\{'unit': 'kilogram', 'value': 5.0\}]\} | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.attributes: + input: Tools + output: attributes + responseContent: span + attribute_type: mass # Specific attributes +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +_No sample available._ + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.attributes` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.attributes` | + +**Sources** + +- [WranglesPY extract.attributes implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.attributes Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/attributes.md) + +
+ + +--- + +## Brackets {#brackets} + +Extract text properties in brackets from the input. + +Extract text in brackets from the input. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, array | — | Yes | +| Options | | | | | +| `find` | (Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets. | string, array | `"all"` | No | +| `include_brackets` | (Optional) Include the brackets in the output. | boolean | `false` | No | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.brackets: + input: Data + output: Output +``` + +
+ +
+ +| Data | +| --- | +| `{Hello}` | +| `[Wrangles]` | +| `(!)` | +| `` | + +
+ +
+ +| Output | +| --- | +| Hello | +| Wrangles | +| ! | +| ! | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.brackets` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.brackets` | + +**Sources** + +- [WranglesPY extract.brackets implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.brackets Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/brackets.md) + +
+ + +--- + +## Codes {#codes} + +Extract alphanumeric codes from the input. Requires WrangleWorks Account. + +Extract alphanumeric codes from unstructured text. Can be performed on one column or multiple columns. Requires WrangleWorks Account. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array | — | Yes | +| Options | | | | | +| `min_length` | Minimum length of allowed results. | integer, string | — | No | +| `max_length` | Maximum length of allowed results. | integer, string | — | No | +| `strategy` | Controls filtering of likely false positives such as measurements. Lenient skips this filter; balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict. | string; one of:
  • lenient
  • balanced
  • strict
| — | No | +| `disallowed_patterns` | A pattern or JSON array of regex patterns to not include in the found codes. | string | — | No | +| `include_multi_part_tokens` | Whether to include multi-part tokens that have a space. Default True. | boolean | — | No | +| `extract_raw` | Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False. | boolean | — | No | +| Formatting | | | | | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| `sort_order` | Default is input order. Also allows longest or shortest. | string; one of:
  • input
  • longest
  • shortest
| — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +# One column input +wrangles: + - extract.codes: + input: Secret + output: Code Extract +``` + +
+ +
+ +| Secret | +| --- | +| to gain access use Z1ON0101 | + +
+ +
+ +| Code Extract | +| --- | +| ['Z1ON0101'] | + +
+ +
+ + + + + +```yaml +# Multi column input +wrangles: + - extract.codes: + input: + - code1 + - code2 + output: Codes +``` + +
+ +
+ +| code1 | code2 | +| --- | --- | +| code CH465517080-1 | code CH465517080-2 | + +
+ +
+ +| Codes | +| --- | +| ['CH465517080-1', 'CH465517080-2'] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.codes` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.codes` | + +**Sources** + +- [WranglesPY extract.codes implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.codes Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/codes.md) + +
+ + +--- + +## Custom {#custom} + +Extract data from the input using a DIY or bespoke extraction wrangle. Requires WrangleWorks Account and Subscription. + +Extract data from the input using a DIY or bespoke extraction wrangle. Can be performed on one column or multiple columns. Requires WrangleWorks Account and Subscription. + +:::info +Non-regex pattern matching extracts whole-word matches separated by word boundaries. Word boundaries include anything that is not a letter, number, or underscore. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| `extract_raw` | Extract the raw data from the wrangle. | boolean | `false` | No | +| `use_spellcheck` | Use spellcheck to also find minor mispellings compared to the reference data. | boolean | `false` | No | +| `sort` | Sort the results. | string; one of:
  • training_order
  • input_order
  • longest
  • shortest
  • alphabetical
  • reverse_alphabetical
  • ascending
  • descending
| `"training_order"` | No | +| Formatting | | | | | +| `use_labels` | Use Labels in the extract output {label: value}. | boolean | `false` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `include_empty_labels` | Include labels with no found values in the output when using use_labels=True. | boolean | `true` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use. | string, array | — | Yes | + +
+ +### Examples + +```yaml +# One column input +wrangles: + - extract.custom: + input: Product + output: Wood Types + model_id: model_id_here +``` + +
+ +
+ +| Product | +| --- | +| Dining Oakwood Chair | +| Living Room Teakwood Frame Mirror | + +
+ +
+ +| Wood Types | +| --- | +| Oakwood | +| Teakwood | + +
+ +
+ + + + + +```yaml +# Multi column input +wrangles: + - extract.custom: + input: + - Part 1 of 2 + - Part 2 of 2 + output: Wood Types + model_id: model_id_here +``` + +
+ +
+ +| Part 1 of 2 | Part 2 of 2 | +| --- | --- | +| Dining Acacia Wood Table | Imitation Wood Table Chairs | + +
+ +
+ +| Wood Types | +| --- | +| ['Acacia Wood', 'Imitation Wood'] | + +
+ +
+ + + + + +```yaml +# Multiple Models +wrangles: + - extract.custom: + input: + - Product + - Product + output: + - Wood Types + - Item Type + model_id: + - wood_Type_model_id + - item_type_model_id +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| Product | Item Type | Wood Types | +| --- | --- | --- | +| Dining Oakwood Chair | Chair | Oakwood | +| Living Room Teakwood Frame Mirror | Mirror | Teakwood | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.custom` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.custom` | + +**Sources** + +- [WranglesPY extract.custom implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.custom Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/custom.md) + +
+ + +--- + +## Date Properties {#date-properties} + +Extract date properties from a date (day, month, year, etc...). + +Extract date properties from a date, such as day, month, year, weekday, or quarter. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, null | `null` | No | +| Options | | | | | +| `property` | Property to extract from date. | string; one of:
  • day
  • day_of_year
  • month
  • month_name
  • weekday
  • week_day_name
  • week_year
  • quarter
| — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.date_properties: + input: Date + output: Output + property: month_name +``` + +
+ +
+ +| Date | +| --- | +| 1992-08-13 00:00:00 | + +
+ +
+ +| Output | +| --- | +| August | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.date_properties` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.date_properties` | + +**Sources** + +- [WranglesPY extract.date_properties implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.date_properties Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/date-properties.md) + +
+ + +--- + +## Date Range {#date-range} + +Extract date range frequency from two dates. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `start_time` | Name of the start date column. | string | — | Yes | +| `end_time` | Name of the end date column. | string | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `range` | Type of frequency to count. | string; one of:
  • business days
  • days
  • weeks
  • months
  • semi months
  • business month ends
  • month starts
  • semi month starts
  • business month starts
  • quarters
  • quarter starts
  • years
  • business hours
  • hours
  • minutes
  • seconds
  • milliseconds
| `"day"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.date_range: + start_time: Start + end_time: End + output: Output + range: months +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| End | Start | Output | +| --- | --- | --- | +| 2023-08-13 00:00:00 | 1992-08-13 00:00:00 | 371 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.date_range` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.date_range` | + +**Sources** + +- [WranglesPY extract.date_range implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.date_range Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/date-range.md) + +
+ + +--- + +## HTML {#html} + +Extract elements from strings containing html. Requires WrangleWorks Account. + +Extract text and links from HTML elements. Requires WrangleWorks Account. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `data_type` | The type of data to extract. | string; one of:
  • text
  • links
| — | Yes | +| Formatting | | | | | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.html: + input: HTML + output: Text + data_type: text +``` + +
+ +
+ +| HTML | +| --- | +| ` | + +
+ +
+ +| Text | +| --- | +| | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.html: + input: HTML + output: Links + data_type: links +``` + +
+ +
+ +| HTML | +| --- | +| ` | + +
+ +
+ +| Links | +| --- | +| | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.html` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.html` | + +**Sources** + +- [WranglesPY extract.html implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.html Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/html.md) + +
+ + +--- + +## Properties {#properties} + +Extract text properties from the input. Requires WrangleWorks Account. + +Extract categorical properties from unstructured text, such as colours or materials. Requires WrangleWorks Account. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output columns. | string, array | — | Yes | +| Options | | | | | +| `property_type` | The specific type of properties to extract. | string, null; one of:
  • Colours
  • Materials
  • Shapes
  • Standards
| `null` | No | +| Formatting | | | | | +| `return_data_type` | Legacy format option. Prefer output_format. | string; one of:
  • list
  • string
| `"list"` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • dictionary
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.properties: + input: water bottles + output: properties +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| Products | Product Properties | +| --- | --- | +| Stainless Steel Blue Bottle | \{'Colours': ['Blue'], 'Materials': ['Stainless Steel']\} | +| Plastic Yellow Bottle | \{'Colours': ['Yellow'], 'Materials': ['Plastic']\} | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.properties: + input: water bottles + output: properties + property_type: colours # Optional +``` + +
+ +
+ +_No sample available._ + +
+ +
+ +| Products | Product Colours | +| --- | --- | +| Stainless Steel Blue Bottle | ['Blue'] | +| Plastic Yellow Bottle | ['Yellow'] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.properties` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.properties` | + +**Sources** + +- [WranglesPY extract.properties implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.properties Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/properties.md) + +
+ + +--- + +## Regex {#regex} + +Extract matches or specific capture groups using regex. + +Extract single values, matches, or specific capture groups using regex. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array | — | Yes | +| Options | | | | | +| `find` | Pattern to find using regex. | string | — | Yes | +| Formatting | | | | | +| `output_pattern` | Specifies the format to output matches and specific capture groups using backreferences (e.g., `\1`, `\2`). Default is to return entire matches. **Example**: For a regex pattern `r'(\d+)\s(\w+)'` and `output_pattern = '\2 \1'`, with input `'120 volt'`, the output would be `'volt 120'`. | string, null | `null` | No | +| `first_element` | Get the first element from results. | boolean | `false` | No | +| `output_format` | Format of the extract output. | string, null; one of:
  • list
  • columns
  • concatenate
| `null` | No | +| `char` | Character to use when output_format is concatenate. | string | `", "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - extract.regex: + input: Product + output: GPM + find: \d\.?\d? ?gpm +``` + +
+ +
+ +| Product | +| --- | +| 3.4 gpm water pump | +| 2gpm water pump | + +
+ +
+ +| GPM | +| --- | +| 3.4 gpm | +| 2gpm | + +
+ +
+ + + + + +```yaml +wrangles: + - extract.regex: + input: Product + output: GPM + find: (\d\.?\d?) ?gpm + output_pattern: \1 Gallons Per Minute +``` + +
+ +
+ +| Product | +| --- | +| 3.4 gpm water pump for 5.5 gallon tank | +| 2gpm water pump for 2 gal tank | + +
+ +
+ +| GPM | +| --- | +| 3.4 Gallons Per Minute | +| 2 Gallons Per Minute | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `extract.regex` | +| Lifecycle status | active | +| Namespace | `extract` | +| Documentation group | `extract` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.extract.regex` | + +**Sources** + +- [WranglesPY extract.regex implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/extract.py) +- [Existing extract.regex Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/extract/_sources/regex.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/format.md b/wrangles-docs/registry-docs/namespaces/format.md new file mode 100644 index 0000000..3338245 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/format.md @@ -0,0 +1,1304 @@ +--- +title: "Format Wrangles" +description: "Format wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Format" +slug: "/namespaces/format" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Format Wrangles + +Format wrangles, with recipe examples, parameters, and behavior. + +## Clean Whitespaces {#clean-whitespaces} + +Condense multiple spaces to a single space and convert special space characters to a standard space. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `trim` | Whether to trim leading and trailing spaces. Default True. | boolean | `true` | No | +| `remove_literals` | Whether to remove special space characters such as new lines etc. Default True. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - clean_whitespaces: + input: Products +``` + +
+ +
+ +| Products | +| --- | +| `Hello world!` | +| `Hello universe!` | + +
+ +
+ +| Products | +| --- | +| Hello world! | +| Hello universe! | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `clean_whitespaces` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.clean_whitespaces` | + +**Sources** + +- [WranglesPY clean_whitespaces implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing clean_whitespaces Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/clean-whitespaces.md) + +
+ + +--- + +## Dates {#dates} + +Format a date. + +Format a date into the desired pattern. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Formatting | | | | | +| `format` | String pattern to format date. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.dates: + input: Date + output: Output Format + format: '%Y-%m-%d' # must be wrapped by quotes " or ' +``` + +
+ +
+ +| Date | +| --- | +| 6/23/1912 | + +
+ +
+ +| Output Format | +| --- | +| 1912-06-23 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.dates` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.dates` | + +**Sources** + +- [WranglesPY format.dates implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.dates Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/dates.md) + +
+ + +--- + +## Pad {#pad} + +Pad a string to a fixed length. + +Pad a string to a fixed length + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `side` | Side from which to fill resulting string. | string | — | Yes | +| `skip_empty` | If true, skip padding for empty or whitespace-only values. | boolean | `false` | No | +| Formatting | | | | | +| `pad_length` | Length for the output. | integer | — | Yes | +| `char` | The character to pad the input with. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.pad: + input: Part Number + pad_length: 5 + side: left + char: "-" +``` + +
+ +
+ +| Part Number | +| --- | +| 0458 | +| 396 | +| 84 | +| 98516 | + +
+ +
+ +| Part Number | +| --- | +| -0458 | +| --396 | +| ---84 | +| 98516 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.pad` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.pad` | + +**Sources** + +- [WranglesPY format.pad implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.pad Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/pad.md) + +
+ + +--- + +## Prefix {#prefix} + +Add a prefix to a column. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | (Optional) Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `value` | Prefix value to add. | string, integer, number | — | Yes | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.prefix: + input: Data + output: Prefix + value: anti +``` + +
+ +
+ +| Data | +| --- | +| freeze | +| dote | +| hero | + +
+ +
+ +| Prefix | +| --- | +| antifreeze | +| antidote | +| antihero | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.prefix` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.prefix` | + +**Sources** + +- [WranglesPY format.prefix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.prefix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/prefix.md) + +
+ + +--- + +## Format Price Breaks {#price-breaks} + +Expand non-empty price-break cells into paired category and value columns. + +### Behavior + +Expand non-empty price-break cells into paired category and value columns. + +This guidance was derived from the callable signature. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | array | — | Yes | +| Formatting | | | | | +| `categoryLabel` | Prefix for output columns that identify the source price-break category. | string | — | Yes | +| `valueLabel` | Prefix for output columns that contain the corresponding price-break value. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ + + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.price_breaks` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.price_breaks` | + +**Sources** + +- [WranglesPY format.price_breaks implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) + +
+ + +--- + +## Remove Duplicates {#remove-duplicates} + +Remove duplicates from a list. Preserves input order. + +Remove duplicate values in a list. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `ignore_case` | Ignore case when removing duplicates. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.remove_duplicates: + input: Attack of the Clones + output: Commander + where: Rank = Commander +``` + +
+ +
+ +| Rank | Attack of the Clones | +| --- | --- | +| Commander | ['Cody', 'Cody', 'Cody'] | +| Captain | ['Rex', 'Rex', 'Rex'] | + +
+ +
+ +| Commander | +| --- | +| ['Cody'] | +| | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.remove_duplicates` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.remove_duplicates` | + +**Sources** + +- [WranglesPY format.remove_duplicates implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.remove_duplicates Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/remove-duplicates.md) + +
+ + +--- + +## Significant Figures {#significant-figures} + +Format a value to a specific number of significant figures. + +Format a value to a specific number of significant figures + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Formatting | | | | | +| `significant_figures` | Number of significant figures to format to. Default is 3. | integer | `3` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.significant_figures: + input: Data + significant_figures: 2 + output: Data to 2 Figures +``` + +
+ +
+ +| Data | +| --- | +| 1.25 | +| 12.3 | +| 55.6 | + +
+ +
+ +| Data to 2 Figures | +| --- | +| 1.2 | +| 12 | +| 55 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.significant_figures` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.significant_figures` | + +**Sources** + +- [WranglesPY format.significant_figures implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.significant_figures Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/significant-figures.md) + +
+ + +--- + +## Suffix {#suffix} + +Add a suffix to a column. + +Add a suffix to a column + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | (Optional) Name of the output column. | string, null | `null` | No | +| Options | | | | | +| `value` | Suffix value to add. | string, integer, number, array | — | Yes | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.suffix: + input: Data + output: Suffix + value: ic +``` + +
+ +
+ +| Data | +| --- | +| sto | +| hero | +| icon | + +
+ +
+ +| Suffix | +| --- | +| stoic | +| heroic | +| iconic | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.suffix` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.suffix` | + +**Sources** + +- [WranglesPY format.suffix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.suffix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/suffix.md) + +
+ + +--- + +## Trim {#trim} + +Remove excess whitespace at the start and end of text. + +Remove excess whitespace at the start and end of text. Can accept multiple columns. + +:::note +Non-string values pass through unaltered. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - format.trim: + input: + - col1 + output: col1 trimmed +``` + +
+ +
+ +| col1 | +| --- | +| ` Hello World ` | + +
+ +
+ +| col1 | col1 trimmed | +| --- | --- | +| Hello World | Hello World | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `format.trim` | +| Lifecycle status | active | +| Namespace | `format` | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.format.trim` | + +**Sources** + +- [WranglesPY format.trim implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/format.py) +- [Existing format.trim Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/trim.md) + +
+ + +--- + +## Remove Words {#remove-words} + +Remove all the elements that occur in one list from another. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of column to remove words from. | string, integer, array | — | Yes | +| `to_remove` | Column or list of columns with a list of words to be removed. | string | — | Yes | +| `output` | Name of the output columns. | string, array, null | `null` | No | +| Options | | | | | +| `tokenize_to_remove` | Tokenize all to_remove inputs. | boolean | `false` | No | +| `ignore_case` | Ignore input and to_remove case. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - remove_words: + input: Description + to_remove: # To Remove columns must be list + - Materials + - Colours + output: Product + tokenize_to_remove: True + ignore_case: False +``` + +
+ +
+ +| Colours | Materials | Description | +| --- | --- | --- | +| ['Blue'] | ['Steel'] | Steel Blue Bottle | +| ['Blue'] | ['Steel'] | ['Steel', 'Blue', 'Bottle'] | + +
+ +
+ +| Product | +| --- | +| Bottle | +| Bottle | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `remove_words` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.remove_words` | + +**Sources** + +- [WranglesPY remove_words implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing remove_words Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/remove-words.md) + +
+ + +--- + +## Replace {#replace} + +Quick find and replace for simple values. Can use regex if 'input' in params and isinstance(params['input'], list):in the find field. + +Quick find and replace for simple values. Can use regex in the `find` field. + +:::note +Values that are not a number or a string pass through unaltered. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input column. | string, integer, array | — | Yes | +| `output` | Name or list of output column. | string, array, null | `null` | No | +| Options | | | | | +| `find` | Pattern to find using regex. | string | — | Yes | +| `replace` | Value to replace the pattern found. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - replace: + input: Product Data + find: brg + replace: bearing +``` + +
+ +
+ +| Product Data | +| --- | +| SKF ball brg | +| brg seal | + +
+ +
+ +| Product Data | +| --- | +| SKF ball bearing | +| bearing seal | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `replace` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.replace` | + +**Sources** + +- [WranglesPY replace implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing replace Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/replace.md) + +
+ + +--- + +## Round {#round} + +Round column(s) to the specified decimals. + +Round numbers in a column to the nearest decimal point of your choosing. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Formatting | | | | | +| `decimals` | Number of decimal places to round column. | integer | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - round: + input: Cost Per Unit + output: Cost Rounded + decimals: 2 +``` + +
+ +
+ +| Cost Per Unit | +| --- | +| 3.14159 | +| 2.71828 | + +
+ +
+ +| Cost Rounded | +| --- | +| 3.14 | +| 2.72 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `round` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.round` | + +**Sources** + +- [WranglesPY round implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing round Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/round.md) + +
+ + +--- + +## Translate {#translate} + +Translate the input to a different language. Requires WrangleWorks Account and DeepL API Key (A free account for up to 500,000 characters per month is available). + +Translate the input column to another language. Powered by DeepL. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to translate. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `target_language` | Code of the language to translate to. | string; one of:
  • Bulgarian
  • Chinese
  • Czech
  • Danish
  • Dutch
  • English (American)
  • English (British)
  • Estonian
  • Finnish
  • French
  • German
  • Greek
  • Hungarian
  • Italian
  • Japanese
  • Latvian
  • Lithuanian
  • Polish
  • Portuguese
  • Portuguese (Brazilian)
  • Romanian
  • Russian
  • Slovak
  • Slovenian
  • Spanish
  • Swedish
| — | Yes | +| `source_language` | Code of the language to translate from. If omitted, automatically detects the input language. | string; one of:
  • Auto
  • Bulgarian
  • Chinese
  • Czech
  • Danish
  • Dutch
  • English
  • Estonian
  • Finnish
  • French
  • German
  • Greek
  • Hungarian
  • Italian
  • Japanese
  • Latvian
  • Lithuanian
  • Polish
  • Portuguese
  • Romanian
  • Russian
  • Slovak
  • Slovenian
  • Spanish
  • Swedish
| `"AUTO"` | No | +| `case` | Allow changing the case of the input prior to translation. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - translate: + input: Español + output: English + source_language: Spanish + target_language: English (British) +``` + +
+ +
+ +| Español | +| --- | +| ¡Hola Mundo! | + +
+ +
+ +| English | +| --- | +| Hello World! | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `translate` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.translate` | + +**Sources** + +- [WranglesPY translate implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing translate Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/translate.md) + +
+ + +--- + +## Standardize (Deprecated) {#standardize} + +:::warning Deprecated +This compatibility wrangle remains available for existing recipes. Use [`standardize.custom`](/wrangles/namespaces/standardize#custom) for new recipes. +::: + +Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model. + +Run a standardize wrangle, such as one that expands abbreviations. A standardization wrangle must be trained first. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use (do not include 'find' and 'replace'). | string, array | — | Yes | + +
+ +### Examples + +```yaml +wrangles: + - standardize: + input: Abbrev + output: Abbreviations + model_id: code_here +``` + +
+ +
+ +| Abbrev | +| --- | +| ASAP | +| ETA | + +
+ +
+ +| Abbreviations | +| --- | +| As Soon As Possible | +| Estimated Time of Arrival | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `standardize` | +| Lifecycle status | deprecated | +| Replaced by | [`standardize.custom`](/wrangles/namespaces/standardize#custom) | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.standardize` | + +**Sources** + +- [WranglesPY standardize implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing standardize Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/standardize.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/generate.md b/wrangles-docs/registry-docs/namespaces/generate.md new file mode 100644 index 0000000..3ecc7c2 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/generate.md @@ -0,0 +1,125 @@ +--- +title: "Generate Wrangles" +description: "Generate wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Generate" +slug: "/namespaces/generate" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Generate Wrangles + +Generate wrangles, with recipe examples, parameters, and behavior. + +## Ai {#ai} + +Generate structured AI output for each recipe row. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Target schema; string/array shorthands are expanded automatically. | string, array, object | — | Yes | +| `input` | Column(s) to concatenate into the prompt (defaults to all columns). | string, array, null | `null` | No | +| Options | | | | | +| `messages` | Optional extra messages forwarded to the inner generate helper. | array, object, null | `null` | No | +| `web_search` | Enable DuckDuckGo context lookup per row. | boolean | `false` | No | +| `summary` | Request summary text to be merged into the output. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Maximum concurrent requests (default 20). | integer | `20` | No | +| `timeout` | Per-request timeout in seconds. | integer | `90` | No | +| Errors | | | | | +| `retries` | Number of retry attempts on failure. | integer | `0` | No | +| Details | | | | | +| `api_key` | OpenAI-compatible API key. | string | — | Yes | +| `model` | Responses model name (e.g. gpt-5-mini). | string | `"gpt-5"` | No | +| `url` | Override for the OpenAI-compatible endpoint. | string | `"https://api.openai.com/v1/responses"` | No | +| `strict` | Enforce JSON-schema validation on the response. | boolean | `false` | No | +| `reasoning` | Responses API reasoning options (forwarded verbatim). | string, object | `{"effort":"low"}` | No | +| `previous_response` | Chain responses by reusing previous_response_id for field-by-field calls. | boolean | `false` | No | + +
+ +### Examples + +This template generates a category from a product description. AI-generated values can vary by model and request. + + + +```yaml +wrangles: + - generate.ai: + input: + - Product Description + output: + Category: + type: string + description: Broad product category + api_key: Your OpenAI-compatible API key + model: gpt-5-mini +``` + +
+ +
+ +| Product Description | +| --- | +| Stainless steel insulated water bottle, 750 ml | + +
+ +
+ +| Category | +| --- | +| Drinkware | + +
+ +
+ +_Template based on the documented parameters; no published source example is currently available._ + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `generate.ai` | +| Lifecycle status | active | +| Namespace | `generate` | +| Documentation group | `generate` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.generate.ai` | + +**Sources** + +- [WranglesPY generate.ai implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/generate.py) +- [Existing generate.ai Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/ai/_sources/ai.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/lookup.md b/wrangles-docs/registry-docs/namespaces/lookup.md new file mode 100644 index 0000000..7d91a91 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/lookup.md @@ -0,0 +1,108 @@ +--- +title: "Lookup Wrangles" +description: "Lookup wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Lookup" +slug: "/namespaces/lookup" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Lookup Wrangles + +Lookup wrangles, with recipe examples, parameters, and behavior. + +## Lookup {#lookup} + +Lookup values from a saved lookup wrangle. + +Look up data from a saved Lookup Wrangle. Data is output as a dictionary if an output is not specified or the output does not match any columns in the lookup. If specific lookup columns are named in the output, they will be output as individual columns. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to lookup. | string | — | Yes | +| `output` | Name of the output column(s). When n is provided and the output list length equals n, each output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. "Top *" with n: 3 becomes "Top 1", "Top 2", "Top 3". | string, array, null | `null` | No | +| Options | | | | | +| `lookup_mode` | How to perform lookups. 'by_row' (default): lookup each row individually. 'by_dataframe': lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation. | string; one of:
  • by_row
  • by_matrix
  • by_dataframe
| `"by_row"` | No | +| `n` | Number of matches to return per input value. When the output list length equals n, each output column receives the corresponding match. Otherwise all n matches are stored as a list in each output column. | integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The model_id to use lookup against. | string, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - lookup: + input: State + output: + - Abbreviation + model_id: 55555555-5555-5555 +``` + +
+ +
+ +| State | +| --- | +| Texas | +| New York | +| Virginia | + +
+ +
+ +| Abbreviation | +| --- | +| TX | +| NY | +| VA | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `lookup` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `lookup` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.lookup` | + +**Sources** + +- [WranglesPY lookup implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing lookup Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/lookup/_sources/lookup.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/merge.md b/wrangles-docs/registry-docs/namespaces/merge.md new file mode 100644 index 0000000..4e7d7f5 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/merge.md @@ -0,0 +1,807 @@ +--- +title: "Merge Wrangles" +description: "Merge wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Merge" +slug: "/namespaces/merge" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Merge Wrangles + +Merge wrangles, with recipe examples, parameters, and behavior. + +## Coalesce {#coalesce} + +Take the first non-empty value from a series of columns or lists. + +Take the first non-empty value from a series of columns. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns or a single column containing lists. | array | — | Yes | +| `output` | Name of the output columns. This is required if multiple input columns are provided. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.coalesce: + input: + - Col1 + - Col2 + - Col3 + output: Output Col + where: Col2 = E +``` + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| A | B | C | +| D | E | F | +| G | H | I | + +
+ +
+ +| Output Col | +| --- | +| | +| D | +| | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.coalesce` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.coalesce` | + +**Sources** + +- [WranglesPY merge.coalesce implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.coalesce Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/coalesce.md) + +
+ + +--- + +## Concatenate {#concatenate} + +Concatenate a list of columns or a list within a single column. + +If the input is a list of columns, concatenate multiple columns into one as a delimited string. If the input is a single column, concatenate a list within that column into a delimited string. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Either a single column name or list of columns. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty values, defaults to false. | boolean | `false` | No | +| Formatting | | | | | +| `char` | (Optional) Character to add between successive values. | string | `","` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +# Using concatenate to combine multiple columns +wrangles: + - merge.concatenate: + input: + - Col1 + - Col2 + - Col3 + output: Join Col + char: ', ' +``` + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| A | B | C | + +
+ +
+ +| Join Col | +| --- | +| A, B, C | + +
+ +
+ + + + + +```yaml +# Using concatenate to join a column that is a list +wrangles: + - merge.concatenate: + input: Col1 + output: Join List + char: ' ' +``` + +
+ +
+ +| Col1 | +| --- | +| ['A', 'B', 'C'] | + +
+ +
+ +| Join List | +| --- | +| A B C | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.concatenate` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.concatenate` | + +**Sources** + +- [WranglesPY merge.concatenate implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.concatenate Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/concatenate.md) + +
+ + +--- + +## Dictionaries {#dictionaries} + +Take dictionaries in multiple columns and merge them to a single dictionary. + +:::note +For duplicate keys, the last key in the input list takes precedence in the merged dictionary. +::: + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty dictionaries when merging. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.dictionaries: + input: + - Dict 1 + - Dict 2 + output: Merged +``` + +
+ +
+ +| Dict 1 | Dict 2 | +| --- | --- | +| \{'First': 'One'\} | \{'Second': 'Two'\} | + +
+ +
+ +| Merged | +| --- | +| \{'First': 'One', 'Second': 'Two'\} | + +
+ +
+ + + + + +```yaml +# Using a Wildcard (*) +wrangles: + - merge.dictionaries: + input: Dict * + output: Merged +``` + +
+ +
+ +| Dict 1 | Dict 2 | +| --- | --- | +| \{'First': 'One'\} | \{'Second': 'Two'\} | + +
+ +
+ +| Merged | +| --- | +| \{'First': 'One', 'Second': 'Two'\} | + +
+ +
+ + + + + +```yaml +# Using a Wildcard (*) +wrangles: + - merge.dictionaries: + input: + - Dict * + - -Dict 2 + output: Merged +``` + +Note the extra dash in front of `Dict 2` excludes that column from the wildcard selection. + +
+ +
+ + + +| Dict 1 | Dict 2 | Dict 3 | +| --- | --- | --- | +| \{'First': 'One'\} | \{'Second': 'Two'\} | \{'Third': 'Three'\} | + +
+ +
+ + + +| Merged | +| --- | +| \{'First': 'One', 'Third': 'Three'\} | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.dictionaries` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.dictionaries` | + +**Sources** + +- [WranglesPY merge.dictionaries implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.dictionaries Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/dictionaries.md) + +
+ + +--- + +## Key Value Pairs {#key-value-pairs} + +Create a dictionary from keys and values in paired columns e.g. COLUMN_NAME_1, COLUMN_VALUE_1, COLUMN_NAME_2, COLUMN_VALUE_2 ... + +Create a dictionary from keys and values in paired columns. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Matched pairs of key and value columns. | object | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `skip_empty` | Whether to skip empty keys or values when creating the dictionary. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.key_value_pairs: + input: + Letter: Number + output: Pairs +``` + +
+ +
+ +| Letter | Number | +| --- | --- | +| A | 1 | +| B | 2 | +| C | 3 | + +
+ +
+ +| Pairs | +| --- | +| \{'A': 1\} | +| \{'B': 2\} | +| \{'C': 3\} | + +
+ +
+ + + + + +```yaml +# Using a Wildcard (*) +wrangles: + - merge.key_value_pairs: + input: + key*: value* + output: Object +``` + +
+ +
+ +| key 1 | key 2 | value 1 | value 2 | +| --- | --- | --- | --- | +| A | One | a | First | +| B | Two | b | Second | +| C | three | c | Third | + +
+ +
+ +| Object | +| --- | +| \{'A': 'a', 'One': 'First'\} | +| \{'B': 'b', 'Two': 'Second'\} | +| \{'C': 'c', 'three': 'Third'\} | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.key_value_pairs` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.key_value_pairs` | + +**Sources** + +- [WranglesPY merge.key_value_pairs implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.key_value_pairs Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/key-value-pairs.md) + +
+ + +--- + +## Lists {#lists} + +Take lists in multiple columns and merge them to a single list. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `remove_duplicates` | Whether to remove duplicates from the created list. | boolean | `false` | No | +| `ignore_case` | Ignore case when removing duplicates. | boolean | `false` | No | +| `include_empty` | Whether to include empty values in the created list. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.lists: + input: + - col1 + - col2 + output: Combined Col + remove_duplicates: false +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| ['A', 'B'] | ['D', 'E'] | + +
+ +
+ +| Combined Col | +| --- | +| ['A', 'B', 'D', 'E'] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.lists` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.lists` | + +**Sources** + +- [WranglesPY merge.lists implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.lists Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/lists.md) + +
+ + +--- + +## To Dict {#to-dict} + +Take multiple columns and merge them to a dictionary (aka object) using the column headers as keys. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `include_empty` | Whether to include empty columns in the created dictionary. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.to_dict: + input: + - Col1 + - Col2 + output: Dict Col +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| A | B | + +
+ +
+ +| Dict Col | +| --- | +| \{'Col1': 'A', 'Col2': 'B'\} | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.to_dict` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.to_dict` | + +**Sources** + +- [WranglesPY merge.to_dict implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.to_dict Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/to-dict.md) + +
+ + +--- + +## To List {#to-list} + +Take multiple columns and merge them to a list. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of input columns. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `include_empty` | Whether to include empty columns in the created list. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - merge.to_list: + input: + - Col1 + - Col2 + - Col3 + output: List Col +``` + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| A | B | C | + +
+ +
+ +| List Col | +| --- | +| ['A', 'B', 'C'] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `merge.to_list` | +| Lifecycle status | active | +| Namespace | `merge` | +| Documentation group | `merge` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.merge.to_list` | + +**Sources** + +- [WranglesPY merge.to_list implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/merge.py) +- [Existing merge.to_list Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/merge/_sources/to-list.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/search.md b/wrangles-docs/registry-docs/namespaces/search.md new file mode 100644 index 0000000..7fa9e55 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/search.md @@ -0,0 +1,230 @@ +--- +title: "Search Wrangles" +description: "Search wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Search" +slug: "/namespaces/search" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Search Wrangles + +Search wrangles, with recipe examples, parameters, and behavior. + +## Find Links {#find-links} + +Perform web searches to find links. Returns structured search results with titles, links, snippets, and optional pricing. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `queries` | Name or list of input columns containing search queries. | string, array | — | Yes | +| `id` | Name of the column containing the row ID to append to each search result. | string | — | Yes | +| `output` | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | string, array, null | `null` | No | +| Options | | | | | +| `n_results` | Number of search results to return per query (default 10, max 100). | integer | `10` | No | +| `country` | Country code for search results (default 'us'). Alias: gl. | string | — | No | +| `language` | Language code for search results (default 'en'). Alias: hl. | string | — | No | +| `location` | Location for search results (e.g., 'Austin, Texas'). | string | — | No | +| `device` | Device type for search results. | string; one of:
  • desktop
  • mobile
  • tablet
| — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Number of concurrent threads for parallel processing (default 10). | integer | `10` | No | +| Details | | | | | +| `client` | The search provider to use. | string; one of:
  • serpapi
| `"serpapi"` | No | +| `api_key` | API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY). | string, null | `null` | No | + +
+ +### Examples + +This template searches once per query and returns structured result dictionaries. Search results vary by provider, location, and time. + + + +```yaml +wrangles: + - search.find_links: + id: Row ID + queries: + - Search Query + output: + - Search Results + api_key: Your SerpApi API key + client: serpapi + n_results: 5 +``` + +
+ +
+ +| Row ID | Search Query | +| --- | --- | +| 1001 | SKF 6202 bearing supplier | + +
+ +
+ +| Search Results | +| --- | +| `[{"title": "Example result", "link": "https://example.com/6202", "snippet": "SKF 6202 bearing"}]` | + +
+ +
+ +_Template based on the documented parameters; no published source example is currently available._ + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `search.find_links` | +| Lifecycle status | active | +| Namespace | `search` | +| Documentation group | `search` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.search.find_links` | + +**Sources** + +- [WranglesPY search.find_links implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/search.py) +- [Existing search.find_links Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/search/_sources/find-links.md) + +
+ + +--- + +## Retrieve Link Content {#retrieve-link-content} + +Retrieves targeted content from web pages using LLM URL extraction. Can optionally output a second column containing a clean, human-readable text summary of the retrieved data. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns containing URLs or Scored Search Result dictionaries. | string, array | — | Yes | +| `output` | Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]). | string, array, null | `null` | No | +| Options | | | | | +| `prompt` | Optional custom system prompt to guide the extraction behavior and output format. | string, null | `null` | No | +| Formatting | | | | | +| `output_format` | The desired format for the extracted content. | string; one of:
  • markdown
  • json
| `"json"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Number of concurrent threads for parallel processing (default 10). | integer | `10` | No | +| Details | | | | | +| `client` | The retrieval provider to use. | string; one of:
  • google_url_context
| `"google_url_context"` | No | +| `api_key` | API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY). | string, null | `null` | No | +| `model_id` | The specific model ID to use (default models/gemini-3-flash-preview). | string | `"models/gemini-3-flash-preview"` | No | + +
+ +### Examples + +This template extracts JSON content from a URL. Returned fields depend on the page, prompt, and retrieval model. + + + +```yaml +wrangles: + - search.retrieve_link_content: + input: + - Product URL + output: + - Page Data + api_key: Your Google API key + client: google_url_context + output_format: json + prompt: Extract the product title and manufacturer. +``` + +
+ +
+ +| Product URL | +| --- | +| https://example.com/products/6202 | + +
+ +
+ +| Page Data | +| --- | +| `{"title": "6202 Bearing", "manufacturer": "SKF"}` | + +
+ +
+ +_Template based on the documented parameters; no published source example is currently available._ + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `search.retrieve_link_content` | +| Lifecycle status | active | +| Namespace | `search` | +| Documentation group | `search` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.search.retrieve_link_content` | + +**Sources** + +- [WranglesPY search.retrieve_link_content implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/search.py) +- [Existing search.retrieve_link_content Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/search/_sources/retrieve-link-content.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/select.md b/wrangles-docs/registry-docs/namespaces/select.md new file mode 100644 index 0000000..3b12a6f --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/select.md @@ -0,0 +1,1929 @@ +--- +title: "Select Wrangles" +description: "Select wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Select" +slug: "/namespaces/select" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Select Wrangles + +Select wrangles, with recipe examples, parameters, and behavior. + +## Drop {#drop} + +Drop (Delete) selected column(s). + +Drop (Delete) selected column(s) + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | Name of the column(s) to drop. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | + +
+ +### Examples + +```yaml +wrangles: + - drop: + columns: + - Material +``` + +
+ +
+ +| Material | Product Data | +| --- | --- | +| Ceramic | SKF ball brg | +| Rubber | brg seal | + +
+ +
+ +| Product Data | +| --- | +| SKF ball bearing | +| bearing seal | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `drop` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.drop` | + +**Sources** + +- [WranglesPY drop implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing drop Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/drop.md) + +
+ + +--- + +## Filter {#filter} + +Filter the dataframe based on the contents. If multiple filters are specified, all must be correct. For complex filters, use the where parameter. + +Filter the dataframe based on the contents. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to filter on. If multiple are provided, all must match the criteria. | string, integer, array | `[]` | No | +| Options | | | | | +| `equal` | Select rows where the values equal a given value. | string, array, null | `null` | No | +| `not_equal` | Select rows where the values do not equal a given value. | string, array, null | `null` | No | +| `is_in` | Select rows where the values are in a given list. | string, array, null | `null` | No | +| `not_in` | Select rows where the values are not in a given list. | string, array, null | `null` | No | +| `greater_than` | Select rows where the values are greater than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `greater_than_equal_to` | Select rows where the values are greater than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `less_than` | Select rows where the values are less than a specified value. Does not include the value itself. | integer, number, null | `null` | No | +| `less_than_equal_to` | Select rows where the values are less than a specified value. Does include the value itself. | integer, number, null | `null` | No | +| `between` | Value or list of values to filter that are in between two parameter values. | array, null | `null` | No | +| `contains` | Select rows where the input contains the value. Allows regular expressions. | string, null | `null` | No | +| `not_contains` | Select rows where the input does not contain the value. Allows regular expressions. | string, null | `null` | No | +| `is_null` | If true, select all rows where the value is NULL. If false, where is not NULL. | boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + # Select only red fruits + - filter: + input: Color + equal: + - red +``` + +
+ +
+ +| Color | Fruit | +| --- | --- | +| red | Apple | +| green | Apple | +| orange | Orange | +| red | Strawberry | + +
+ +
+ +| Color | Fruit | +| --- | --- | +| red | Apple | +| red | Strawberry | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `filter` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.filter` | + +**Sources** + +- [WranglesPY filter implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing filter Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/filter.md) + +
+ + +--- + +## Columns {#columns} + +Select columns from the dataframe. + +Select columns from the dataframe + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to select. | string, integer, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.columns: + input: Manufacturer +``` + +
+ +
+ +| Part Number | Manufacturer | +| --- | --- | +| 1234 | SKF | +| 5678 | Timken | + +
+ +
+ +| Manufacturer | +| --- | +| SKF | +| Timken | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.columns` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.columns` | + +**Sources** + +- [WranglesPY select.columns implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.columns Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/columns.md) + +
+ + +--- + +## Dictionary Element {#dictionary-element} + +Select one or more element of a dictionary. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be replaced. | string, array, null | `null` | No | +| Options | | | | | +| `element` | The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set. | string, number, array, object, boolean, null | `""` | No | + +
+ +### Examples + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + output: Shapes + element: shapes + default: square + where: Part Number = 1234 +``` + +
+ +
+ +| Properties | Part Number | +| --- | --- | +| \{'colours': ['red', 'white', 'blue'], 'shapes': 'round', 'materials': 'tungsten'\} | 1234 | +| \{'colours': ['green', 'yellow', 'orange'], 'shapes': 'square', 'materials': 'tungsten'\} | 5678 | + +
+ +
+ +| Shapes | +| --- | +| round | +| | + +
+ +
+ + + + + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + element: + - shapes + - materials +``` + +
+ +
+ +| Properties | +| --- | +| \{'colours': ['red', 'white', 'blue'], 'shapes': 'round', 'materials': 'tungsten'\} | +| \{'colours': ['green', 'yellow', 'orange'], 'shapes': 'square', 'materials': 'tungsten'\} | + +
+ +
+ +| Properties | +| --- | +| \{'shapes': 'round', 'materials': 'tungsten'\} | +| \{'shapes': 'square', 'materials': 'tungsten'\} | + +
+ +
+ + + + + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + output: Output Dict + element: + - Col* +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Other3': 'C'\} | + +
+ +
+ +| Column | Output Dict | +| --- | --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | \{'Col1': 'A', 'Col2': 'B'\} | + +
+ +
+ + + + + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + output: Output Dict + element: + - "regex: .*2" +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Output Dict | +| --- | +| \{'Col2': 'B'\} | + +
+ +
+ + + + + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + output: Output Dict + element: + - Col1: Column 1 + - Col2: Column 2 +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Output Dict | +| --- | +| \{'Column 1': 'A', 'Column 2': 'B'\} | + +
+ +
+ + + + + +```yaml +wrangles: + - select.dictionary_element: + input: Properties + output: Output Dict + element: + - Col1 + - Col3 + default: + Col1: Z + Col3: Y +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | +| \{'Col1': 'D', 'Col2': 'E'\} | + +
+ +
+ +| Output Dict | +| --- | +| \{'Col1': 'A', 'Col3': 'C'\} | +| \{'Col1': 'D', 'Col3': 'Y'\} | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.dictionary_element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.dictionary_element` | + +**Sources** + +- [WranglesPY select.dictionary_element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.dictionary_element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/dictionary-element.md) + +
+ + +--- + +## Element {#element} + +Select elements of lists or dicts using python syntax like col[0]['key']. + +Select elements of lists or dictionaries using Python syntax like `col[1:3]['key']`. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{"key":"val"}] -> "val". | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. | string, number, array, object, boolean, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - select.element: + input: Column A[0] + output: First Element +``` + +
+ +
+ +| Column A | +| --- | +| [A, 0.9] | +| [B, 0.8] | + +
+ +
+ +| First Element | +| --- | +| A | +| B | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.element` | + +**Sources** + +- [WranglesPY select.element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/element.md) + +
+ + +--- + +## Group By {#group-by} + +Group and aggregate the data. + +Group and aggregate the data + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `by` | List of the input columns to group on. | string, array | `[]` | No | +| Options | | | | | +| `list` | Group and return all values for these column(s) as a list. | string, array | — | No | +| `first` | The first value for these column(s). | string, array | — | No | +| `last` | The last value for these column(s). | string, array | — | No | +| `min` | The minimum value for these column(s). | string, array | — | No | +| `max` | The maximum value for these column(s). | string, array | — | No | +| `mean` | The mean (average) value for these column(s). | string, array | — | No | +| `median` | The median value for these column(s). | string, array | — | No | +| `nunique` | The count of unique values for these column(s). | string, array | — | No | +| `count` | The count of values for these column(s). | string, array | — | No | +| `counts` | Return a dictionary containing the count of each distinct value for these column(s). Keys are converted to JSON-safe strings; missing values use the key "null" and booleans use lowercase "true"/"false". | string, array | — | No | +| `std` | The standard deviation of values for these column(s). | string, array | — | No | +| `sum` | The total of values for these column(s). | string, array | — | No | +| `any` | Return true if any of the values for these column(s) are true. | string, array | — | No | +| `all` | Return true if all of the values for these column(s) are true. | string, array | — | No | +| `p75` | Get a percentile. Note, you can use any integer here for the corresponding percentile. | string, array | — | No | +| `custom.*` | Placeholder for custom functions. Replace 'placeholder' with the name of the function. | string, array | — | No | +| Formatting | | | | | +| `auto_rename_columns` | If true (default), aggregated column names include the operation as a suffix (e.g. Value.sum). If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total). | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.group_by: + by: + - Product Type + sum: Quantity + mean: Price ($) +``` + +
+ +
+ +| Product | Quantity | Price ($) | Product Type | +| --- | --- | --- | --- | +| Hammer | 3 | 12.99 | Hand Tools | +| Ratchet Wrench | 12 | 6.99 | Hand Tools | +| Cordless Drill | 2 | 49.99 | Power Tools | +| Reciprocating Saw | 7 | 29.99 | Power Tools | + +
+ +
+ +| Product Type | Quantity.sum | Price ($).mean | +| --- | --- | --- | +| Hand Tools | 15 | 9.99 | +| Power Tools | 9 | 39.99 | + +
+ +
+ + + + + +```yaml +wrangles: + - select.group_by: + by: Category + custom.sum_times_two: Quantity +``` + +
+ +
+ +| Category | Quantity | +| --- | --- | +| Hand Tools | 3 | +| Hand Tools | 1 | +| Hand Tools | 2 | +| Power Tools | 4 | + +
+ +
+ +| Category | Quantity.sum_times_two | +| --- | --- | +| Hand Tools | 12 | +| Power Tools | 4 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.group_by` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.group_by` | + +**Sources** + +- [WranglesPY select.group_by implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.group_by Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/group-by.md) + +
+ + +--- + +## Head {#head} + +Return the first n rows. + +Return the first n rows + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `n` | Number of rows to return. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.head: + n: 2 +``` + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Race | Timken | +| Needle Bearing | Acme Bearings | +| Roller Bearing | General Bearing Co. | + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Race | Timken | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.head` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.head` | + +**Sources** + +- [WranglesPY select.head implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.head Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/head.md) + +
+ + +--- + +## Highest Confidence {#highest-confidence} + +Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form [<<value>>, <<confidence_score>>]. + +Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form `[value, confidence_score]`. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of the input columns to select from. | array | — | Yes | +| `output` | If two columns; the result and confidence. If one column; [result, confidence]. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.highest_confidence: + input: + - Col1 + - Col2 + - Col3 + output: Highest Confidence +``` + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| ['A', 0.79] | ['B', 0.77] | ['C', 0.99] | + +
+ +
+ +| Highest Confidence | +| --- | +| ['C', 0.99] | + +
+ +
+ + + + + +```yaml +wrangles: + - select.highest_confidence: + input: + - Col1 + - Col2 + - Col3 + output: + - Item + - Confidence +``` + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| ['A', 0.79] | ['B', 0.77] | ['C', 0.99] | + +
+ +
+ +| Item | Confidence | +| --- | --- | +| C | 0.99 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.highest_confidence` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.highest_confidence` | + +**Sources** + +- [WranglesPY select.highest_confidence implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.highest_confidence Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/highest-confidence.md) + +
+ + +--- + +## Left {#left} + +Return characters from the left of text. Strings shorter than the length defined will be unaffected. + +Select characters from the left of the input. Using a negative length reverses the side of selection, selecting from the right. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `length` | Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.left: + input: Column + output: Result + length: 3 +``` + +
+ +
+ +| Column | +| --- | +| pudding | + +
+ +
+ +| Result | +| --- | +| pud | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.left` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.left` | + +**Sources** + +- [WranglesPY select.left implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.left Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/left.md) + +
+ + +--- + +## Length {#length} + +Calculate the lengths of data in a column. The length depends on the data type e.g. text will be the length of the text, lists will be the number of elements in the list. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.length: + input: Part Code + output: Part Code Length +``` + +
+ +
+ +| Part Code | +| --- | +| 6202 | + +
+ +
+ +| Part Code Length | +| --- | +| 4 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.length` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.length` | + +**Sources** + +- [WranglesPY select.length implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.length Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/length.md) + +
+ + +--- + +## List Element {#list-element} + +Select a numbered element of a list (zero indexed). + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `element` | The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list. | integer | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. | string, number, array, object, boolean, null | `""` | No | + +
+ +### Examples + +```yaml +wrangles: + - select.list_element: + input: Col1 + output: Second Element + element: 2 # Zero indexed + default: F +``` + +
+ +
+ +| Col1 | +| --- | +| ['A', 'B', 'C'] | +| ['D', 'E'] | + +
+ +
+ +| Third Element | +| --- | +| C | +| F | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.list_element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.list_element` | + +**Sources** + +- [WranglesPY select.list_element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.list_element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/list-element.md) + +
+ + +--- + +## Right {#right} + +Return characters from the right of text. Strings shorter than the length defined will be unaffected. + +Select characters from the right of the input. Using a negative length reverses the side of selection, selecting from the left. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `length` | Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.right: + input: Column + output: Result + length: 3 +``` + +
+ +
+ +| Column | +| --- | +| pudding | + +
+ +
+ +| Result | +| --- | +| ing | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.right` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.right` | + +**Sources** + +- [WranglesPY select.right implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.right Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/right.md) + +
+ + +--- + +## Sample {#sample} + +Return a random sample of the rows. + +Return a random sample of the rows + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `rows` | If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned. | integer, number | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.sample: + rows: 2 +``` + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Race | Timken | +| Needle Bearing | Acme Bearings | +| Roller Bearing | General Bearing Co. | + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Needle Bearing | Acme Bearings | + +
+ +
+ + + + + +```yaml +wrangles: + - select.sample: + rows: .25 +``` + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Race | Timken | +| Needle Bearing | Acme Bearings | +| Roller Bearing | General Bearing Co. | + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Needle Bearing | Acme Bearings | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.sample` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.sample` | + +**Sources** + +- [WranglesPY select.sample implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.sample Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/sample.md) + +
+ + +--- + +## Substring {#substring} + +Return characters from the middle of text. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `start` | The position of the first character to select. If ommited will start from the beginning and length must be provided. | integer, null | `null` | No | +| `length` | The length of the string to select. If ommited will select to the end of the string and start must be provided. | integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.substring: + input: Column + output: Result + start: 2 + length: 3 +``` + +
+ +
+ +| Column | +| --- | +| pudding | + +
+ +
+ +| Result | +| --- | +| udd | + +
+ +
+ + + + + +```yaml +wrangles: + - select.substring: + input: Column + output: Result + start: 2 +``` + +
+ +
+ +| Column | +| --- | +| pudding | + +
+ +
+ +| Result | +| --- | +| udding | + +
+ +
+ + + + + +```yaml +wrangles: + - select.substring: + input: Column + output: Result + length: 3 +``` + +
+ +
+ +| Column | +| --- | +| pudding | + +
+ +
+ +| Result | +| --- | +| pud | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.substring` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.substring` | + +**Sources** + +- [WranglesPY select.substring implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.substring Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/substring.md) + +
+ + +--- + +## Tail {#tail} + +Return the last n rows. + +Return the last n rows + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `n` | Number of rows to return. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.tail: + n: 2 +``` + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Race | Timken | +| Needle Bearing | Acme Bearings | +| Roller Bearing | General Bearing Co. | + +
+ +
+ +| Product | Manufacturer | +| --- | --- | +| Needle Bearing | Acme Bearings | +| Roller Bearing | General Bearing Co. | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.tail` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.tail` | + +**Sources** + +- [WranglesPY select.tail implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.tail Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/tail.md) + +
+ + +--- + +## Threshold {#threshold} + +Select the first option if it exceeds a given threshold, else the second option. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of the input columns to select from. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `threshold` | Threshold above which to choose the first option, otherwise the second. | number | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - select.threshold: + input: + - Col1 + - Col2 + output: Result + threshold: .77 +``` + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| ['A', 0.6] | ['B', 0.79] | + +
+ +
+ +| Result | +| --- | +| B | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.threshold` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.threshold` | + +**Sources** + +- [WranglesPY select.threshold implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) +- [Existing select.threshold Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/threshold.md) + +
+ + +--- + +## Sort {#sort} + +Sort the data. + +Sort the data + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `by` | Name or list of the column(s) to sort by. | string, array | — | Yes | +| Options | | | | | +| `ascending` | Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by. | boolean, array | — | No | +| Formatting | | | | | +| `ignore_index` | Ignore Index value accepted by the runtime. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - sort: + by: Price + ascending: true +``` + +
+ +
+ +| Item | Price | +| --- | --- | +| Hammer | 11.99 | +| Chisel | 4.99 | +| Drill | 29.99 | +| Wrench | 6.99 | +| Saw | 13.99 | + +
+ +
+ +| Item | Price | +| --- | --- | +| Chisel | 4.99 | +| Wrench | 6.99 | +| Hammer | 11.99 | +| Saw | 13.99 | +| Drill | 29.99 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `sort` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.sort` | + +**Sources** + +- [WranglesPY sort implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing sort Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/sort.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/split.md b/wrangles-docs/registry-docs/namespaces/split.md new file mode 100644 index 0000000..2755ce1 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/split.md @@ -0,0 +1,827 @@ +--- +title: "Split Wrangles" +description: "Split wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Split" +slug: "/namespaces/split" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Split Wrangles + +Split wrangles, with recipe examples, parameters, and behavior. + +## Explode {#explode} + +Explode a column of lists into rows. + +Explode a column of lists into rows + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length. | string, integer, array | — | Yes | +| Options | | | | | +| `drop_empty` | If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False. | boolean | `false` | No | +| Formatting | | | | | +| `reset_index` | Reset the index after exploding. Default True. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - explode: + input: Products +``` + +
+ +
+ +| Products | Manufacturer | +| --- | --- | +| [Ball Bearing, Bearing Seal] | SKF | +| [Angle Grinder, Drill, Impact Driver] | Milwaukee | +| Solid State Relay | Schneider | + +
+ +
+ +| Products | Manufacturer | +| --- | --- | +| Ball Bearing | SKF | +| Bearing Seal | SKF | +| Angle Grinder | Milwaukee | +| Drill | Milwaukee | +| Impact Driver | Milwaukee | +| Solid State Relay | Schneider | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `explode` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.explode` | + +**Sources** + +- [WranglesPY explode implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing explode Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/explode.md) + +
+ + +--- + +## Dictionary {#dictionary} + +Split one or more dictionaries into columns. The dictionary keys will be returned as the new column headers. If the dictionaries contain overlapping values, the last value will be returned. + +Split a dictionary into columns. The dictionary keys are used as the new column headers. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned. | string, integer, array | — | Yes | +| `output` | In columns output_format, this is an optional subset of keys to extract from the dictionary. If not provided, all keys will be returned. Columns can be renamed with the following syntax: output: - key1: new_column_name1 - key2: new_column_name2 In to_lists output_format, this must be two output columns for the keys and values lists. If not provided, Keys and Values will be used. | string, array, null | `null` | No | +| Formatting | | | | | +| `output_format` | How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values. | string; one of:
  • columns
  • to_lists
| `"columns"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Provide a set of default headings and values if they are not found within the input. | object, null | `null` | No | + +
+ +### Examples + +```yaml +wrangles: + - split.dictionary: + input: Column + # Output not required +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Col1 | Col2 | Col3 | +| --- | --- | --- | +| A | B | C | + +
+ +
+ + + + + +```yaml +wrangles: + - split.dictionary: + input: Column + output: Col2 +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Col2 | +| --- | +| B | + +
+ +
+ + + + + +```yaml +wrangles: + - split.dictionary: + input: Column + output: Col* +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Other': 'C'\} | + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| A | B | + +
+ +
+ + + + + +```yaml +wrangles: + - split.dictionary: + input: Column + output: "regex: .*3" +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Col3 | +| --- | +| C | + +
+ +
+ + + + + +```yaml +wrangles: + - split.dictionary: + input: Column + output: + - Col1: Column 1 + - Col2: Column 2 +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Column 1 | Column 2 | +| --- | --- | +| A | B | + +
+ +
+ + + + + +```yaml +wrangles: + - split.dictionary: + input: Column + output: + - Col*: Column * +``` + +
+ +
+ +| Column | +| --- | +| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | + +
+ +
+ +| Column 1 | Column 2 | Column 3 | +| --- | --- | --- | +| A | B | C | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.dictionary` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.dictionary` | + +**Sources** + +- [WranglesPY split.dictionary implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) +- [Existing split.dictionary Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/dictionary.md) + +
+ + +--- + +## List {#list} + +Split a list in a single column to multiple columns. + +Split a list into multiple columns. If only one output is given, `split.list` returns the same list it was given, so output should be a list of columns or a column name with a wildcard (`*`). + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to be split. | string, integer | — | Yes | +| `output` | Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - split.list: + input: Column + output: Column* +``` + +
+ +
+ +| Column | +| --- | +| ['A', 'B', 'C'] | + +
+ +
+ +| Column1 | Column2 | Column3 | +| --- | --- | --- | +| A | B | C | + +
+ +
+ + + + + +```yaml +wrangles: + - split.list: + input: Column + output: + - Heading A + - Heading B + - Heading C +``` + +
+ +
+ +| Column | +| --- | +| ['A', 'B', 'C'] | + +
+ +
+ +| Heading A | Heading B | Heading C | +| --- | --- | --- | +| A | B | C | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.list` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.list` | + +**Sources** + +- [WranglesPY split.list implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) +- [Existing split.list Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/list.md) + +
+ + +--- + +## Text {#text} + +Split a string to multiple columns or a list. + +Split text strings on certain characters. The text can be split into either multiple columns or a list. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to be split. | string | — | Yes | +| `output` | Name of the output column(s) If a single column is provided, the results will be returned as a list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input. | string, array, null | `null` | No | +| Options | | | | | +| `char` | Set the character(s) to split on. Default comma (,) Can also prefix with "regex:" to split on a pattern. | string | `","` | No | +| `element` | Select a specific element or range after splitting using slicing syntax. e.g. 0, ":5", "5:", "2:8:2". | string, integer, null | `null` | No | +| `inclusive` | If true, include the split character in the output. Default False. | boolean | `false` | No | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Formatting | | | | | +| `pad` | Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists. | boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - split.text: + input: Column1 + output: Column2 + char: ', ' +``` + +
+ +
+ +| Column1 | +| --- | +| Hello, Wrangles! | + +
+ +
+ +| Column2 | +| --- | +| ['Hello', 'Wrangles!'] | + +
+ +
+ + + + + +```yaml +# Split on x, case insensitive. +wrangles: + - split.text: + input: Col1 + output: Col2 + char: 'regex:(?i)x' +``` + +
+ +
+ +| Col1 | +| --- | +| 1x2 | +| 1X2 | + +
+ +
+ +| Col2 | +| --- | +| ['1', '2'] | +| ['1', '2'] | + +
+ +
+ + + + + +```yaml +wrangles: + - split.text: + input: Column1 + output: Column2 + char: ', ' + element: 0 +``` + +
+ +
+ +| Column1 | +| --- | +| Hello, Wrangles! | + +
+ +
+ +| Column2 | +| --- | +| Hello | + +
+ +
+ + + + + +```yaml +wrangles: + - split.text: + input: Col + output: Col* # Optional + char: ', ' +``` + +
+ +
+ +| Col | +| --- | +| Hello, Wrangles! | + +
+ +
+ +| Col1 | Col2 | +| --- | --- | +| Hello | Wrangles! | + +
+ +
+ + + + + +```yaml +wrangles: + - split.text: + input: Col + output: + - Col 1 + - Col 2 + - Col 3 + char: ', ' +``` + +
+ +
+ +| Col | +| --- | +| Wrangles, are, Cool! | + +
+ +
+ +| Col 1 | Col 2 | Col 3 | +| --- | --- | --- | +| Wrangles | are | Cool! | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.text` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.text` | + +**Sources** + +- [WranglesPY split.text implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) +- [Existing split.text Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/text.md) + +
+ + +--- + +## Tokenize {#tokenize} + +Split text into tokens. A variety of methods are available. The default method is to split on spaces. + +Tokenize elements in a list or string into individual tokens. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Column(s) to be split into tokens. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `method` | Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`. | string; one of:
  • space
  • boundary
  • boundary_ignore_space
or string | `"space"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - split.tokenize: + input: Materials + output: Tokenized List +``` + +
+ +
+ +| Materials | +| --- | +| Stainless Steel Oak Wood | + +
+ +
+ +| Tokenized List | +| --- | +| ['Stainless', 'Steel', 'Oak', 'Wood'] | + +
+ +
+ + + + + + +```yaml +wrangles: + - split.tokenize: + input: Materials + output: Tokenized List +``` + +
+ +
+ +| Materials | +| --- | +| ['Stainless Steel', 'Oak Wood'] | + +
+ +
+ +| Tokenized List | +| --- | +| ['Stainless', 'Steel', 'Oak', 'Wood'] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.tokenize` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.tokenize` | + +**Sources** + +- [WranglesPY split.tokenize implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) +- [Existing split.tokenize Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/tokenize.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/standardize.md b/wrangles-docs/registry-docs/namespaces/standardize.md new file mode 100644 index 0000000..308ccab --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/standardize.md @@ -0,0 +1,152 @@ +--- +title: "Standardize Wrangles" +description: "Standardize wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Standardize" +slug: "/namespaces/standardize" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Standardize Wrangles + +Standardize wrangles, with recipe examples, parameters, and behavior. + +## Standardize Clean {#clean} + +Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. + +### Behavior + +Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. + +This guidance was derived from the callable signature and its embedded Python schema docstring. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. Defaults to overwriting input. | string, integer, array, null | `null` | No | +| Options | | | | | +| `fix_encoding` | Repair mojibake and other reversible encoding errors. | boolean | `true` | No | +| `unescape_html` | Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup. | boolean or string; one of:
  • auto
| `"auto"` | No | +| `normalization` | Unicode normalization form. | string; one of:
  • NFC
  • NFKC
  • NFD
  • NFKD
  • null
| `"NFC"` | No | +| `fix_character_width` | Normalize fullwidth and halfwidth characters. | boolean | `true` | No | +| `uncurl_quotes` | Replace typographic quotes with straight quotes. | boolean | `true` | No | +| `remove_control_chars` | Remove C0 and C1 control characters. | boolean | `true` | No | +| `collapse_whitespace` | Collapse runs of Unicode whitespace. | boolean | `true` | No | +| `preserve_line_breaks` | Preserve line breaks while collapsing other whitespace. | boolean | `false` | No | +| `trim` | Remove leading and trailing whitespace. | boolean | `true` | No | +| Formatting | | | | | +| `separator` | Text used to join multiple input columns into one output. | string | `" "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ + + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `standardize.clean` | +| Lifecycle status | active | +| Namespace | `standardize` | +| Documentation group | `standardize` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.standardize.clean` | + +**Sources** + +- [WranglesPY standardize.clean implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/standardize.py) + +
+ + +--- + +## Standardize Custom {#custom} + +Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. + +### Behavior + +Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. + +This guidance was derived from the callable signature and its embedded Python schema docstring. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use (do not include 'find' and 'replace'). | string, array | — | Yes | + +
+ + + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | Yes | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `standardize.custom` | +| Lifecycle status | active | +| Namespace | `standardize` | +| Documentation group | `standardize` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.standardize.custom` | + +**Sources** + +- [WranglesPY standardize.custom implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/standardize.py) + +
diff --git a/wrangles-docs/registry-docs/namespaces/transform.md b/wrangles-docs/registry-docs/namespaces/transform.md new file mode 100644 index 0000000..ccf873b --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/transform.md @@ -0,0 +1,502 @@ +--- +title: "Transform Wrangles" +description: "Transform wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Transform" +slug: "/namespaces/transform" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Transform Wrangles + +Transform wrangles, with recipe examples, parameters, and behavior. + +## Copy {#copy} + +Make a copy of a column or a list of columns. + +Create a copy of columns in a dataframe. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input columns or columns. | string, integer, array, null | `null` | No | +| `output` | Name of the output columns or columns. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - copy: + input: Product Data + output: Product Data (copy) +``` + +
+ +
+ +| Product Data | +| --- | +| SKF ball brg | +| brg seal | + +
+ +
+ +| Product Data | Product Data (copy) | +| --- | --- | +| SKF ball bearing | SKF ball bearing | +| bearing seal | bearing seal | + +
+ +
+ + + + + +```yaml +wrangles: + - copy: + Product Data: Product Data (copy) +``` + +
+ +
+ +| Product Data | +| --- | +| SKF ball brg | +| brg seal | + +
+ +
+ +| Product Data | Product Data (copy) | +| --- | --- | +| SKF ball bearing | SKF ball bearing | +| bearing seal | bearing seal | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `copy` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.copy` | + +**Sources** + +- [WranglesPY copy implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing copy Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/copy.md) + +
+ + +--- + +## Reindex {#reindex} + +Changes the row labels and column labels of a DataFrame. + +Conform a DataFrame to a new index with optional filling logic. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | New labels for the columns. Preferably an Index object to avoid duplicating data. | array, null | `null` | No | +| Options | | | | | +| `labels` | New labels / index to conform the axis specified by ‘axis’ to. | array, null | `null` | No | +| `index` | New labels for the index. Preferably an Index object to avoid duplicating data. | array, null | `null` | No | +| `axis` | Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1). | string, integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | + +
+ +### Examples + +:::note +Cannot specify both `axis` and any of `index` or `columns`. Reindex is not compatible with `where` filtering. +::: + + + + + +```yaml +wrangles: + - reindex: + index: + - 5 + - 4 + - 3 + - 2 + - 1 +``` + +
+ +
+ +| | Material | Product Data | +| --- | --- | --- | +| 1 | Ceramic | SKF ball brg | +| 2 | Rubber | brg seal | +| 3 | Brass | Ball valve | +| 4 | Ceramic | Ceramic cartridge | +| 5 | Stainless Steel | Needle Bearing | + +
+ +
+ +| | Material | Product Data | +| --- | --- | --- | +| 5 | Stainless Steel | Needle Bearing | +| 4 | Ceramic | Ceramic cartridge | +| 3 | Brass | Ball valve | +| 2 | Rubber | brg seal | +| 1 | Ceramic | SKF ball brg | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `reindex` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.reindex` | + +**Sources** + +- [WranglesPY reindex implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing reindex Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/reindex.md) + +
+ + +--- + +## Rename {#rename} + +Rename a column or list of columns. + + + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array, null | `null` | No | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Execution | | | | | +| `wrangles` | Use wrangles to transform the column names. The input is named 'columns' and the final result must also include the column named 'columns'. This can only be used instead of the standard rename. | array, null | `null` | No | + +
+ +### Examples + +:::note +Rename is not compatible with `where` filtering. +::: + + + + + +```yaml +wrangles: + - rename: + input: + - Manufacturer Name + - Manufacturer Part Number + output: + - Manufacturer + - MPN +``` + +
+ +
+ +| Manufacturer Name | Manufacturer Part Number | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ +| Manufacturer | MPN | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ + + + + +```yaml +wrangles: + - rename: + Manufacturer Name: Manufacturer + Manufacturer Part Number: MPN +``` + +
+ +
+ +| Manufacturer Name | Manufacturer Part Number | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ +| Manufacturer | MPN | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ + + + + +```yaml +wrangles: + - rename: + wrangles: + - convert.case: + input: columns + case: upper +``` + +
+ +
+ +| Manufacturer Name | Manufacturer Part Number | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ +| MANUFACTURER NAME | MANUFACTURER PART NUMBER | +| --- | --- | +| SKF | 302-2 | +| Timken | PF48 | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `rename` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.rename` | + +**Sources** + +- [WranglesPY rename implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing rename Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/rename.md) + +
+ + +--- + +## Transpose {#transpose} + +Transpose the DataFrame (swap columns to rows). + +Transpose a dataframe. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `header_column` | Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header. | string, integer, null | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +:::note +Transpose is not compatible with `where` filtering. +::: + + + + + +```yaml +wrangles: + - transpose: + header_column: Material +``` + +
+ +
+ +| Material | Product Data | +| --- | --- | +| Ceramic | SKF ball brg | +| Rubber | brg seal | + +
+ +
+ +| Material | Ceramic | Rubber | +| --- | --- | --- | +| Product Data | SKF ball brg | brg seal | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `transpose` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.transpose` | + +**Sources** + +- [WranglesPY transpose implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) +- [Existing transpose Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/transpose.md) + +
diff --git a/wrangles-docs/registry-docs/namespaces/utility.md b/wrangles-docs/registry-docs/namespaces/utility.md new file mode 100644 index 0000000..6a65a09 --- /dev/null +++ b/wrangles-docs/registry-docs/namespaces/utility.md @@ -0,0 +1,795 @@ +--- +title: "Utility Wrangles" +description: "Utility wrangles, with recipe examples, parameters, and behavior." +sidebar_label: "Utility" +slug: "/namespaces/utility" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# Utility Wrangles + +Utility wrangles, with recipe examples, parameters, and behavior. + +## Accordion {#accordion} + +Apply a series of wrangles to column(s) containing lists. The wrangles will be applied to each element in the list and the results will be returned back as a list. + +Apply a series of wrangles to the individual elements of one or more lists. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | The column(s) containing the list(s) that the wrangles will be applied to the elements of. | string, integer, array | — | Yes | +| `output` | Output of the wrangles to save back to the dataframe. | string, array, null | `null` | No | +| `propagate` | Limit the column(s) that will be available to the wrangles and replicated for each element. If not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | List of wrangles to apply. | array | — | Yes | + +
+ +### Examples + +This example applies `convert.case` to each string in a list, where the wrangle would not normally operate on the list as a whole. + + + +```yaml +wrangles: + - accordion: + input: list_column + output: modified_lists + wrangles: + - convert.case: + input: list_column + output: modified_lists + case: upper +``` + +
+ +
+ +| list_column | +| --- | +| ["a", "b", "c"] | +| ["e", "f", "g"] | + +
+ +
+ +| modified_lists | +| --- | +| ["A", "B", "C"] | +| ["E", "F", "G"] | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `accordion` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.accordion` | + +**Sources** + +- [WranglesPY accordion implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing accordion Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/accordion.md) + +
+ + +--- + +## Batch {#batch} + +Split the data into batches for executing a list of wrangles. Use this in situations such as where the intermediate data is too large to fit in memory. + +Execute a series of wrangles in batches. Batches can run in parallel with `threads` and can provide fallback output when an error occurs. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size. | array | — | Yes | +| `batch_size` | The number of rows to split each batch into. | integer | `1000` | No | +| `threads` | The number of threads to use for parallel processing. Default 1. | integer | `1` | No | +| `timeout` | The number of seconds to wait for a batch to complete before raising an error. | number, null | `null` | No | +| `use_multiprocessing` | Use process-based workers instead of threads. This is an advanced runtime option. | boolean | `false` | No | +| Errors | | | | | +| `on_error` | A dictionary of column_name: value to return if an error occurs while attempting to run a batch. | object, null | `null` | No | + +
+ +### Examples + +This example processes product descriptions in batches of two rows. + + + +```yaml +wrangles: + - batch: + batch_size: 2 + threads: 1 + wrangles: + - extract.ai: + api_key: Your OpenAI API key + input: Product Description + output: + Title: + type: string + description: Title of the product +``` + +
+ +
+ +| Product Description | +| --- | +| Sleep better with our Memory Foam Pillow, designed to contour to your head and neck. | +| Stay comfortable and stylish with our Organic Cotton T-Shirt, made from soft, breathable fabric. | +| Keep drinks hot or cold with our Stainless Steel Water Bottle, featuring durable insulation. | +| Enjoy crisp sound and long battery life with our Wireless Bluetooth Earbuds. | + +
+ +
+ +| Title | +| --- | +| Memory Foam Pillow | +| Organic Cotton T-Shirt | +| Stainless Steel Water Bottle | +| Wireless Bluetooth Earbuds | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `batch` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.batch` | + +**Sources** + +- [WranglesPY batch implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing batch Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/batch.md) + +
+ + +--- + +## Concurrent {#concurrent} + +Run multiple wrangles concurrently rather than sequentially. Wrangles must specify output columns to be used concurrently. When using concurrent, Wrangles may not complete in a predictable order and it is not recommended to update overlapping columns with different wrangles. + +Run multiple wrangles concurrently instead of sequentially. Concurrent wrangles must declare output columns, may finish in any order, and should not update overlapping columns. + +See the [Concurrent connector](/python/connectors/concurrent) for the connector equivalent. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | The wrangles section of a recipe to execute for each combination of variables. | array | — | Yes | +| `max_concurrency` | The maximum number of wrangles to execute in parallel. | integer | `10` | No | +| `use_multiprocessing` | Use process-based workers instead of threads. This is an advanced runtime option. | boolean | `false` | No | + +
+ +### Examples + +```yaml +wrangles: + - concurrent: + wrangles: + - extract.codes: + input: Products + output: Part Codes + - extract.attributes: + input: Products + output: Attributes +``` + +
+ +
+ +| Products | +| --- | +| SKF ball brg 2" od 6202 | +| brg seal 1" id 5493 | +| 3lb hammer 87102 | + +
+ +
+ +| Part Codes | Attributes | +| --- | --- | +| 6202 | `{"length":["2in"]}` | +| 5493 | `{"length":["1in"]}` | +| 87102 | `{"weight":["3lb"]}` | + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `concurrent` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.concurrent` | + +**Sources** + +- [WranglesPY concurrent implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing concurrent Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/concurrent.md) + +
+ + +--- + +## Log {#log} + +Log the current status of the dataframe. + +Print the current status of the dataframe. Only a sample of rows is logged. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | (Optional, default all columns) List of specific columns to log. | array, null | `null` | No | +| Options | | | | | +| `write` | (Optional) Allows for an intermediate output to a file/dataframe/database etc. | array, null | `null` | No | +| `warning` | Log a warning to the console. | string, null | `null` | No | +| `info` | Log info to the console. | string, null | `null` | No | +| `log_data` | Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise. | boolean, null | `null` | No | +| `error` | Log an error to the console. | string | — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - log: {} +``` + +
+ +
+ +_Logging side-effect; no tabular input sample._ + +
+ +
+ +_Logs a sample of rows or status information to the configured destination._ + +
+ +
+ + + + + +```yaml +wrangles: + - log: + columns: + - column1 + - column2 +``` + +
+ +
+ +_Logging side-effect; no tabular input sample._ + +
+ +
+ +_Logs a sample of rows or status information to the configured destination._ + +
+ +
+ + + + + +```yaml +wrangles: + - log: + write: + - file: + name: output/filepath + columns: + - column 1 + - column 2 +``` + +
+ +
+ +_Logging side-effect; no tabular input sample._ + +
+ +
+ +_Logs a sample of rows or status information to the configured destination._ + +
+ +
+ + + +| Variable | Function | +| --- | --- | +| `${column_count}` | Number of columns. | +| `${columns}` | List of all columns. | +| `${df}` | Current dataframe. | +| `${row_count}` | Number of visible rows processed in each batch. | + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `log` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.log` | + +**Sources** + +- [WranglesPY log implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing log Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/log.md) + +
+ + +--- + +## Matrix {#matrix} + +Apply a matrix of wrangles to the dataframe. This will run the wrangles for each combination of the variables. + +Apply a matrix of wrangles to the dataframe. Each wrangle runs for the configured combinations of variables, including recipe variables and variables declared by the `variables` parameter. + +See the [Matrix connector](/python/connectors/matrix) for the connector equivalent. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `strategy` | Determines how to combine variables when there are multiple. loop (default) iterates over each set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables. | string; one of:
  • permutations
  • loop
| `"loop"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `variables` | A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values. | object | — | Yes | +| `wrangles` | The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables. | array | — | Yes | + +
+ +### Examples + +Run a custom function once for each configured variable value. + + + +```yaml +wrangles: + - matrix: + variables: + var: [A, B, C] + wrangles: + - custom.test_fn: + input: Part Code + output: Part Code ${var} + value: ${var} +``` + +```python +def test_fn(part_code, value): + return part_code + value +``` + +
+ +
+ +| Part Code | +| --- | +| 6202 | +| br549 | +| 554-114 | +| 554-112 | + +
+ +
+ +| Part Code A | Part Code B | Part Code C | +| --- | --- | --- | +| 6202A | 6202B | 6202C | +| br549A | br549B | br549C | +| 554-114A | 554-114B | 554-114C | +| 554-112A | 554-112B | 554-112C | + +
+ +
+ + + +This example runs `extract.custom` once for each unique model ID. + + + +```yaml +wrangles: + - matrix: + variables: + model_id: set(Model ID) + wrangles: + - extract.custom: + input: Description + output: Extracted Values + model_id: ${model_id} + where: "[Model ID] = ?" + where_params: + - ${model_id} +``` + +
+ +
+ +| Description | Model ID | +| --- | --- | +| The SKF 6202 bearing is the best bearing in the world | xxxxxxxx-xxxx-xxxx | +| The Timken 6102 bearing is indestructible | yyyyyyyy-yyyy-yyyy | +| The Milwaukee impact has 1200ft-lbs of torque | zzzzzzzz-zzzz-zzzz | + +
+ +
+ +| Extracted Values | +| --- | +| 6202 | +| indestructible | +| 1200ft-lbs | + +
+ +
+ + + +| Variable | Function | +| --- | --- | +| `${column_count}` | Number of columns. | +| `${columns}` | List of all columns. | +| `${df}` | Current dataframe. | +| `${row_count}` | Number of visible rows processed in each batch. | + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `matrix` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.matrix` | + +**Sources** + +- [WranglesPY matrix implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing matrix Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/matrix.md) + +
+ + +--- + +## Recipe {#recipe} + +Run another recipe as a wrangle against the current dataframe. + +### Behavior + +Run another recipe as a wrangle against the current dataframe. + +This guidance was derived from the callable signature and its embedded Python schema docstring. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array, null | `null` | No | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `name` | File name of the recipe. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `variables` | A dictionary of variables to pass to the recipe. | object, null | `null` | No | + +
+ + + +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `recipe` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.recipe` | + +**Sources** + +- [WranglesPY recipe implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) + +
+ + +--- + +## Try {#try} + +Try a list of wrangles and catch any errors that occur. + +Run a list of wrangles and catch errors. When `except` is provided, its wrangles or fallback column values run after an error; otherwise, the error is logged and the recipe continues. + +### Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | List of wrangles to apply. | array | — | Yes | +| Errors | | | | | +| `retries` | Number of times to retry the wrangles if an error occurs. Default 0. | integer | `0` | No | +| `except` | An action to take if the wrangles encounter an error. This can contain a list of wrangles or a dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue. | object | — | No | + +
+ +### Examples + +```yaml +wrangles: + - try: + wrangles: + - risky_wrangle: + input: column + except: + - backup_wrangle: + input: column +``` + +
+ +
+ +_The input dataframe is passed to the primary wrangles._ + +
+ +
+ +_The primary result is returned on success; the `except` result is returned after an error._ + +
+ +
+ +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `try` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.Try` | + +**Sources** + +- [WranglesPY try implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) +- [Existing try Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/try.md) + +
diff --git a/wrangles-docs/registry-docs/python.md b/wrangles-docs/registry-docs/python.md index 399faa6..0688f97 100644 --- a/wrangles-docs/registry-docs/python.md +++ b/wrangles-docs/registry-docs/python.md @@ -3,51 +3,42 @@ title: "Python" description: "Apply a simple single-line python command. For more complex python use a custom function. Note, this evaluates the python command - be especially cautious including variables from untrusted sources within the command string. The python command will be evaluated once for each row and the result returned. Reference column values by using their name. Non-alphanumeric characters within column names are replaced by underscores (_) Additionally, all columns are available as a dict named kwargs. Additional parameters set for the wrangle will also be available to the command." sidebar_label: "Python" slug: "/python" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Python Apply a simple single-line python command. For more complex python use a custom function. Note, this evaluates the python command - be especially cautious including variables from untrusted sources within the command string. The python command will be evaluated once for each row and the result returned. Reference column values by using their name. Non-alphanumeric characters within column names are replaced by underscores (_) Additionally, all columns are available as a dict named kwargs. Additional parameters set for the wrangle will also be available to the command. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `command` | Yes | string | Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_). | — | -| `output` | Yes | string, array | Name or list of output column(s). To output multiple columns, return a list of the corresponding length. | — | -| `input` | No | string, integer, array, null | Name or list of input column(s) to filter the data available to the command. Useful in conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available. | `null` | -| `except` | No | string, array, number, integer, boolean, object | Value to return for the row if an exception occurs during the evaluation. If not provided, an exception will be raised as normal. If multiple output columns are specified, this must match the length. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - The Python wrangle executes simple Python commands inline within a recipe. Row values are referenced by column name and commands are evaluated once per row. Spaces within column names are replaced by underscores (`_`). All columns are also available as a dictionary named `kwargs`. For more complex Python, use custom functions. :::caution This wrangle evaluates the Python command. Be cautious when including variables from untrusted sources in the command string. ::: -## Migrated examples -#### Python Wrangle +## Parameters -##### Recipe +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `output` | Name or list of output column(s). To output multiple columns, return a list of the corresponding length. | string, array | — | Yes | +| `input` | Name or list of input column(s) to filter the data available to the command. Useful in conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available. | string, integer, array, null | `null` | No | +| Options | | | | | +| `command` | Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_). | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `except` | Value to return for the row if an exception occurs during the evaluation. If not provided, an exception will be raised as normal. If multiple output columns are specified, this must match the length. | string, array, number, integer, boolean, object | — | No | + +
+ +## Examples ```yaml wrangles: @@ -58,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| My Column | | --- | @@ -68,23 +57,21 @@ wrangles:
-
+
-##### Output Sample - -| My Column | result | -| --- | --- | -| example text | EXAMPLE TEXT | +| result | +| --- | +| EXAMPLE TEXT |
-#### Including Your Own Parameters + The Python wrangle supports parameters so values of unknown origin can be injected safely. -##### Recipe + ```yaml wrangles: @@ -96,17 +83,13 @@ wrangles:
-
- -##### Input Sample +
_No sample available._
-
- -##### Output Sample +
_No sample available._ @@ -114,17 +97,35 @@ _No sample available._
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `python` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.python` | + +**Sources** - [WranglesPY python implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing python Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/python.md) -## Registry metadata - -- Registry ID: `c0398a11-7731-4e47-8df0-b07eea0b1d6c` -- Namespace: root-level runtime key -- Recipe key: `python` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.python` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/recipe.md b/wrangles-docs/registry-docs/recipe.md index add4f58..0824f71 100644 --- a/wrangles-docs/registry-docs/recipe.md +++ b/wrangles-docs/registry-docs/recipe.md @@ -3,57 +3,71 @@ title: "Recipe" description: "Run another recipe as a wrangle against the current dataframe." sidebar_label: "Recipe" slug: "/recipe" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Recipe Run another recipe as a wrangle against the current dataframe. -> Pilot Registry entry. Runtime contract status: `verified`. +## Behavior + +Run another recipe as a wrangle against the current dataframe. + +This guidance was derived from the callable signature and its embedded Python schema docstring. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | +
+ +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | -| `input` | No | string, integer, array, null | Name, index, or list of input columns. | `null` | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `name` | No | string, null | File name of the recipe. | `null` | -| `variables` | No | object, null | A dictionary of variables to pass to the recipe. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | +| I/O | | | | | +| `input` | Name, index, or list of input columns. | string, integer, array, null | `null` | No | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `name` | File name of the recipe. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `variables` | A dictionary of variables to pass to the recipe. | object, null | `null` | No | + +
-## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-## Behavior +
-Run another recipe as a wrangle against the current dataframe. +Technical details -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +| Field | Value | +| --- | --- | +| Recipe key | `recipe` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.recipe` | -## Provenance +**Sources** - [WranglesPY recipe implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `recipe` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.recipe` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/reindex.md b/wrangles-docs/registry-docs/reindex.md index 30895d5..f19161e 100644 --- a/wrangles-docs/registry-docs/reindex.md +++ b/wrangles-docs/registry-docs/reindex.md @@ -3,49 +3,43 @@ title: "Reindex" description: "Changes the row labels and column labels of a DataFrame." sidebar_label: "Reindex" slug: "/reindex" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Reindex Changes the row labels and column labels of a DataFrame. -> Pilot Registry entry. Runtime contract status: `verified`. +Conform a DataFrame to a new index with optional filling logic. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `labels` | No | array, null | New labels / index to conform the axis specified by ‘axis’ to. | `null` | -| `index` | No | array, null | New labels for the index. Preferably an Index object to avoid duplicating data. | `null` | -| `columns` | No | array, null | New labels for the columns. Preferably an Index object to avoid duplicating data. | `null` | -| `axis` | No | string, integer, null | Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `columns` | New labels for the columns. Preferably an Index object to avoid duplicating data. | array, null | `null` | No | +| Options | | | | | +| `labels` | New labels / index to conform the axis specified by ‘axis’ to. | array, null | `null` | No | +| `index` | New labels for the index. Preferably an Index object to avoid duplicating data. | array, null | `null` | No | +| `axis` | Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1). | string, integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | -## Guidance +
-Conform a DataFrame to a new index with optional filling logic. +## Examples -## Migrated examples :::note Cannot specify both `axis` and any of `index` or `columns`. Reindex is not compatible with `where` filtering. ::: -#### Reindexing a Dataframe -##### Recipe + + ```yaml wrangles: @@ -60,9 +54,7 @@ wrangles:
-
- -##### Input Sample +
| | Material | Product Data | | --- | --- | --- | @@ -74,9 +66,7 @@ wrangles:
-
- -##### Output Sample +
| | Material | Product Data | | --- | --- | --- | @@ -90,17 +80,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `reindex` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.reindex` | + +**Sources** - [WranglesPY reindex implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing reindex Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/reindex.md) -## Registry metadata - -- Registry ID: `def87df8-72da-4e34-83c1-1fde25126257` -- Namespace: root-level runtime key -- Recipe key: `reindex` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.reindex` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/remove-words.md b/wrangles-docs/registry-docs/remove-words.md index 6d2ba7b..565ff96 100644 --- a/wrangles-docs/registry-docs/remove-words.md +++ b/wrangles-docs/registry-docs/remove-words.md @@ -3,48 +3,38 @@ title: "Remove Words" description: "Remove all the elements that occur in one list from another." sidebar_label: "Remove Words" slug: "/remove-words" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Remove Words Remove all the elements that occur in one list from another. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of column to remove words from. | — | -| `to_remove` | Yes | string | Column or list of columns with a list of words to be removed. | — | -| `output` | No | string, array, null | Name of the output columns. | `null` | -| `tokenize_to_remove` | No | boolean | Tokenize all to_remove inputs. | `false` | -| `ignore_case` | No | boolean | Ignore input and to_remove case. | `true` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Remove all the elements that occur in one list from another. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of column to remove words from. | string, integer, array | — | Yes | +| `to_remove` | Column or list of columns with a list of words to be removed. | string | — | Yes | +| `output` | Name of the output columns. | string, array, null | `null` | No | +| Options | | | | | +| `tokenize_to_remove` | Tokenize all to_remove inputs. | boolean | `false` | No | +| `ignore_case` | Ignore input and to_remove case. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Removing Words From a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -60,9 +50,7 @@ wrangles:
-
- -##### Input Sample +
| Colours | Materials | Description | | --- | --- | --- | @@ -71,9 +59,7 @@ wrangles:
-
- -##### Output Sample +
| Product | | --- | @@ -84,17 +70,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `remove_words` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.remove_words` | + +**Sources** - [WranglesPY remove_words implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing remove_words Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/remove-words.md) -## Registry metadata - -- Registry ID: `543b96c3-f354-48be-8046-bf0cb9fbaf56` -- Namespace: root-level runtime key -- Recipe key: `remove_words` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.remove_words` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/rename.md b/wrangles-docs/registry-docs/rename.md index ddc28e1..613f9df 100644 --- a/wrangles-docs/registry-docs/rename.md +++ b/wrangles-docs/registry-docs/rename.md @@ -3,48 +3,42 @@ title: "Rename" description: "Rename a column or list of columns." sidebar_label: "Rename" slug: "/rename" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Rename Rename a column or list of columns. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | No | string, integer, array, null | Name or list of input columns. | `null` | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `wrangles` | No | array, null | Use wrangles to transform the column names. The input is named 'columns' and the final result must also include the column named 'columns'. This can only be used instead of the standard rename. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -## Verified examples +## Parameters -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array, null | `null` | No | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| Execution | | | | | +| `wrangles` | Use wrangles to transform the column names. The input is named 'columns' and the final result must also include the column named 'columns'. This can only be used instead of the standard rename. | array, null | `null` | No | -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +## Examples -Rename a column or list of columns. - -## Migrated examples :::note Rename is not compatible with `where` filtering. ::: -#### Renaming Columns With Input and Output -##### Recipe + + ```yaml wrangles: @@ -59,9 +53,7 @@ wrangles:
-
- -##### Input Sample +
| Manufacturer Name | Manufacturer Part Number | | --- | --- | @@ -70,9 +62,7 @@ wrangles:
-
- -##### Output Sample +
| Manufacturer | MPN | | --- | --- | @@ -83,9 +73,9 @@ wrangles:
-#### Renaming Columns Without Using Input and Output -##### Recipe + + ```yaml wrangles: @@ -96,9 +86,7 @@ wrangles:
-
- -##### Input Sample +
| Manufacturer Name | Manufacturer Part Number | | --- | --- | @@ -107,9 +95,7 @@ wrangles:
-
- -##### Output Sample +
| Manufacturer | MPN | | --- | --- | @@ -120,9 +106,9 @@ wrangles:
-#### Using Wrangles in Rename -##### Recipe + + ```yaml wrangles: @@ -135,9 +121,7 @@ wrangles:
-
- -##### Input Sample +
| Manufacturer Name | Manufacturer Part Number | | --- | --- | @@ -146,9 +130,7 @@ wrangles:
-
- -##### Output Sample +
| MANUFACTURER NAME | MANUFACTURER PART NUMBER | | --- | --- | @@ -159,17 +141,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `rename` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.rename` | + +**Sources** - [WranglesPY rename implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing rename Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/rename.md) -## Registry metadata - -- Registry ID: `ab06898f-faf7-42e7-8275-5e3034a4d727` -- Namespace: root-level runtime key -- Recipe key: `rename` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.rename` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/replace.md b/wrangles-docs/registry-docs/replace.md index 8d7dadd..da21b96 100644 --- a/wrangles-docs/registry-docs/replace.md +++ b/wrangles-docs/registry-docs/replace.md @@ -3,51 +3,41 @@ title: "Replace" description: "Quick find and replace for simple values. Can use regex if 'input' in params and isinstance(params['input'], list):in the find field." sidebar_label: "Replace" slug: "/replace" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Replace Quick find and replace for simple values. Can use regex if 'input' in params and isinstance(params['input'], list):in the find field. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input column. | — | -| `find` | Yes | string | Pattern to find using regex. | — | -| `replace` | Yes | string | Value to replace the pattern found. | — | -| `output` | No | string, array, null | Name or list of output column. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Quick find and replace for simple values. Can use regex in the `find` field. :::note Values that are not a number or a string pass through unaltered. ::: -## Migrated examples -#### Replacing an Abbreviation +## Parameters + +
-##### Recipe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input column. | string, integer, array | — | Yes | +| `output` | Name or list of output column. | string, array, null | `null` | No | +| Options | | | | | +| `find` | Pattern to find using regex. | string | — | Yes | +| `replace` | Value to replace the pattern found. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +## Examples ```yaml wrangles: @@ -59,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| Product Data | | --- | @@ -70,9 +58,7 @@ wrangles:
-
- -##### Output Sample +
| Product Data | | --- | @@ -83,17 +69,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `replace` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.replace` | + +**Sources** - [WranglesPY replace implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing replace Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/replace.md) -## Registry metadata - -- Registry ID: `f0ab715e-9e0e-4614-83e4-5cd8ea08a09f` -- Namespace: root-level runtime key -- Recipe key: `replace` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.replace` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/round.md b/wrangles-docs/registry-docs/round.md index 0d2d461..faffd8e 100644 --- a/wrangles-docs/registry-docs/round.md +++ b/wrangles-docs/registry-docs/round.md @@ -3,46 +3,36 @@ title: "Round" description: "Round column(s) to the specified decimals." sidebar_label: "Round" slug: "/round" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Round Round column(s) to the specified decimals. -> Pilot Registry entry. Runtime contract status: `verified`. +Round numbers in a column to the nearest decimal point of your choosing. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column(s). | — | -| `decimals` | No | integer | Number of decimal places to round column. | `0` | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Round numbers in a column to the nearest decimal point of your choosing. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Formatting | | | | | +| `decimals` | Number of decimal places to round column. | integer | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Rounding a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Cost Per Unit | | --- | @@ -65,30 +53,46 @@ wrangles:
-
- -##### Output Sample +
-| Cost Per Unit | Cost Rounded | -| --- | --- | -| 3.14159 | 3.14 | -| 2.71828 | 2.72 | +| Cost Rounded | +| --- | +| 3.14 | +| 2.72 |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `round` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.round` | + +**Sources** - [WranglesPY round implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing round Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/round.md) -## Registry metadata - -- Registry ID: `12f3111b-8511-4e42-8d3f-b5302dc3b4e4` -- Namespace: root-level runtime key -- Recipe key: `round` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.round` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/search/find-links.md b/wrangles-docs/registry-docs/search/find-links.md index 1e8820a..5dc0cbb 100644 --- a/wrangles-docs/registry-docs/search/find-links.md +++ b/wrangles-docs/registry-docs/search/find-links.md @@ -3,56 +3,50 @@ title: "Find Links" description: "Perform web searches to find links. Returns structured search results with titles, links, snippets, and optional pricing." sidebar_label: "Find Links" slug: "/search/find-links" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Find Links Perform web searches to find links. Returns structured search results with titles, links, snippets, and optional pricing. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `queries` | Yes | string, array | Name or list of input columns containing search queries. | — | -| `id` | Yes | string | Name of the column containing the row ID to append to each search result. | — | -| `output` | No | string, array, null | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | `null` | -| `client` | No | string; one of: serpapi | The search provider to use. | `"serpapi"` | -| `api_key` | No | string, null | API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY). | `null` | -| `n_results` | No | integer | Number of search results to return per query (default 10, max 100). | `10` | -| `threads` | No | integer | Number of concurrent threads for parallel processing (default 10). | `10` | -| `country` | No | string | Country code for search results (default 'us'). Alias: gl. | — | -| `language` | No | string | Language code for search results (default 'en'). Alias: hl. | — | -| `location` | No | string | Location for search results (e.g., 'Austin, Texas'). | — | -| `device` | No | string; one of: desktop, mobile, tablet | Device type for search results. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +## Parameters -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `queries` | Name or list of input columns containing search queries. | string, array | — | Yes | +| `id` | Name of the column containing the row ID to append to each search result. | string | — | Yes | +| `output` | Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. | string, array, null | `null` | No | +| Options | | | | | +| `n_results` | Number of search results to return per query (default 10, max 100). | integer | `10` | No | +| `country` | Country code for search results (default 'us'). Alias: gl. | string | — | No | +| `language` | Language code for search results (default 'en'). Alias: hl. | string | — | No | +| `location` | Location for search results (e.g., 'Austin, Texas'). | string | — | No | +| `device` | Device type for search results. | string; one of:
  • desktop
  • mobile
  • tablet
| — | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Number of concurrent threads for parallel processing (default 10). | integer | `10` | No | +| Details | | | | | +| `client` | The search provider to use. | string; one of:
  • serpapi
| `"serpapi"` | No | +| `api_key` | API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY). | string, null | `null` | No | -Perform web searches to find links. Returns structured search results with titles, links, snippets, and optional pricing. +
-## Migrated examples -#### Find Product Links +## Examples This template searches once per query and returns structured result dictionaries. Search results vary by provider, location, and time. -##### Recipe + ```yaml wrangles: @@ -69,9 +63,7 @@ wrangles:
-
- -##### Input Sample +
| Row ID | Search Query | | --- | --- | @@ -79,13 +71,11 @@ wrangles:
-
- -##### Output Sample +
-| Row ID | Search Query | Search Results | -| --- | --- | --- | -| 1001 | SKF 6202 bearing supplier | `[{"title": "Example result", "link": "https://example.com/6202", "snippet": "SKF 6202 bearing"}]` | +| Search Results | +| --- | +| `[{"title": "Example result", "link": "https://example.com/6202", "snippet": "SKF 6202 bearing"}]` |
@@ -93,17 +83,35 @@ wrangles: _Template based on the documented parameters; no published source example is currently available._ -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `search.find_links` | +| Lifecycle status | active | +| Namespace | `search` | +| Documentation group | `search` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.search.find_links` | + +**Sources** - [WranglesPY search.find_links implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/search.py) - [Existing search.find_links Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/search/_sources/find-links.md) -## Registry metadata - -- Registry ID: `ec1f661b-2cbc-4a47-88e3-646179104376` -- Namespace: `search` -- Recipe key: `search.find_links` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.search.find_links` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/search/retrieve-link-content.md b/wrangles-docs/registry-docs/search/retrieve-link-content.md index 577ddda..dafae77 100644 --- a/wrangles-docs/registry-docs/search/retrieve-link-content.md +++ b/wrangles-docs/registry-docs/search/retrieve-link-content.md @@ -3,53 +3,48 @@ title: "Retrieve Link Content" description: "Retrieves targeted content from web pages using LLM URL extraction. Can optionally output a second column containing a clean, human-readable text summary of the retrieved data." sidebar_label: "Retrieve Link Content" slug: "/search/retrieve-link-content" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Retrieve Link Content Retrieves targeted content from web pages using LLM URL extraction. Can optionally output a second column containing a clean, human-readable text summary of the retrieved data. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, array | Name or list of input columns containing URLs or Scored Search Result dictionaries. | — | -| `output` | No | string, array, null | Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]). | `null` | -| `client` | No | string; one of: google_url_context | The retrieval provider to use. | `"google_url_context"` | -| `api_key` | No | string, null | API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY). | `null` | -| `prompt` | No | string, null | Optional custom system prompt to guide the extraction behavior and output format. | `null` | -| `model_id` | No | string | The specific model ID to use (default models/gemini-3-flash-preview). | `"models/gemini-3-flash-preview"` | -| `output_format` | No | string; one of: markdown, json | The desired format for the extracted content. | `"json"` | -| `threads` | No | integer | Number of concurrent threads for parallel processing (default 10). | `10` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +## Parameters -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns containing URLs or Scored Search Result dictionaries. | string, array | — | Yes | +| `output` | Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]). | string, array, null | `null` | No | +| Options | | | | | +| `prompt` | Optional custom system prompt to guide the extraction behavior and output format. | string, null | `null` | No | +| Formatting | | | | | +| `output_format` | The desired format for the extracted content. | string; one of:
  • markdown
  • json
| `"json"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `threads` | Number of concurrent threads for parallel processing (default 10). | integer | `10` | No | +| Details | | | | | +| `client` | The retrieval provider to use. | string; one of:
  • google_url_context
| `"google_url_context"` | No | +| `api_key` | API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY). | string, null | `null` | No | +| `model_id` | The specific model ID to use (default models/gemini-3-flash-preview). | string | `"models/gemini-3-flash-preview"` | No | -Retrieves targeted content from web pages using LLM URL extraction. Can optionally output a second column containing a clean, human-readable text summary of the retrieved data. +
-## Migrated examples -#### Retrieve Structured Page Content +## Examples This template extracts JSON content from a URL. Returned fields depend on the page, prompt, and retrieval model. -##### Recipe + ```yaml wrangles: @@ -66,9 +61,7 @@ wrangles:
-
- -##### Input Sample +
| Product URL | | --- | @@ -76,13 +69,11 @@ wrangles:
-
- -##### Output Sample +
-| Product URL | Page Data | -| --- | --- | -| https://example.com/products/6202 | `{"title": "6202 Bearing", "manufacturer": "SKF"}` | +| Page Data | +| --- | +| `{"title": "6202 Bearing", "manufacturer": "SKF"}` |
@@ -90,17 +81,35 @@ wrangles: _Template based on the documented parameters; no published source example is currently available._ -## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `search.retrieve_link_content` | +| Lifecycle status | active | +| Namespace | `search` | +| Documentation group | `search` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.search.retrieve_link_content` | + +**Sources** - [WranglesPY search.retrieve_link_content implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/search.py) - [Existing search.retrieve_link_content Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/search/_sources/retrieve-link-content.md) -## Registry metadata - -- Registry ID: `3d5faa87-9e96-48c0-8226-5ae566e8d76c` -- Namespace: `search` -- Recipe key: `search.retrieve_link_content` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.search.retrieve_link_content` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/columns.md b/wrangles-docs/registry-docs/select/columns.md index 8d30ba7..b8c0b20 100644 --- a/wrangles-docs/registry-docs/select/columns.md +++ b/wrangles-docs/registry-docs/select/columns.md @@ -3,44 +3,33 @@ title: "Columns" description: "Select columns from the dataframe." sidebar_label: "Columns" slug: "/select/columns" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Columns Select columns from the dataframe. -> Pilot Registry entry. Runtime contract status: `verified`. +Select columns from the dataframe ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column(s) to select. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Select columns from the dataframe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to select. | string, integer, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting Columns +
-##### Recipe +## Examples ```yaml wrangles: @@ -50,9 +39,7 @@ wrangles:
-
- -##### Input Sample +
| Part Number | Manufacturer | | --- | --- | @@ -61,9 +48,7 @@ wrangles:
-
- -##### Output Sample +
| Manufacturer | | --- | @@ -74,17 +59,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.columns` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.columns` | + +**Sources** - [WranglesPY select.columns implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.columns Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/columns.md) -## Registry metadata - -- Registry ID: `bca12c60-3957-4dc5-83f7-5ee460df2a11` -- Namespace: `select` -- Recipe key: `select.columns` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.columns` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/dictionary-element.md b/wrangles-docs/registry-docs/select/dictionary-element.md index 5c0a3e3..c872aca 100644 --- a/wrangles-docs/registry-docs/select/dictionary-element.md +++ b/wrangles-docs/registry-docs/select/dictionary-element.md @@ -3,47 +3,38 @@ title: "Dictionary Element" description: "Select one or more element of a dictionary." sidebar_label: "Dictionary Element" slug: "/select/dictionary-element" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Dictionary Element Select one or more element of a dictionary. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `element` | Yes | string | The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary. | — | -| `output` | No | string, array, null | Name of the output column. If omitted, the input column will be replaced. | `null` | -| `default` | No | string, number, array, object, boolean, null | Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set. | `""` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Select one or more element of a dictionary. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. If omitted, the input column will be replaced. | string, array, null | `null` | No | +| Options | | | | | +| `element` | The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary. | string | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set. | string, number, array, object, boolean, null | `""` | No | -## Migrated examples -#### Selecting Dictionary Element Using Where +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,9 +48,7 @@ wrangles:
-
- -##### Input Sample +
| Properties | Part Number | | --- | --- | @@ -68,9 +57,7 @@ wrangles:
-
- -##### Output Sample +
| Shapes | | --- | @@ -81,9 +68,9 @@ wrangles:
-#### Selecting Multiple Dictionary Elements -##### Recipe + + ```yaml wrangles: @@ -96,9 +83,7 @@ wrangles:
-
- -##### Input Sample +
| Properties | | --- | @@ -107,9 +92,7 @@ wrangles:
-
- -##### Output Sample +
| Properties | | --- | @@ -120,9 +103,9 @@ wrangles:
-#### Selecting Multiple Dictionary Elements Using a Wildcard -##### Recipe + + ```yaml wrangles: @@ -135,9 +118,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -145,9 +126,7 @@ wrangles:
-
- -##### Output Sample +
| Column | Output Dict | | --- | --- | @@ -157,9 +136,9 @@ wrangles:
-#### Selecting Multiple Dictionary Elements Using Regex -##### Recipe + + ```yaml wrangles: @@ -172,9 +151,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -182,21 +159,19 @@ wrangles:
-
- -##### Output Sample +
-| Column | Output Dict | -| --- | --- | -| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | \{'Col2': 'B'\} | +| Output Dict | +| --- | +| \{'Col2': 'B'\} |
-#### Selecting Elements While Renaming -##### Recipe + + ```yaml wrangles: @@ -210,9 +185,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -220,21 +193,19 @@ wrangles:
-
- -##### Output Sample +
-| Column | Output Dict | -| --- | --- | -| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | \{'Column 1': 'A', 'Column 2': 'B'\} | +| Output Dict | +| --- | +| \{'Column 1': 'A', 'Column 2': 'B'\} |
-#### Using Default to Fill Missing Elements -##### Recipe + + ```yaml wrangles: @@ -251,9 +222,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -262,30 +231,46 @@ wrangles:
-
- -##### Output Sample +
-| Column | Output Dict | -| --- | --- | -| \{'Col1': 'A', 'Col2': 'B', 'Col3': 'C'\} | \{'Col1': 'A', 'Col3': 'C'\} | -| \{'Col1': 'D', 'Col2': 'E'\} | \{'Col1': 'D', 'Col3': 'Y'\} | +| Output Dict | +| --- | +| \{'Col1': 'A', 'Col3': 'C'\} | +| \{'Col1': 'D', 'Col3': 'Y'\} |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.dictionary_element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.dictionary_element` | + +**Sources** - [WranglesPY select.dictionary_element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.dictionary_element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/dictionary-element.md) -## Registry metadata - -- Registry ID: `5adaeada-6da8-464f-84cb-9fb5ecc17e48` -- Namespace: `select` -- Recipe key: `select.dictionary_element` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.dictionary_element` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/element.md b/wrangles-docs/registry-docs/select/element.md index 4211766..dbe0083 100644 --- a/wrangles-docs/registry-docs/select/element.md +++ b/wrangles-docs/registry-docs/select/element.md @@ -3,46 +3,36 @@ title: "Element" description: "Select elements of lists or dicts using python syntax like col[0]['key']." sidebar_label: "Element" slug: "/select/element" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Element Select elements of lists or dicts using python syntax like col[0]['key']. -> Pilot Registry entry. Runtime contract status: `verified`. +Select elements of lists or dictionaries using Python syntax like `col[1:3]['key']`. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{"key":"val"}] -> "val". | — | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `default` | No | string, number, array, object, boolean, null | Set the default value to return if the specified element doesn't exist. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Select elements of lists or dictionaries using Python syntax like `col[1:3]['key']`. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{"key":"val"}] -> "val". | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. | string, number, array, object, boolean, null | `null` | No | -## Migrated examples -#### Selecting The First Element +
-##### Recipe +## Examples ```yaml wrangles: @@ -53,9 +43,7 @@ wrangles:
-
- -##### Input Sample +
| Column A | | --- | @@ -64,30 +52,46 @@ wrangles:
-
- -##### Output Sample +
-| Column A | First Element | -| --- | --- | -| [A, 0.9] | A | -| [B, 0.8] | B | +| First Element | +| --- | +| A | +| B |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.element` | + +**Sources** - [WranglesPY select.element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/element.md) -## Registry metadata - -- Registry ID: `223d2f4e-3247-4189-8b6c-e73fe44c4266` -- Namespace: `select` -- Recipe key: `select.element` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.element` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/group-by.md b/wrangles-docs/registry-docs/select/group-by.md index 495de76..05f43e2 100644 --- a/wrangles-docs/registry-docs/select/group-by.md +++ b/wrangles-docs/registry-docs/select/group-by.md @@ -3,61 +3,52 @@ title: "Group By" description: "Group and aggregate the data." sidebar_label: "Group By" slug: "/select/group-by" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Group By Group and aggregate the data. -> Pilot Registry entry. Runtime contract status: `verified`. +Group and aggregate the data ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `by` | No | string, array | List of the input columns to group on. | `[]` | -| `auto_rename_columns` | No | boolean | If true (default), aggregated column names include the operation as a suffix (e.g. Value.sum). If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total). | `true` | -| `list` | No | string, array | Group and return all values for these column(s) as a list. | — | -| `first` | No | string, array | The first value for these column(s). | — | -| `last` | No | string, array | The last value for these column(s). | — | -| `min` | No | string, array | The minimum value for these column(s). | — | -| `max` | No | string, array | The maximum value for these column(s). | — | -| `mean` | No | string, array | The mean (average) value for these column(s). | — | -| `median` | No | string, array | The median value for these column(s). | — | -| `nunique` | No | string, array | The count of unique values for these column(s). | — | -| `count` | No | string, array | The count of values for these column(s). | — | -| `counts` | No | string, array | Return a dictionary containing the count of each distinct value for these column(s). Keys are converted to JSON-safe strings; missing values use the key "null" and booleans use lowercase "true"/"false". | — | -| `std` | No | string, array | The standard deviation of values for these column(s). | — | -| `sum` | No | string, array | The total of values for these column(s). | — | -| `any` | No | string, array | Return true if any of the values for these column(s) are true. | — | -| `all` | No | string, array | Return true if all of the values for these column(s) are true. | — | -| `p75` | No | string, array | Get a percentile. Note, you can use any integer here for the corresponding percentile. | — | -| `custom.*` | No | string, array | Placeholder for custom functions. Replace 'placeholder' with the name of the function. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +
-| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Group and aggregate the data +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `by` | List of the input columns to group on. | string, array | `[]` | No | +| Options | | | | | +| `list` | Group and return all values for these column(s) as a list. | string, array | — | No | +| `first` | The first value for these column(s). | string, array | — | No | +| `last` | The last value for these column(s). | string, array | — | No | +| `min` | The minimum value for these column(s). | string, array | — | No | +| `max` | The maximum value for these column(s). | string, array | — | No | +| `mean` | The mean (average) value for these column(s). | string, array | — | No | +| `median` | The median value for these column(s). | string, array | — | No | +| `nunique` | The count of unique values for these column(s). | string, array | — | No | +| `count` | The count of values for these column(s). | string, array | — | No | +| `counts` | Return a dictionary containing the count of each distinct value for these column(s). Keys are converted to JSON-safe strings; missing values use the key "null" and booleans use lowercase "true"/"false". | string, array | — | No | +| `std` | The standard deviation of values for these column(s). | string, array | — | No | +| `sum` | The total of values for these column(s). | string, array | — | No | +| `any` | Return true if any of the values for these column(s) are true. | string, array | — | No | +| `all` | Return true if all of the values for these column(s) are true. | string, array | — | No | +| `p75` | Get a percentile. Note, you can use any integer here for the corresponding percentile. | string, array | — | No | +| `custom.*` | Placeholder for custom functions. Replace 'placeholder' with the name of the function. | string, array | — | No | +| Formatting | | | | | +| `auto_rename_columns` | If true (default), aggregated column names include the operation as a suffix (e.g. Value.sum). If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total). | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Grouping By One Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -70,9 +61,7 @@ wrangles:
-
- -##### Input Sample +
| Product | Quantity | Price ($) | Product Type | | --- | --- | --- | --- | @@ -83,9 +72,7 @@ wrangles:
-
- -##### Output Sample +
| Product Type | Quantity.sum | Price ($).mean | | --- | --- | --- | @@ -96,9 +83,9 @@ wrangles:
-#### Grouping With Custom Function Aggregation -##### Recipe + + ```yaml wrangles: @@ -109,9 +96,7 @@ wrangles:
-
- -##### Input Sample +
| Category | Quantity | | --- | --- | @@ -122,9 +107,7 @@ wrangles:
-
- -##### Output Sample +
| Category | Quantity.sum_times_two | | --- | --- | @@ -135,17 +118,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.group_by` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.group_by` | + +**Sources** - [WranglesPY select.group_by implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.group_by Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/group-by.md) -## Registry metadata - -- Registry ID: `c0af10b1-423a-416c-8cb5-7e7fe1164964` -- Namespace: `select` -- Recipe key: `select.group_by` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.group_by` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/head.md b/wrangles-docs/registry-docs/select/head.md index 7f3b914..f5d90fd 100644 --- a/wrangles-docs/registry-docs/select/head.md +++ b/wrangles-docs/registry-docs/select/head.md @@ -3,44 +3,33 @@ title: "Head" description: "Return the first n rows." sidebar_label: "Head" slug: "/select/head" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Head Return the first n rows. -> Pilot Registry entry. Runtime contract status: `verified`. +Return the first n rows ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `n` | Yes | integer | Number of rows to return. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Return the first n rows +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `n` | Number of rows to return. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting The First n Rows +
-##### Recipe +## Examples ```yaml wrangles: @@ -50,9 +39,7 @@ wrangles:
-
- -##### Input Sample +
| Product | Manufacturer | | --- | --- | @@ -63,9 +50,7 @@ wrangles:
-
- -##### Output Sample +
| Product | Manufacturer | | --- | --- | @@ -76,17 +61,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.head` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.head` | + +**Sources** - [WranglesPY select.head implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.head Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/head.md) -## Registry metadata - -- Registry ID: `237af1ec-db7d-415a-88b7-70586a2191fb` -- Namespace: `select` -- Recipe key: `select.head` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.head` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/highest-confidence.md b/wrangles-docs/registry-docs/select/highest-confidence.md index 46d3cc3..f7e41cd 100644 --- a/wrangles-docs/registry-docs/select/highest-confidence.md +++ b/wrangles-docs/registry-docs/select/highest-confidence.md @@ -3,45 +3,34 @@ title: "Highest Confidence" description: "Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form [<>, <>]." sidebar_label: "Highest Confidence" slug: "/select/highest-confidence" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Highest Confidence Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form [<<value>>, <<confidence_score>>]. -> Pilot Registry entry. Runtime contract status: `verified`. +Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form `[value, confidence_score]`. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of the input columns to select from. | — | -| `output` | Yes | string, array | If two columns; the result and confidence. If one column; [result, confidence]. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Select the option with the highest confidence from multiple columns. Inputs are expected to be of the form `[value, confidence_score]`. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of the input columns to select from. | array | — | Yes | +| `output` | If two columns; the result and confidence. If one column; [result, confidence]. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting Highest Confidence Single Output +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -65,9 +52,7 @@ wrangles:
-
- -##### Output Sample +
| Highest Confidence | | --- | @@ -77,9 +62,9 @@ wrangles:
-#### Selecting Highest Confidence Two Outputs -##### Recipe + + ```yaml wrangles: @@ -95,9 +80,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -105,9 +88,7 @@ wrangles:
-
- -##### Output Sample +
| Item | Confidence | | --- | --- | @@ -117,17 +98,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.highest_confidence` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.highest_confidence` | + +**Sources** - [WranglesPY select.highest_confidence implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.highest_confidence Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/highest-confidence.md) -## Registry metadata - -- Registry ID: `00aad85d-8cc8-42e5-86f3-e4ff916e8ac2` -- Namespace: `select` -- Recipe key: `select.highest_confidence` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.highest_confidence` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/left.md b/wrangles-docs/registry-docs/select/left.md index 7eb9c68..0a6f1d4 100644 --- a/wrangles-docs/registry-docs/select/left.md +++ b/wrangles-docs/registry-docs/select/left.md @@ -3,46 +3,36 @@ title: "Left" description: "Return characters from the left of text. Strings shorter than the length defined will be unaffected." sidebar_label: "Left" slug: "/select/left" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Left Return characters from the left of text. Strings shorter than the length defined will be unaffected. -> Pilot Registry entry. Runtime contract status: `verified`. +Select characters from the left of the input. Using a negative length reverses the side of selection, selecting from the right. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column(s) to edit. | — | -| `length` | Yes | integer | Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0. | — | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Select characters from the left of the input. Using a negative length reverses the side of selection, selecting from the right. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `length` | Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting Three Leftmost Elements +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -64,9 +52,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -76,17 +62,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.left` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.left` | + +**Sources** - [WranglesPY select.left implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.left Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/left.md) -## Registry metadata - -- Registry ID: `5cdd9857-0c77-43bf-80d7-d0a8cb6f980b` -- Namespace: `select` -- Recipe key: `select.left` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.left` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/length.md b/wrangles-docs/registry-docs/select/length.md index 31c6687..5a8c546 100644 --- a/wrangles-docs/registry-docs/select/length.md +++ b/wrangles-docs/registry-docs/select/length.md @@ -3,45 +3,34 @@ title: "Length" description: "Calculate the lengths of data in a column. The length depends on the data type e.g. text will be the length of the text, lists will be the number of elements in the list." sidebar_label: "Length" slug: "/select/length" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Length Calculate the lengths of data in a column. The length depends on the data type e.g. text will be the length of the text, lists will be the number of elements in the list. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column(s). | — | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Calculate the lengths of data in a column. The length depends on the data type e.g. text will be the length of the text, lists will be the number of elements in the list. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column(s). | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting the Length of Data Within a Column +
-##### Recipe +## Examples ```yaml wrangles: @@ -52,9 +41,7 @@ wrangles:
-
- -##### Input Sample +
| Part Code | | --- | @@ -62,9 +49,7 @@ wrangles:
-
- -##### Output Sample +
| Part Code Length | | --- | @@ -74,17 +59,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.length` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.length` | + +**Sources** - [WranglesPY select.length implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.length Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/length.md) -## Registry metadata - -- Registry ID: `4d7a5f66-0a4a-40e0-8298-d5c55754423d` -- Namespace: `select` -- Recipe key: `select.length` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.length` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/list-element.md b/wrangles-docs/registry-docs/select/list-element.md index 1ec8e0e..6ee779b 100644 --- a/wrangles-docs/registry-docs/select/list-element.md +++ b/wrangles-docs/registry-docs/select/list-element.md @@ -3,47 +3,38 @@ title: "List Element" description: "Select a numbered element of a list (zero indexed)." sidebar_label: "List Element" slug: "/select/list-element" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # List Element Select a numbered element of a list (zero indexed). -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the input column. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `element` | No | integer | The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list. | `0` | -| `default` | No | string, number, array, object, boolean, null | Set the default value to return if the specified element doesn't exist. | `""` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Select a numbered element of a list (zero indexed). +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the input column. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `element` | The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list. | integer | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Set the default value to return if the specified element doesn't exist. | string, number, array, object, boolean, null | `""` | No | -## Migrated examples -#### Selecting the Second Element in a List +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +47,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | | --- | @@ -67,9 +56,7 @@ wrangles:
-
- -##### Output Sample +
| Third Element | | --- | @@ -80,17 +67,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.list_element` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.list_element` | + +**Sources** - [WranglesPY select.list_element implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.list_element Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/list-element.md) -## Registry metadata - -- Registry ID: `ec40495d-d29a-4f62-86dd-eafa43cf388a` -- Namespace: `select` -- Recipe key: `select.list_element` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.list_element` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/right.md b/wrangles-docs/registry-docs/select/right.md index 0384bbc..15c6e79 100644 --- a/wrangles-docs/registry-docs/select/right.md +++ b/wrangles-docs/registry-docs/select/right.md @@ -3,46 +3,36 @@ title: "Right" description: "Return characters from the right of text. Strings shorter than the length defined will be unaffected." sidebar_label: "Right" slug: "/select/right" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Right Return characters from the right of text. Strings shorter than the length defined will be unaffected. -> Pilot Registry entry. Runtime contract status: `verified`. +Select characters from the right of the input. Using a negative length reverses the side of selection, selecting from the left. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column(s) to edit. | — | -| `length` | Yes | integer | Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0. | — | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Select characters from the right of the input. Using a negative length reverses the side of selection, selecting from the left. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `length` | Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting the Three Rightmost Elements +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -64,9 +52,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -76,17 +62,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.right` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.right` | + +**Sources** - [WranglesPY select.right implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.right Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/right.md) -## Registry metadata - -- Registry ID: `89ee82ec-3bc5-4bfa-899b-7a1260ef9bdb` -- Namespace: `select` -- Recipe key: `select.right` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.right` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/sample.md b/wrangles-docs/registry-docs/select/sample.md index d79ac89..bbbd3ea 100644 --- a/wrangles-docs/registry-docs/select/sample.md +++ b/wrangles-docs/registry-docs/select/sample.md @@ -3,44 +3,33 @@ title: "Sample" description: "Return a random sample of the rows." sidebar_label: "Sample" slug: "/select/sample" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Sample Return a random sample of the rows. -> Pilot Registry entry. Runtime contract status: `verified`. +Return a random sample of the rows ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `rows` | Yes | integer, number | If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Return a random sample of the rows +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `rows` | If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned. | integer, number | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting 2 Random Rows +
-##### Recipe +## Examples ```yaml wrangles: @@ -50,9 +39,7 @@ wrangles:
-
- -##### Input Sample +
| Product | Manufacturer | | --- | --- | @@ -63,9 +50,7 @@ wrangles:
-
- -##### Output Sample +
| Product | Manufacturer | | --- | --- | @@ -76,9 +61,9 @@ wrangles:
-#### Selecting a Random 25% of All Rows -##### Recipe + + ```yaml wrangles: @@ -88,9 +73,7 @@ wrangles:
-
- -##### Input Sample +
| Product | Manufacturer | | --- | --- | @@ -101,9 +84,7 @@ wrangles:
-
- -##### Output Sample +
| Product | Manufacturer | | --- | --- | @@ -113,17 +94,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.sample` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.sample` | + +**Sources** - [WranglesPY select.sample implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.sample Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/sample.md) -## Registry metadata - -- Registry ID: `95a84ab6-a66e-450f-8a4a-7a87e3a77932` -- Namespace: `select` -- Recipe key: `select.sample` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.sample` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/substring.md b/wrangles-docs/registry-docs/select/substring.md index 9ba1f16..9b56470 100644 --- a/wrangles-docs/registry-docs/select/substring.md +++ b/wrangles-docs/registry-docs/select/substring.md @@ -3,47 +3,37 @@ title: "Substring" description: "Return characters from the middle of text." sidebar_label: "Substring" slug: "/select/substring" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Substring Return characters from the middle of text. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column(s) to edit. | — | -| `start` | No | integer, null | The position of the first character to select. If ommited will start from the beginning and length must be provided. | `null` | -| `length` | No | integer, null | The length of the string to select. If ommited will select to the end of the string and start must be provided. | `null` | -| `output` | No | string, array, null | Name of the output column(s). | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Return characters from the middle of text. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column(s) to edit. | string, integer, array | — | Yes | +| `output` | Name of the output column(s). | string, array, null | `null` | No | +| Options | | | | | +| `start` | The position of the first character to select. If ommited will start from the beginning and length must be provided. | integer, null | `null` | No | +| `length` | The length of the string to select. If ommited will select to the end of the string and start must be provided. | integer, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting a Substring With Start and Length +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +46,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -66,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -78,9 +64,9 @@ wrangles:
-#### Selecting a Substring With Start Only -##### Recipe + + ```yaml wrangles: @@ -92,9 +78,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -102,9 +86,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -114,9 +96,9 @@ wrangles:
-#### Selecting a Substring With Length Only -##### Recipe + + ```yaml wrangles: @@ -128,9 +110,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -138,9 +118,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -150,17 +128,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.substring` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.substring` | + +**Sources** - [WranglesPY select.substring implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.substring Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/substring.md) -## Registry metadata - -- Registry ID: `8befddf8-602e-4fa9-8f16-4c547210ebec` -- Namespace: `select` -- Recipe key: `select.substring` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.substring` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/tail.md b/wrangles-docs/registry-docs/select/tail.md index ba07a82..306515c 100644 --- a/wrangles-docs/registry-docs/select/tail.md +++ b/wrangles-docs/registry-docs/select/tail.md @@ -3,44 +3,33 @@ title: "Tail" description: "Return the last n rows." sidebar_label: "Tail" slug: "/select/tail" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Tail Return the last n rows. -> Pilot Registry entry. Runtime contract status: `verified`. +Return the last n rows ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `n` | Yes | integer | Number of rows to return. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Return the last n rows +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `n` | Number of rows to return. | integer | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting The Last n Rows +
-##### Recipe +## Examples ```yaml wrangles: @@ -50,9 +39,7 @@ wrangles:
-
- -##### Input Sample +
| Product | Manufacturer | | --- | --- | @@ -63,9 +50,7 @@ wrangles:
-
- -##### Output Sample +
| Product | Manufacturer | | --- | --- | @@ -76,17 +61,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.tail` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.tail` | + +**Sources** - [WranglesPY select.tail implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.tail Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/tail.md) -## Registry metadata - -- Registry ID: `60853759-c160-49a4-87eb-036516a9d823` -- Namespace: `select` -- Recipe key: `select.tail` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.tail` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/select/threshold.md b/wrangles-docs/registry-docs/select/threshold.md index 1718f62..71fff8b 100644 --- a/wrangles-docs/registry-docs/select/threshold.md +++ b/wrangles-docs/registry-docs/select/threshold.md @@ -3,46 +3,36 @@ title: "Threshold" description: "Select the first option if it exceeds a given threshold, else the second option." sidebar_label: "Threshold" slug: "/select/threshold" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Threshold Select the first option if it exceeds a given threshold, else the second option. -> Pilot Registry entry. Runtime contract status: `verified`. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | List of the input columns to select from. | — | -| `output` | Yes | string | Name of the output column. | — | -| `threshold` | Yes | number | Threshold above which to choose the first option, otherwise the second. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Select the first option if it exceeds a given threshold, else the second option. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | List of the input columns to select from. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `threshold` | Threshold above which to choose the first option, otherwise the second. | number | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Selecting Results Above a Threshold +
-##### Recipe +## Examples ```yaml wrangles: @@ -56,9 +46,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | Col2 | | --- | --- | @@ -66,9 +54,7 @@ wrangles:
-
- -##### Output Sample +
| Result | | --- | @@ -78,17 +64,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `select.threshold` | +| Lifecycle status | active | +| Namespace | `select` | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.select.threshold` | + +**Sources** - [WranglesPY select.threshold implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/select.py) - [Existing select.threshold Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/threshold.md) -## Registry metadata - -- Registry ID: `af2a5dcc-0ec0-48d7-8fb4-f58d9c5391d2` -- Namespace: `select` -- Recipe key: `select.threshold` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.select.threshold` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/similarity.md b/wrangles-docs/registry-docs/similarity.md index 2da662f..9024a3a 100644 --- a/wrangles-docs/registry-docs/similarity.md +++ b/wrangles-docs/registry-docs/similarity.md @@ -3,50 +3,40 @@ title: "Similarity" description: "Calculate the cosine similarity of two vectors." sidebar_label: "Similarity" slug: "/similarity" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Similarity Calculate the cosine similarity of two vectors. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | array | Two columns of vectors to compare the similarity of. | — | -| `output` | Yes | string | Name of the output column. | — | -| `method` | No | string; one of: cosine, adjusted cosine, euclidean | The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons. | `"cosine"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Calculate the similarity of two vectors. :::info Similarity only works on vectors. To produce vectors from a column of strings, use `create.embeddings` first. ::: -## Migrated examples -#### Similarity Between Embeddings +## Parameters + +
+ +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Two columns of vectors to compare the similarity of. | array | — | Yes | +| `output` | Name of the output column. | string | — | Yes | +| Options | | | | | +| `method` | The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons. | string; one of:
  • cosine
  • adjusted cosine
  • euclidean
| `"cosine"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
-##### Recipe +## Examples ```yaml wrangles: @@ -70,9 +60,7 @@ wrangles:
-
- -##### Input Sample +
| col1 | col2 | | --- | --- | @@ -81,30 +69,46 @@ wrangles:
-
- -##### Output Sample +
-| col1 | col2 | col1 embeddings | col2 embeddings | similarity | -| --- | --- | --- | --- | --- | -| SKF | Timken | [1, 2, 3, 4] | [4, 3, 2, 1] | 0.158931 | -| Ball Bearing | Roller Bearing | [5, 6, 7, 8] | [5, 6, 7, 9] | 0.942437 | +| col1 embeddings | col2 embeddings | similarity | +| --- | --- | --- | +| [1, 2, 3, 4] | [4, 3, 2, 1] | 0.158931 | +| [5, 6, 7, 8] | [5, 6, 7, 9] | 0.942437 |
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `similarity` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compare` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.similarity` | + +**Sources** - [WranglesPY similarity implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing similarity Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compare/_sources/similarity.md) -## Registry metadata - -- Registry ID: `7c733344-4cce-4938-8013-53742fb46a90` -- Namespace: root-level runtime key -- Recipe key: `similarity` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.similarity` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/sort.md b/wrangles-docs/registry-docs/sort.md index 89af6ad..55c9191 100644 --- a/wrangles-docs/registry-docs/sort.md +++ b/wrangles-docs/registry-docs/sort.md @@ -3,46 +3,37 @@ title: "Sort" description: "Sort the data." sidebar_label: "Sort" slug: "/sort" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Sort Sort the data. -> Pilot Registry entry. Runtime contract status: `verified`. +Sort the data ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `ignore_index` | No | boolean | Ignore Index value accepted by the runtime. | `true` | -| `by` | Yes | string, array | Name or list of the column(s) to sort by. | — | -| `ascending` | No | boolean, array | Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Sort the data +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `by` | Name or list of the column(s) to sort by. | string, array | — | Yes | +| Options | | | | | +| `ascending` | Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by. | boolean, array | — | No | +| Formatting | | | | | +| `ignore_index` | Ignore Index value accepted by the runtime. | boolean | `true` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Replacing Abbreviations +
-##### Recipe +## Examples ```yaml wrangles: @@ -53,9 +44,7 @@ wrangles:
-
- -##### Input Sample +
| Item | Price | | --- | --- | @@ -67,9 +56,7 @@ wrangles:
-
- -##### Output Sample +
| Item | Price | | --- | --- | @@ -83,17 +70,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `sort` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `select` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.sort` | + +**Sources** - [WranglesPY sort implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing sort Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/select/_sources/sort.md) -## Registry metadata - -- Registry ID: `1dcf06ad-898a-4d83-862c-4774be37a687` -- Namespace: root-level runtime key -- Recipe key: `sort` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.sort` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/split/dictionary.md b/wrangles-docs/registry-docs/split/dictionary.md index cefc7a7..e887a1c 100644 --- a/wrangles-docs/registry-docs/split/dictionary.md +++ b/wrangles-docs/registry-docs/split/dictionary.md @@ -3,47 +3,38 @@ title: "Dictionary" description: "Split one or more dictionaries into columns. The dictionary keys will be returned as the new column headers. If the dictionaries contain overlapping values, the last value will be returned." sidebar_label: "Dictionary" slug: "/split/dictionary" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Dictionary Split one or more dictionaries into columns. The dictionary keys will be returned as the new column headers. If the dictionaries contain overlapping values, the last value will be returned. -> Pilot Registry entry. Runtime contract status: `verified`. +Split a dictionary into columns. The dictionary keys are used as the new column headers. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned. | — | -| `output` | No | string, array, null | In columns output_format, this is an optional subset of keys to extract from the dictionary. If not provided, all keys will be returned. Columns can be renamed with the following syntax: output: - key1: new_column_name1 - key2: new_column_name2 In to_lists output_format, this must be two output columns for the keys and values lists. If not provided, Keys and Values will be used. | `null` | -| `default` | No | object, null | Provide a set of default headings and values if they are not found within the input. | `null` | -| `output_format` | No | string; one of: columns, to_lists | How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values. | `"columns"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance +
-Split a dictionary into columns. The dictionary keys are used as the new column headers. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned. | string, integer, array | — | Yes | +| `output` | In columns output_format, this is an optional subset of keys to extract from the dictionary. If not provided, all keys will be returned. Columns can be renamed with the following syntax: output: - key1: new_column_name1 - key2: new_column_name2 In to_lists output_format, this must be two output columns for the keys and values lists. If not provided, Keys and Values will be used. | string, array, null | `null` | No | +| Formatting | | | | | +| `output_format` | How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values. | string; one of:
  • columns
  • to_lists
| `"columns"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Errors | | | | | +| `default` | Provide a set of default headings and values if they are not found within the input. | object, null | `null` | No | -## Migrated examples -#### Splitting an Entire Dictionary +
-##### Recipe +## Examples ```yaml wrangles: @@ -54,9 +45,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -64,9 +53,7 @@ wrangles:
-
- -##### Output Sample +
| Col1 | Col2 | Col3 | | --- | --- | --- | @@ -76,9 +63,9 @@ wrangles:
-#### Choosing Specific Keys by Name -##### Recipe + + ```yaml wrangles: @@ -89,9 +76,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -99,9 +84,7 @@ wrangles:
-
- -##### Output Sample +
| Col2 | | --- | @@ -111,9 +94,9 @@ wrangles:
-#### Using a Wildcard Output to Choose Specific Keys -##### Recipe + + ```yaml wrangles: @@ -124,9 +107,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -134,9 +115,7 @@ wrangles:
-
- -##### Output Sample +
| Col1 | Col2 | | --- | --- | @@ -146,9 +125,9 @@ wrangles:
-#### Using Regular Expressions to Choose Specific Keys -##### Recipe + + ```yaml wrangles: @@ -159,9 +138,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -169,9 +146,7 @@ wrangles:
-
- -##### Output Sample +
| Col3 | | --- | @@ -181,9 +156,9 @@ wrangles:
-#### Choosing Specific Keys While Renaming the Output -##### Recipe + + ```yaml wrangles: @@ -196,9 +171,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -206,9 +179,7 @@ wrangles:
-
- -##### Output Sample +
| Column 1 | Column 2 | | --- | --- | @@ -218,9 +189,9 @@ wrangles:
-#### Using a Wildcard While Renaming -##### Recipe + + ```yaml wrangles: @@ -232,9 +203,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -242,9 +211,7 @@ wrangles:
-
- -##### Output Sample +
| Column 1 | Column 2 | Column 3 | | --- | --- | --- | @@ -254,17 +221,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.dictionary` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.dictionary` | + +**Sources** - [WranglesPY split.dictionary implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) - [Existing split.dictionary Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/dictionary.md) -## Registry metadata - -- Registry ID: `06ca98e4-d026-43f7-84eb-af246d401ba9` -- Namespace: `split` -- Recipe key: `split.dictionary` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.split.dictionary` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/split/list.md b/wrangles-docs/registry-docs/split/list.md index 4767569..f48f9eb 100644 --- a/wrangles-docs/registry-docs/split/list.md +++ b/wrangles-docs/registry-docs/split/list.md @@ -3,45 +3,34 @@ title: "List" description: "Split a list in a single column to multiple columns." sidebar_label: "List" slug: "/split/list" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # List Split a list in a single column to multiple columns. -> Pilot Registry entry. Runtime contract status: `verified`. +Split a list into multiple columns. If only one output is given, `split.list` returns the same list it was given, so output should be a list of columns or a column name with a wildcard (`*`). ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer | Name of the column to be split. | — | -| `output` | Yes | string, array | Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Split a list into multiple columns. If only one output is given, `split.list` returns the same list it was given, so output should be a list of columns or a column name with a wildcard (`*`). +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to be split. | string, integer | — | Yes | +| `output` | Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer. | string, array | — | Yes | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Using a Wildcard +
-##### Recipe +## Examples ```yaml wrangles: @@ -52,9 +41,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -62,9 +49,7 @@ wrangles:
-
- -##### Output Sample +
| Column1 | Column2 | Column3 | | --- | --- | --- | @@ -74,9 +59,9 @@ wrangles:
-#### Named Columns -##### Recipe + + ```yaml wrangles: @@ -90,9 +75,7 @@ wrangles:
-
- -##### Input Sample +
| Column | | --- | @@ -100,9 +83,7 @@ wrangles:
-
- -##### Output Sample +
| Heading A | Heading B | Heading C | | --- | --- | --- | @@ -112,17 +93,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.list` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.list` | + +**Sources** - [WranglesPY split.list implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) - [Existing split.list Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/list.md) -## Registry metadata - -- Registry ID: `3260b9f7-aae2-499f-8004-d211c2cf643e` -- Namespace: `split` -- Recipe key: `split.list` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.split.list` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/split/text.md b/wrangles-docs/registry-docs/split/text.md index 3dc4bb6..785bc82 100644 --- a/wrangles-docs/registry-docs/split/text.md +++ b/wrangles-docs/registry-docs/split/text.md @@ -3,50 +3,41 @@ title: "Text" description: "Split a string to multiple columns or a list." sidebar_label: "Text" slug: "/split/text" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Text Split a string to multiple columns or a list. -> Pilot Registry entry. Runtime contract status: `verified`. +Split text strings on certain characters. The text can be split into either multiple columns or a list. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string | Name of the column to be split. | — | -| `output` | No | string, array, null | Name of the output column(s) If a single column is provided, the results will be returned as a list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input. | `null` | -| `char` | No | string | Set the character(s) to split on. Default comma (,) Can also prefix with "regex:" to split on a pattern. | `","` | -| `pad` | No | boolean, null | Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists. | `null` | -| `element` | No | string, integer, null | Select a specific element or range after splitting using slicing syntax. e.g. 0, ":5", "5:", "2:8:2". | `null` | -| `inclusive` | No | boolean | If true, include the split character in the output. Default False. | `false` | -| `skip_empty` | No | boolean | Whether to skip empty values. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples +
-_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Split text strings on certain characters. The text can be split into either multiple columns or a list. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to be split. | string | — | Yes | +| `output` | Name of the output column(s) If a single column is provided, the results will be returned as a list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input. | string, array, null | `null` | No | +| Options | | | | | +| `char` | Set the character(s) to split on. Default comma (,) Can also prefix with "regex:" to split on a pattern. | string | `","` | No | +| `element` | Select a specific element or range after splitting using slicing syntax. e.g. 0, ":5", "5:", "2:8:2". | string, integer, null | `null` | No | +| `inclusive` | If true, include the split character in the output. Default False. | boolean | `false` | No | +| `skip_empty` | Whether to skip empty values. | boolean | `false` | No | +| Formatting | | | | | +| `pad` | Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists. | boolean, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### To a List +
-##### Recipe +## Examples ```yaml wrangles: @@ -58,9 +49,7 @@ wrangles:
-
- -##### Input Sample +
| Column1 | | --- | @@ -68,9 +57,7 @@ wrangles:
-
- -##### Output Sample +
| Column2 | | --- | @@ -80,9 +67,9 @@ wrangles:
-#### Split Using Regex -##### Recipe + + ```yaml # Split on x, case insensitive. @@ -95,9 +82,7 @@ wrangles:
-
- -##### Input Sample +
| Col1 | | --- | @@ -106,9 +91,7 @@ wrangles:
-
- -##### Output Sample +
| Col2 | | --- | @@ -119,9 +102,9 @@ wrangles:
-#### Slice the Output -##### Recipe + + ```yaml wrangles: @@ -134,9 +117,7 @@ wrangles:
-
- -##### Input Sample +
| Column1 | | --- | @@ -144,9 +125,7 @@ wrangles:
-
- -##### Output Sample +
| Column2 | | --- | @@ -156,9 +135,9 @@ wrangles:
-#### Split to Columns (Wildcard) -##### Recipe + + ```yaml wrangles: @@ -170,9 +149,7 @@ wrangles:
-
- -##### Input Sample +
| Col | | --- | @@ -180,9 +157,7 @@ wrangles:
-
- -##### Output Sample +
| Col1 | Col2 | | --- | --- | @@ -192,9 +167,9 @@ wrangles:
-#### Split to Columns (Named) -##### Recipe + + ```yaml wrangles: @@ -209,9 +184,7 @@ wrangles:
-
- -##### Input Sample +
| Col | | --- | @@ -219,9 +192,7 @@ wrangles:
-
- -##### Output Sample +
| Col 1 | Col 2 | Col 3 | | --- | --- | --- | @@ -231,17 +202,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.text` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.text` | + +**Sources** - [WranglesPY split.text implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) - [Existing split.text Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/text.md) -## Registry metadata - -- Registry ID: `e76e43f7-d129-4bf8-87b4-a304a378b130` -- Namespace: `split` -- Recipe key: `split.text` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.split.text` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/split/tokenize.md b/wrangles-docs/registry-docs/split/tokenize.md index ab726f2..3c03404 100644 --- a/wrangles-docs/registry-docs/split/tokenize.md +++ b/wrangles-docs/registry-docs/split/tokenize.md @@ -3,46 +3,36 @@ title: "Tokenize" description: "Split text into tokens. A variety of methods are available. The default method is to split on spaces." sidebar_label: "Tokenize" slug: "/split/tokenize" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Tokenize Split text into tokens. A variety of methods are available. The default method is to split on spaces. -> Pilot Registry entry. Runtime contract status: `verified`. +Tokenize elements in a list or string into individual tokens. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Column(s) to be split into tokens. | — | -| `output` | No | string, array, null | Name of the output column. | `null` | -| `method` | No | string; one of: space, boundary, boundary_ignore_space or string | Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`. | `"space"` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +
-## Guidance - -Tokenize elements in a list or string into individual tokens. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Column(s) to be split into tokens. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array, null | `null` | No | +| Options | | | | | +| `method` | Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`. | string; one of:
  • space
  • boundary
  • boundary_ignore_space
or string | `"space"` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Tokenizing a String +
-##### Recipe +## Examples ```yaml wrangles: @@ -53,9 +43,7 @@ wrangles:
-
- -##### Input Sample +
| Materials | | --- | @@ -63,9 +51,7 @@ wrangles:
-
- -##### Output Sample +
| Tokenized List | | --- | @@ -76,9 +62,9 @@ wrangles:
-#### Tokenizing a List -##### Recipe + + ```yaml wrangles: @@ -89,9 +75,7 @@ wrangles:
-
- -##### Input Sample +
| Materials | | --- | @@ -99,9 +83,7 @@ wrangles:
-
- -##### Output Sample +
| Tokenized List | | --- | @@ -111,17 +93,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `split.tokenize` | +| Lifecycle status | active | +| Namespace | `split` | +| Documentation group | `split` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.split.tokenize` | + +**Sources** - [WranglesPY split.tokenize implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/split.py) - [Existing split.tokenize Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/split/_sources/tokenize.md) -## Registry metadata - -- Registry ID: `6cc88418-ae0c-43f6-84ee-31e0d5f838c3` -- Namespace: `split` -- Recipe key: `split.tokenize` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.split.tokenize` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/sql.md b/wrangles-docs/registry-docs/sql.md index fb889e5..c434d2b 100644 --- a/wrangles-docs/registry-docs/sql.md +++ b/wrangles-docs/registry-docs/sql.md @@ -3,51 +3,41 @@ title: "SQL" description: "Apply a SQL command to the current dataframe. Only SELECT statements are supported - the result will be the output." sidebar_label: "SQL" slug: "/sql" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # SQL Apply a SQL command to the current dataframe. Only SELECT statements are supported - the result will be the output. -> Pilot Registry entry. Runtime contract status: `verified`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `command` | Yes | string | SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect. | — | -| `params` | No | array, object, null | Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name). | `null` | -| `preserve_index` | No | boolean | Preserve Index value accepted by the runtime. | `false` | -| `preserve_data_types` | No | boolean | Preserve Data Types value accepted by the runtime. | `true` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - Apply a SQL command to the current dataframe. Only `SELECT` statements are supported; the result becomes the output. The current table is called `df`. :::info SQL does not currently work with objects. If your table contains objects, use `convert.to_json` before using SQL. SQL is not compatible with `where` filtering. ::: -## Migrated examples -#### Selecting a Subset of Data +## Parameters + +
-##### Recipe +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Options | | | | | +| `command` | SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect. | string | — | Yes | +| `params` | Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name). | array, object, null | `null` | No | +| `preserve_data_types` | Preserve Data Types value accepted by the runtime. | boolean | `true` | No | +| Formatting | | | | | +| `preserve_index` | Preserve Index value accepted by the runtime. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ +## Examples ```yaml wrangles: @@ -60,9 +50,7 @@ wrangles:
-
- -##### Input Sample +
| header1 | header2 | header3 | | --- | --- | --- | @@ -72,9 +60,7 @@ wrangles:
-
- -##### Output Sample +
| header1 | header2 | | --- | --- | @@ -85,17 +71,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `sql` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `compute` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.sql` | + +**Sources** - [WranglesPY sql implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing sql Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/compute/_sources/sql.md) -## Registry metadata - -- Registry ID: `467a06b1-a697-4d31-8061-7d83a719fd79` -- Namespace: root-level runtime key -- Recipe key: `sql` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.sql` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/standardize.md b/wrangles-docs/registry-docs/standardize.md index 335f51a..c9bf899 100644 --- a/wrangles-docs/registry-docs/standardize.md +++ b/wrangles-docs/registry-docs/standardize.md @@ -1,49 +1,44 @@ --- -title: "Standardize" -description: "Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription." -sidebar_label: "Standardize" +title: "Standardize (Deprecated)" +description: "Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model." +sidebar_label: "Standardize (Deprecated)" slug: "/standardize" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- -# Standardize +# Standardize (Deprecated) -Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. +:::warning Deprecated +This compatibility wrangle remains available for existing recipes. Use [`standardize.custom`](/wrangles/namespaces/standardize#custom) for new recipes. +::: -> Pilot Registry entry. Runtime contract status: `verified`. +Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model. -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `model_id` | Yes | string, array | The ID of the wrangle to use (do not include 'find' and 'replace'). | — | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `case_sensitive` | No | boolean | Allows the wrangle to be case sensitive if set to True, default is False. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +Run a standardize wrangle, such as one that expands abbreviations. A standardization wrangle must be trained first. -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | +## Parameters -## Guidance +
-Run a standardize wrangle, such as one that expands abbreviations. A standardization wrangle must be trained first. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use (do not include 'find' and 'replace'). | string, array | — | Yes | -## Migrated examples -#### Replacing Abbreviations +
-##### Recipe +## Examples ```yaml wrangles: @@ -55,9 +50,7 @@ wrangles:
-
- -##### Input Sample +
| Abbrev | | --- | @@ -66,9 +59,7 @@ wrangles:
-
- -##### Output Sample +
| Abbreviations | | --- | @@ -79,17 +70,36 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `standardize` | +| Lifecycle status | deprecated | +| Replaced by | [`standardize.custom`](/wrangles/namespaces/standardize#custom) | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.standardize` | + +**Sources** - [WranglesPY standardize implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing standardize Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/standardize.md) -## Registry metadata - -- Registry ID: `53cd3fdd-24e2-4411-8655-6014b92a3f3a` -- Namespace: root-level runtime key -- Recipe key: `standardize` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.standardize` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/standardize/clean.md b/wrangles-docs/registry-docs/standardize/clean.md index c089ffa..e0efd02 100644 --- a/wrangles-docs/registry-docs/standardize/clean.md +++ b/wrangles-docs/registry-docs/standardize/clean.md @@ -3,65 +3,79 @@ title: "Standardize Clean" description: "Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally." sidebar_label: "Standardize Clean" slug: "/standardize/clean" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Standardize Clean Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. -> Pilot Registry entry. Runtime contract status: `verified`. +## Behavior + +Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. + +This guidance was derived from the callable signature and its embedded Python schema docstring. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | +
+ +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `output` | No | string, integer, array, null | Name or list of output columns. Defaults to overwriting input. | `null` | -| `fix_encoding` | No | boolean | Repair mojibake and other reversible encoding errors. | `true` | -| `unescape_html` | No | boolean or string; one of: auto | Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup. | `"auto"` | -| `normalization` | No | string; one of: NFC, NFKC, NFD, NFKD, null | Unicode normalization form. | `"NFC"` | -| `fix_character_width` | No | boolean | Normalize fullwidth and halfwidth characters. | `true` | -| `uncurl_quotes` | No | boolean | Replace typographic quotes with straight quotes. | `true` | -| `remove_control_chars` | No | boolean | Remove C0 and C1 control characters. | `true` | -| `collapse_whitespace` | No | boolean | Collapse runs of Unicode whitespace. | `true` | -| `preserve_line_breaks` | No | boolean | Preserve line breaks while collapsing other whitespace. | `false` | -| `trim` | No | boolean | Remove leading and trailing whitespace. | `true` | -| `separator` | No | string | Text used to join multiple input columns into one output. | `" "` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ - -## Access +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. Defaults to overwriting input. | string, integer, array, null | `null` | No | +| Options | | | | | +| `fix_encoding` | Repair mojibake and other reversible encoding errors. | boolean | `true` | No | +| `unescape_html` | Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup. | boolean or string; one of:
  • auto
| `"auto"` | No | +| `normalization` | Unicode normalization form. | string; one of:
  • NFC
  • NFKC
  • NFD
  • NFKD
  • null
| `"NFC"` | No | +| `fix_character_width` | Normalize fullwidth and halfwidth characters. | boolean | `true` | No | +| `uncurl_quotes` | Replace typographic quotes with straight quotes. | boolean | `true` | No | +| `remove_control_chars` | Remove C0 and C1 control characters. | boolean | `true` | No | +| `collapse_whitespace` | Collapse runs of Unicode whitespace. | boolean | `true` | No | +| `preserve_line_breaks` | Preserve line breaks while collapsing other whitespace. | boolean | `false` | No | +| `trim` | Remove leading and trailing whitespace. | boolean | `true` | No | +| Formatting | | | | | +| `separator` | Text used to join multiple input columns into one output. | string | `" "` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | + +
+ + + +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | -## Guidance +
-## Behavior +
-Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. +Technical details -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +| Field | Value | +| --- | --- | +| Recipe key | `standardize.clean` | +| Lifecycle status | active | +| Namespace | `standardize` | +| Documentation group | `standardize` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.standardize.clean` | -## Provenance +**Sources** - [WranglesPY standardize.clean implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/standardize.py) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: `standardize` -- Recipe key: `standardize.clean` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.standardize.clean` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/standardize/custom.md b/wrangles-docs/registry-docs/standardize/custom.md index e642ce5..d26737a 100644 --- a/wrangles-docs/registry-docs/standardize/custom.md +++ b/wrangles-docs/registry-docs/standardize/custom.md @@ -3,57 +3,71 @@ title: "Standardize Custom" description: "Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription." sidebar_label: "Standardize Custom" slug: "/standardize/custom" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Standardize Custom Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. -> Pilot Registry entry. Runtime contract status: `verified`. +## Behavior + +Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. + +This guidance was derived from the callable signature and its embedded Python schema docstring. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | +
+ +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name or list of input columns. | — | -| `model_id` | Yes | string, array | The ID of the wrangle to use (do not include 'find' and 'replace'). | — | -| `output` | No | string, array, null | Name or list of output columns. | `null` | -| `case_sensitive` | No | boolean | Allows the wrangle to be case sensitive if set to True, default is False. | `false` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | +| I/O | | | | | +| `input` | Name or list of input columns. | string, integer, array | — | Yes | +| `output` | Name or list of output columns. | string, array, null | `null` | No | +| Options | | | | | +| `case_sensitive` | Allows the wrangle to be case sensitive if set to True, default is False. | boolean | `false` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Details | | | | | +| `model_id` | The ID of the wrangle to use (do not include 'find' and 'replace'). | string, array | — | Yes | + +
-## Verified examples -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ -## Access +
+ +Access | Requirement | Value | | --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | Yes | -| requires external api key | No | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | Yes | +| Requires external API key | No | -## Guidance +
-## Behavior +
-Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. +Technical details -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +| Field | Value | +| --- | --- | +| Recipe key | `standardize.custom` | +| Lifecycle status | active | +| Namespace | `standardize` | +| Documentation group | `standardize` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.standardize.custom` | -## Provenance +**Sources** - [WranglesPY standardize.custom implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/standardize.py) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: `standardize` -- Recipe key: `standardize.custom` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.standardize.custom` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/translate.md b/wrangles-docs/registry-docs/translate.md index 04cd0eb..49d0f8e 100644 --- a/wrangles-docs/registry-docs/translate.md +++ b/wrangles-docs/registry-docs/translate.md @@ -3,48 +3,38 @@ title: "Translate" description: "Translate the input to a different language. Requires WrangleWorks Account and DeepL API Key (A free account for up to 500,000 characters per month is available)." sidebar_label: "Translate" slug: "/translate" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Translate Translate the input to a different language. Requires WrangleWorks Account and DeepL API Key (A free account for up to 500,000 characters per month is available). -> Pilot Registry entry. Runtime contract status: `verified`. +Translate the input column to another language. Powered by DeepL. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `input` | Yes | string, integer, array | Name of the column to translate. | — | -| `output` | Yes | string, array | Name of the output column. | — | -| `target_language` | Yes | string; one of: Bulgarian, Chinese, Czech, Danish, Dutch, English (American), English (British), Estonian, Finnish, French, German, Greek, Hungarian, Italian, Japanese, Latvian, Lithuanian, Polish, Portuguese, Portuguese (Brazilian), Romanian, Russian, Slovak, Slovenian, Spanish, Swedish | Code of the language to translate to. | — | -| `source_language` | No | string; one of: Auto, Bulgarian, Chinese, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Japanese, Latvian, Lithuanian, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish | Code of the language to translate from. If omitted, automatically detects the input language. | `"AUTO"` | -| `case` | No | string, null | Allow changing the case of the input prior to translation. | `null` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | Yes | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Translate the input column to another language. Powered by DeepL. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `input` | Name of the column to translate. | string, integer, array | — | Yes | +| `output` | Name of the output column. | string, array | — | Yes | +| Options | | | | | +| `target_language` | Code of the language to translate to. | string; one of:
  • Bulgarian
  • Chinese
  • Czech
  • Danish
  • Dutch
  • English (American)
  • English (British)
  • Estonian
  • Finnish
  • French
  • German
  • Greek
  • Hungarian
  • Italian
  • Japanese
  • Latvian
  • Lithuanian
  • Polish
  • Portuguese
  • Portuguese (Brazilian)
  • Romanian
  • Russian
  • Slovak
  • Slovenian
  • Spanish
  • Swedish
| — | Yes | +| `source_language` | Code of the language to translate from. If omitted, automatically detects the input language. | string; one of:
  • Auto
  • Bulgarian
  • Chinese
  • Czech
  • Danish
  • Dutch
  • English
  • Estonian
  • Finnish
  • French
  • German
  • Greek
  • Hungarian
  • Italian
  • Japanese
  • Latvian
  • Lithuanian
  • Polish
  • Portuguese
  • Romanian
  • Russian
  • Slovak
  • Slovenian
  • Spanish
  • Swedish
| `"AUTO"` | No | +| `case` | Allow changing the case of the input prior to translation. | string, null | `null` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Migrated examples -#### Translating Spanish to English +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,9 +47,7 @@ wrangles:
-
- -##### Input Sample +
| Español | | --- | @@ -67,9 +55,7 @@ wrangles:
-
- -##### Output Sample +
| English | | --- | @@ -79,17 +65,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | Yes | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `translate` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `format` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.translate` | + +**Sources** - [WranglesPY translate implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing translate Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/format/_sources/translate.md) -## Registry metadata - -- Registry ID: `73c3ceb6-ffd8-4d74-8389-c83b99d33bb0` -- Namespace: root-level runtime key -- Recipe key: `translate` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.translate` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/transpose.md b/wrangles-docs/registry-docs/transpose.md index 430386b..cbba44b 100644 --- a/wrangles-docs/registry-docs/transpose.md +++ b/wrangles-docs/registry-docs/transpose.md @@ -3,48 +3,41 @@ title: "Transpose" description: "Transpose the DataFrame (swap columns to rows)." sidebar_label: "Transpose" slug: "/transpose" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Transpose Transpose the DataFrame (swap columns to rows). -> Pilot Registry entry. Runtime contract status: `verified`. +Transpose a dataframe. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `header_column` | No | string, integer, null | Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header. | `0` | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| I/O | | | | | +| `header_column` | Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header. | string, integer, null | `0` | No | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | -## Guidance +
-Transpose a dataframe. +## Examples -## Migrated examples :::note Transpose is not compatible with `where` filtering. ::: -#### Transposing a Dataframe -##### Recipe + + ```yaml wrangles: @@ -54,9 +47,7 @@ wrangles:
-
- -##### Input Sample +
| Material | Product Data | | --- | --- | @@ -65,9 +56,7 @@ wrangles:
-
- -##### Output Sample +
| Material | Ceramic | Rubber | | --- | --- | --- | @@ -77,17 +66,35 @@ wrangles:
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `transpose` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `transform` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.pandas.transpose` | + +**Sources** - [WranglesPY transpose implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/pandas.py) - [Existing transpose Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/transform/_sources/transpose.md) -## Registry metadata - -- Registry ID: `8716347f-f286-49b2-8a0b-cb73292e7475` -- Namespace: root-level runtime key -- Recipe key: `transpose` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.pandas.transpose` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/registry-docs/try.md b/wrangles-docs/registry-docs/try.md index f218c63..4792f8b 100644 --- a/wrangles-docs/registry-docs/try.md +++ b/wrangles-docs/registry-docs/try.md @@ -3,46 +3,36 @@ title: "Try" description: "Try a list of wrangles and catch any errors that occur." sidebar_label: "Try" slug: "/try" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 --- # Try Try a list of wrangles and catch any errors that occur. -> Pilot Registry entry. Runtime contract status: `verified`. +Run a list of wrangles and catch errors. When `except` is provided, its wrangles or fallback column values run after an error; otherwise, the error is logged and the recipe continues. ## Parameters -| Parameter | Required | Accepted value | Description | Runtime default | -| --- | --- | --- | --- | --- | -| `wrangles` | Yes | array | List of wrangles to apply. | — | -| `retries` | No | integer | Number of times to retry the wrangles if an error occurs. Default 0. | `0` | -| `except` | No | object | An action to take if the wrangles encounter an error. This can contain a list of wrangles or a dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue. | — | -| `if` | No | string | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | — | -| `where` | No | string | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | — | -| `where_params` | No | array, object | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | — | - -## Verified examples - -_No fixture-backed examples are currently available. See migrated examples under Guidance where present._ +
-## Access - -| Requirement | Value | -| --- | --- | -| ai powered | No | -| requires account | No | -| requires subscription | No | -| requires external api key | No | - -## Guidance - -Run a list of wrangles and catch errors. When `except` is provided, its wrangles or fallback column values run after an error; otherwise, the error is logged and the recipe continues. +| Name | Description | Accepted Values | Default | Required | +| --- | --- | --- | --- | --- | +| Conditions | | | | | +| `if` | Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`. | string | — | No | +| `where` | Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`. | string | — | No | +| `where_params` | Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`. | array, object | — | No | +| Execution | | | | | +| `wrangles` | List of wrangles to apply. | array | — | Yes | +| Errors | | | | | +| `retries` | Number of times to retry the wrangles if an error occurs. Default 0. | integer | `0` | No | +| `except` | An action to take if the wrangles encounter an error. This can contain a list of wrangles or a dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue. | object | — | No | -## Migrated examples -#### Use Fallback Wrangles After an Error +
-##### Recipe +## Examples ```yaml wrangles: @@ -57,17 +47,13 @@ wrangles:
-
- -##### Input Sample +
_The input dataframe is passed to the primary wrangles._
-
- -##### Output Sample +
_The primary result is returned on success; the `except` result is returned after an error._ @@ -75,17 +61,35 @@ _The primary result is returned on success; the `except` result is returned afte
-## Provenance +
+ +Access + +| Requirement | Value | +| --- | --- | +| AI-powered | No | +| Requires WrangleWorks account | No | +| Requires subscription | No | +| Requires external API key | No | + +
+ +
+ +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | `try` | +| Lifecycle status | active | +| Namespace | Root-level | +| Documentation group | `utility` | +| Aliases | None | +| Runtime symbol | `wrangles.recipe_wrangles.main.Try` | + +**Sources** - [WranglesPY try implementation](https://github.com/wrangleworks/WranglesPY/blob/7916bf158e8b7e561270a1bea7b808f88956edc4/wrangles/recipe_wrangles/main.py) - [Existing try Markdown](https://github.com/wrangleworks/Wrangles-Docs/blob/main/wrangles-docs/wrangle-docs/utility/_sources/try.md) -## Registry metadata - -- Registry ID: pending database assignment -- Namespace: root-level runtime key -- Recipe key: `try` -- Aliases: none -- Runtime symbol: `wrangles.recipe_wrangles.main.Try` -- Status: `active` -- Registry version: `0.1.0-pilot` +
diff --git a/wrangles-docs/scripts/bootstrap-registry.cjs b/wrangles-docs/scripts/bootstrap-registry.cjs index 5908155..122f93c 100644 --- a/wrangles-docs/scripts/bootstrap-registry.cjs +++ b/wrangles-docs/scripts/bootstrap-registry.cjs @@ -11,8 +11,11 @@ const entriesRoot = path.join(registryRoot, 'wrangles'); const runtimeManifestPath = path.join(registryRoot, 'runtime', 'wranglespy.json'); const quasiRegistryRoot = path.join(siteRoot, 'wrangle-docs'); const COMMON_CONTROLS = new Set(['if', 'where', 'where_params']); -const CURATED_KEYS = new Set(['convert.case', 'convert.data_type', 'convert.from_json']); const ALL_JSON_TYPES = ['string', 'number', 'integer', 'boolean', 'array', 'object', 'null']; +const DEPRECATED_REPLACEMENTS = { + maths: 'math', + standardize: 'standardize.custom', +}; function posixPath(value) { return value.split(path.sep).join('/'); @@ -230,6 +233,7 @@ const DESCRIPTION_OVERRIDES = { 'format.price_breaks': 'Expand non-empty price-break cells into paired category and value columns.', maths: 'Deprecated alias for `math`; evaluate an expression and write its result to an output column.', recipe: 'Run another recipe as a wrangle against the current dataframe.', + standardize: 'Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model.', }; function parameterDescription(name, embedded, quasi) { @@ -241,15 +245,56 @@ function parameterDescription(name, embedded, quasi) { ); } -function parameterRole(name) { - if (name === 'input' || name === 'by' || name.endsWith('_column')) return 'column-selector'; - if (name === 'output' || name.startsWith('output_')) return 'column-output'; - if (name === 'model_id') return 'model-reference'; - if (name.includes('api_key') || name.includes('secret')) return 'credential'; - if (name === 'wrangles') return 'nested-wrangles'; - if (name === 'variables') return 'variables'; - if (name === 'default' || name === 'on_error') return 'fallback-value'; - return 'option'; +const FORMATTING_PARAMETERS = new Set([ + 'allow_unicode', 'auto_rename_columns', 'categoryLabel', 'char', 'decimal_places', + 'decimals', 'ensure_ascii', 'first_element', 'format', 'ignore_index', + 'include_confidence', 'include_empty_labels', 'include_ratio', 'indent', + 'non_match_char', 'output_format', 'output_pattern', 'output_type', 'pad', + 'pad_length', 'precision', 'preserve_index', 'reset_index', 'return_data_type', + 'separator', 'significant_figures', 'sort_keys', 'sort_order', 'use_labels', + 'valueLabel', +]); +const EXECUTION_PARAMETERS = new Set([ + 'batch_size', 'deadline', 'max_concurrency', 'threads', 'timeout', + 'use_multiprocessing', 'variables', 'wrangles', +]); +const ERROR_PARAMETERS = new Set(['default', 'except', 'on_error', 'retries']); +const DETAIL_PARAMETERS = new Set([ + 'api_key', 'api_token', 'cache', 'cache_ttl', 'client', 'model', 'model_id', + 'previous_response', 'protocol', 'provider', 'reasoning', 'store', 'strict', + 'url', 'verbosity', +]); +const PARAM_GROUP_OVERRIDES = new Map([ + ['accordion.propagate', 'I/O'], + ['compute.case_when.default', 'Options'], + ['extract.date_range.end_time', 'I/O'], + ['extract.date_range.start_time', 'I/O'], + ['log.error', 'Options'], + ['remove_words.to_remove', 'I/O'], + ['search.find_links.device', 'Options'], + ['search.find_links.id', 'I/O'], + ['search.find_links.queries', 'I/O'], +]); + +function parameterGroup(runtimeKey, name) { + const override = PARAM_GROUP_OVERRIDES.get(`${runtimeKey}.${name}`); + if (override) return override; + if (COMMON_CONTROLS.has(name)) return 'Conditions'; + if ( + name === 'input' || + name === 'output' || + name === 'by' || + name === 'columns' || + name.endsWith('_column') + ) return 'I/O'; + if (name === 'char' && ['compare.text', 'split.text'].includes(runtimeKey)) { + return 'Options'; + } + if (FORMATTING_PARAMETERS.has(name)) return 'Formatting'; + if (EXECUTION_PARAMETERS.has(name)) return 'Execution'; + if (ERROR_PARAMETERS.has(name)) return 'Errors'; + if (DETAIL_PARAMETERS.has(name)) return 'Details'; + return 'Options'; } function dynamicParameter(name) { @@ -330,7 +375,7 @@ function buildParameters(runtime, quasi) { name: runtimeParameter.name, description: parameterDescription(runtimeParameter.name, embedded, quasiParameter), required: runtimeParameter.required, - role: parameterRole(runtimeParameter.name), + param_group: parameterGroup(runtime.runtime_key, runtimeParameter.name), }; if (hasOwn(runtimeParameter, 'default')) parameter.runtime_default = runtimeParameter.default; parameter.schema = withCodeTypes(embedded, runtimeParameter); @@ -345,7 +390,7 @@ function buildParameters(runtime, quasi) { ...dynamicParameter(name), description: parameterDescription(name, embedded, quasi?.parameters.get(name)), required: embeddedRequired.has(name), - role: parameterRole(name), + param_group: parameterGroup(runtime.runtime_key, name), schema: withCodeTypes(embedded, null), }; parameters.push(parameter); @@ -413,11 +458,14 @@ function buildMetadata(runtime, manifest, quasi) { slug: namespace ? `${namespace.split('.').map(kebab).join('/')}/${kebab(wrangleName)}` : kebab(wrangleName), - status: runtime.runtime_key === 'maths' ? 'deprecated' : ( + status: DEPRECATED_REPLACEMENTS[runtime.runtime_key] ? 'deprecated' : ( ['draft', 'active', 'deprecated', 'removed'].includes(quasi?.status) ? quasi.status : 'active' ), + ...(DEPRECATED_REPLACEMENTS[runtime.runtime_key] + ? {replaced_by: DEPRECATED_REPLACEMENTS[runtime.runtime_key]} + : {}), visibility: 'public', tags, runtime: { @@ -463,7 +511,7 @@ function entryDocument(metadata, quasi, runtime) { ${metadata.description} -This first-pass guidance is derived from the callable signature${ +This guidance was derived from the callable signature${ runtime.docstring_schema_status === 'available' ? ' and its embedded Python schema docstring' : '' @@ -472,7 +520,6 @@ This first-pass guidance is derived from the callable signature${ } async function main() { - const refreshGenerated = process.argv.includes('--refresh-generated'); const manifest = JSON.parse(await fs.readFile(runtimeManifestPath, 'utf8')); const quasiRegistry = await readQuasiRegistry(); const existing = new Set((await listFiles(entriesRoot, '.md')).map((filename) => path.resolve(filename))); @@ -482,9 +529,7 @@ async function main() { for (const runtime of manifest.wrangles) { const output = entryPath(runtime.runtime_key); - if (existing.has(path.resolve(output)) && ( - !refreshGenerated || CURATED_KEYS.has(runtime.runtime_key) - )) { + if (existing.has(path.resolve(output))) { preserved += 1; continue; } @@ -497,7 +542,7 @@ async function main() { } console.log( - `Created ${created} first-pass Registry entries; preserved ${preserved} existing entries; ` + + `Created ${created} missing Registry entries; preserved ${preserved} existing entries; ` + `${missingIds} newly created entries await a database UUID.`, ); } diff --git a/wrangles-docs/scripts/compile-registry.cjs b/wrangles-docs/scripts/compile-registry.cjs index f784c3e..f0e1ced 100644 --- a/wrangles-docs/scripts/compile-registry.cjs +++ b/wrangles-docs/scripts/compile-registry.cjs @@ -5,7 +5,7 @@ const path = require('node:path'); const crypto = require('node:crypto'); const yaml = require('js-yaml'); -const REGISTRY_VERSION = '0.1.0-pilot'; +const REGISTRY_VERSION = '0.1.0'; const ENTRY_SCHEMA_VERSION = '0.1'; const PUBLIC_ORIGIN = 'https://docs.wrangles.com'; @@ -25,8 +25,42 @@ const runtimeManifestPath = path.join(registryRoot, 'runtime', 'wranglespy.json' const reportsOutputRoot = path.join(registryRoot, 'reports'); const docsOutputRoot = path.join(siteRoot, 'registry-docs'); +const registrySidebarPath = path.join(siteRoot, 'sidebarsRegistry.js'); const rawOutputRoot = path.join(siteRoot, 'static', 'registry'); -const schemaOutputRoot = path.join(siteRoot, 'static', 'schemas', 'recipes', 'pilot'); +const schemaOutputRoot = path.join(siteRoot, 'static', 'schemas', 'recipes', 'registry'); + +const LEGACY_GROUP_OVERRIDES = { + maths: 'compute', + recipe: 'utility', +}; +const GROUP_ORDER = [ + 'convert', + 'merge', + 'split', + 'select', + 'format', + 'create', + 'extract', + 'compare', + 'compute', + 'search', + 'ai', + 'generate', + 'lookup', + 'standardize', + 'transform', + 'utility', + 'date', +]; +const PARAM_GROUP_ORDER = [ + 'I/O', + 'Options', + 'Formatting', + 'Conditions', + 'Execution', + 'Errors', + 'Details', +]; const generatedFiles = new Map(); const errors = []; @@ -422,6 +456,16 @@ function validateTopLevel(metadata, source, entrySchema) { if (!['draft', 'active', 'deprecated', 'removed'].includes(metadata.status)) { fail(source, 'status is invalid'); } + if (metadata.status === 'deprecated' && typeof metadata.replaced_by !== 'string') { + fail(source, 'deprecated entries must declare replaced_by'); + } + if (hasOwn(metadata, 'replaced_by')) { + if (!/^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)*$/.test(metadata.replaced_by || '')) { + fail(source, 'replaced_by must be a valid recipe key'); + } else if (metadata.replaced_by === metadata.wrangle_key) { + fail(source, 'replaced_by must not reference the same wrangle'); + } + } if (!['public', 'internal'].includes(metadata.visibility)) { fail(source, 'visibility is invalid'); } @@ -481,14 +525,14 @@ function validateParameters(metadata, source) { 'name_pattern', 'description', 'required', - 'role', + 'param_group', 'runtime_default', 'schema', ]); for (const key of Object.keys(parameter)) { if (!allowed.has(key)) fail(source, `${label} has unknown field ${key}`); } - for (const key of ['name', 'description', 'required', 'role', 'schema']) { + for (const key of ['name', 'description', 'required', 'param_group', 'schema']) { if (!hasOwn(parameter, key)) fail(source, `${label}.${key} is required`); } if (typeof parameter.name_pattern === 'string') { @@ -513,8 +557,8 @@ function validateParameters(metadata, source) { if (typeof parameter.required !== 'boolean') { fail(source, `${label}.required must be a boolean`); } - if (typeof parameter.role !== 'string' || !parameter.role.trim()) { - fail(source, `${label}.role must be a non-empty string`); + if (!PARAM_GROUP_ORDER.includes(parameter.param_group)) { + fail(source, `${label}.param_group must be one of: ${PARAM_GROUP_ORDER.join(', ')}`); } validateSchemaFragment(parameter.schema, source, label); } @@ -605,17 +649,21 @@ function escapeCell(value) { .replace(/\r?\n/g, '
'); } -function schemaSummary(schema) { +function acceptedValueSummary(schema) { for (const keyword of ['anyOf', 'oneOf']) { if (Array.isArray(schema[keyword])) { - return schema[keyword].map(schemaSummary).filter(Boolean).join(' or '); + return schema[keyword].map(acceptedValueSummary).filter(Boolean).join(' or '); } } - if (typeof schema.$ref === 'string') return `reference: ${schema.$ref}`; + if (typeof schema.$ref === 'string') return escapeCell(`reference: ${schema.$ref}`); const types = Array.isArray(schema.type) ? schema.type : [schema.type]; - const typeLabel = types.filter(Boolean).join(', '); + const typeLabel = escapeCell(types.filter(Boolean).join(', ')); if (Array.isArray(schema.enum)) { - return `${typeLabel}; one of: ${schema.enum.map(String).join(', ')}`; + const values = schema.enum.map((value) => { + const label = typeof value === 'string' ? value : JSON.stringify(value); + return `
  • ${escapeCell(label)}
    • `; + }).join(''); + return `${typeLabel}; one of:
      ${values}
    `; } return typeLabel; } @@ -624,8 +672,249 @@ function jsonValue(value) { return value === undefined ? '—' : `\`${JSON.stringify(value)}\``; } +function sampleValue(value) { + if (value === null || value === undefined) return ''; + if (typeof value === 'object') return escapeCell(JSON.stringify(value)); + return escapeCell(String(value)); +} + +function fixtureTable(fixture) { + if (!Array.isArray(fixture) || !fixture.length || !fixture.every(isObject)) { + return null; + } + + const columns = [...new Set(fixture.flatMap((record) => Object.keys(record)))]; + if (!columns.length) return null; + + const header = `| ${columns.map(escapeCell).join(' | ')} |`; + const separator = `| ${columns.map(() => '---').join(' | ')} |`; + const rows = fixture.map((record) => + `| ${columns.map((column) => sampleValue(record[column])).join(' | ')} |`, + ); + return [header, separator, ...rows].join('\n'); +} + +function parseSampleTable(source) { + const lines = String(source || '').trim().split(/\r?\n/); + if (lines.length < 2) return null; + + const headers = splitMarkdownRow(lines[0]); + const separator = splitMarkdownRow(lines[1]); + if ( + !headers.length || + separator.length !== headers.length || + separator.some((cell) => !/^:?-{3,}:?$/.test(cell)) + ) { + return null; + } + + const rows = lines.slice(2).map(splitMarkdownRow); + if (!rows.length || rows.some((row) => row.length !== headers.length)) return null; + return {headers, rows}; +} + +function renderSampleTable(table) { + const preserveCell = (value) => String(value ?? '').replace(/\|/g, '\\|'); + return [ + `| ${table.headers.map(preserveCell).join(' | ')} |`, + `| ${table.headers.map(() => '---').join(' | ')} |`, + ...table.rows.map((row) => `| ${row.map(preserveCell).join(' | ')} |`), + ].join('\n'); +} + +function selectSampleColumns(table, indexes) { + return { + headers: indexes.map((index) => table.headers[index]), + rows: table.rows.map((row) => indexes.map((index) => row[index] ?? '')), + }; +} + +function normalizedSampleColumn(value) { + return String(value || '') + .replace(/`/g, '') + .replace(/\s*\([^)]*\b(?:input|output)\b[^)]*\)\s*$/i, '') + .replace(/\s+/g, ' ') + .trim() + .toLowerCase(); +} + +function recipeParameterValues(recipeSource, parameter) { + try { + const document = yaml.load(recipeSource); + const wrangle = Array.isArray(document?.wrangles) ? + document.wrangles.find(isObject) : null; + const configuration = wrangle ? Object.values(wrangle)[0] : null; + const value = isObject(configuration) ? configuration[parameter] : null; + const values = Array.isArray(value) ? value : [value]; + return values + .filter((item) => typeof item === 'string' || typeof item === 'number') + .map(String); + } catch { + return []; + } +} + +function inferInputTable(recipeSource, outputTable) { + const inputColumns = recipeParameterValues(recipeSource, 'input'); + if (!inputColumns.length || !outputTable) return null; + + const indexes = inputColumns.map((column) => + outputTable.headers.findIndex((header) => + normalizedSampleColumn(header) === normalizedSampleColumn(column)), + ); + if (indexes.some((index) => index < 0) || new Set(indexes).size !== indexes.length) { + return null; + } + return selectSampleColumns(outputTable, indexes); +} + +function resultOnlyTable(inputTable, outputTable) { + if (!inputTable || !outputTable) return outputTable; + + const retainedIndexes = outputTable.headers.map((header, outputIndex) => { + const inputIndex = inputTable.headers.findIndex((inputHeader) => + normalizedSampleColumn(inputHeader) === normalizedSampleColumn(header)); + if (inputIndex < 0) return outputIndex; + + const inputValues = inputTable.rows.map((row) => row[inputIndex] ?? ''); + const outputValues = outputTable.rows.map((row) => row[outputIndex] ?? ''); + return valuesEqual(inputValues, outputValues) ? null : outputIndex; + }).filter((index) => index !== null); + + // Structural wrangles such as select/drop can return only unchanged columns. + // Retain their full result so the changed table shape remains visible. + return retainedIndexes.length ? selectSampleColumns(outputTable, retainedIndexes) : outputTable; +} + +function rewriteSampleGrids(content) { + const pattern = /```yaml\r?\n([\s\S]*?)\r?\n```\r?\n\r?\n
    \r?\n\r?\n
    \r?\n\r?\n##### Input Sample\r?\n\r?\n([\s\S]*?)\r?\n\r?\n<\/div>\r?\n\r?\n
    \r?\n\r?\n##### Output Sample\r?\n\r?\n([\s\S]*?)\r?\n\r?\n<\/div>\r?\n\r?\n<\/div>/g; + + return content.replace(pattern, (match, recipeSource, inputSource, outputSource) => { + let inputTable = parseSampleTable(inputSource); + let outputTable = parseSampleTable(outputSource); + if (!inputTable && /No sample available/i.test(inputSource)) { + inputTable = inferInputTable(recipeSource, outputTable); + } + if (inputTable && outputTable) outputTable = resultOnlyTable(inputTable, outputTable); + + const normalizedInput = inputTable ? renderSampleTable(inputTable) : inputSource.trim(); + const normalizedOutput = outputTable ? renderSampleTable(outputTable) : outputSource.trim(); + return `\`\`\`yaml +${recipeSource} +\`\`\` + +
    + +
    + +${normalizedInput} + +
    + +
    + +${normalizedOutput} + +
    + +
    `; + }); +} + +function renderFixturePanel(role, fixture, normalizedTable = null) { + const table = normalizedTable ? renderSampleTable(normalizedTable) : fixtureTable(fixture); + const content = table || [ + '```json', + JSON.stringify(fixture, null, 2), + '```', + ].join('\n'); + + return `
    + +${content} + +
    `; +} + +function renderPublicExample(example, grouped = false) { + const inputTable = parseSampleTable(fixtureTable(example._input_fixture)); + const originalOutputTable = parseSampleTable(fixtureTable(example._output_fixture)); + const outputTable = resultOnlyTable(inputTable, originalOutputTable); + + return `\`\`\`yaml +${example.recipe.trim()} +\`\`\` + +
    + +${renderFixturePanel('input', example._input_fixture, inputTable)} + +${renderFixturePanel('output', example._output_fixture, outputTable)} + +
    `; +} + +function normalizePublicBody(body, description, grouped = false) { + let content = rewriteSampleGrids(stripLeadingTitle(body)); + const blocks = content.split(/\r?\n\r?\n/); + const normalizeText = (value) => value.replace(/\s+/g, ' ').trim(); + if (blocks.length && normalizeText(blocks[0]) === normalizeText(description)) { + content = blocks.slice(1).join('\n\n').trim(); + } + + let inMigratedExamples = false; + return content.split(/\r?\n/).map((line) => { + if (/^## Migrated examples\s*$/.test(line)) { + inMigratedExamples = true; + return grouped ? '### Examples' : '## Examples'; + } + if (inMigratedExamples && /^##\s+/.test(line)) { + inMigratedExamples = false; + } + if (inMigratedExamples && ( + /^####\s+/.test(line) || + /^#####\s+(?:Recipe|Input Sample|Output Sample)\s*$/.test(line) + )) { + return ''; + } + return grouped ? line.replace(/^##(\s+)/, '###$1') : line; + }).join('\n'); +} + +function legacyDocumentationGroup(entry) { + for (const source of entry.metadata.sources) { + const match = source.resource.match(/\/wrangle-docs\/([^/]+)\/_sources\//); + if (match) return match[1]; + } + return null; +} + +function entryGroup(entry) { + return entry.metadata.namespace || + legacyDocumentationGroup(entry) || + LEGACY_GROUP_OVERRIDES[entry.metadata.wrangle_key] || + 'other'; +} + +function groupLabel(group) { + if (group === 'ai') return 'AI'; + return group + .split(/[-_]/) + .filter(Boolean) + .map((part) => part[0].toUpperCase() + part.slice(1)) + .join(' '); +} + +function groupRoute(group) { + return `/wrangles/namespaces/${group}`; +} + +function entryAnchor(entry) { + return entry.metadata.slug.split('/').at(-1); +} + function entryRoute(entry) { - return `/wrangles/${entry.metadata.slug}`; + return `${groupRoute(entryGroup(entry))}#${entryAnchor(entry)}`; } function entryOutputRelativePath(entry) { @@ -650,7 +939,7 @@ function combinedParameters(entry, controls) { fail(entry.sourceFile, `capability ${name} duplicates an entry parameter`); continue; } - parameters.push({name, ...controls[name], role: 'common-control'}); + parameters.push({name, ...controls[name]}); } return parameters; } @@ -659,115 +948,237 @@ function stripLeadingTitle(body) { return body.replace(/^#\s+[^\r\n]+\r?\n*/, '').trim(); } -function renderEntryPage(entry, controls) { +function splitExamplesSection(content, sectionHeading) { + const lines = content.split(/\r?\n/); + const heading = `${sectionHeading} Examples`; + const index = lines.findIndex((line) => line.trim() === heading); + if (index === -1) return {guidance: content.trim(), examples: ''}; + return { + guidance: lines.slice(0, index).join('\n').trim(), + examples: lines.slice(index + 1).join('\n').trim(), + }; +} + +function renderEntryContent(entry, controls, {grouped = false, entriesByKey = new Map()} = {}) { const metadata = entry.metadata; + const sectionHeading = grouped ? '###' : '##'; const parameters = combinedParameters(entry, controls); - const parameterRows = parameters.map((parameter) => [ - `\`${parameter.name}\``, - parameter.required ? 'Yes' : 'No', - escapeCell(schemaSummary(parameter.schema)), - escapeCell(parameter.description), - hasOwn(parameter, 'runtime_default') ? jsonValue(parameter.runtime_default) : '—', - ]); + const parameterRows = PARAM_GROUP_ORDER.flatMap((paramGroup) => { + const groupedParameters = parameters.filter( + (parameter) => parameter.param_group === paramGroup, + ); + if (!groupedParameters.length) return []; + return [ + [`${paramGroup}`, '', '', '', ''], + ...groupedParameters.map((parameter) => [ + `\`${parameter.name}\``, + escapeCell(parameter.description), + acceptedValueSummary(parameter.schema), + hasOwn(parameter, 'runtime_default') ? jsonValue(parameter.runtime_default) : '—', + parameter.required ? 'Yes' : 'No', + ]), + ]; + }); - const examples = metadata.examples.map((example) => [ - `### ${example.title}`, - '', - `Verification: \`${example.verification}\``, - '', - '```yaml', - example.recipe.trim(), - '```', - '', - '#### Input', - '', - '```json', - JSON.stringify(example._input_fixture, null, 2), - '```', - '', - '#### Output', - '', - '```json', - JSON.stringify(example._output_fixture, null, 2), - '```', - ].join('\n')).join('\n\n'); + const fixtureExamples = metadata.examples + .map((example) => renderPublicExample(example, grouped)) + .join('\n\n'); const sourceRows = metadata.sources.map((source) => `- [${source.title || source.id}](${source.resource})`, ).join('\n'); + const accessLabels = { + ai_powered: 'AI-powered', + requires_account: 'Requires WrangleWorks account', + requires_subscription: 'Requires subscription', + requires_external_api_key: 'Requires external API key', + }; const accessRows = Object.entries(metadata.access).map(([key, value]) => - `| ${escapeCell(key.replaceAll('_', ' '))} | ${value ? 'Yes' : 'No'} |`, + `| ${accessLabels[key] || escapeCell(key.replaceAll('_', ' '))} | ${value ? 'Yes' : 'No'} |`, ).join('\n'); - const guidance = stripLeadingTitle(entry.body); - return `--- -title: "${metadata.title.replaceAll('"', '\\"')}" -description: "${metadata.description.replaceAll('"', '\\"')}" -sidebar_label: "${metadata.title.replaceAll('"', '\\"')}" -slug: "/${metadata.slug}" ---- - -# ${metadata.title} - -${escapeMdxText(metadata.description)} - -> Pilot Registry entry. Runtime contract status: \`${metadata.runtime.contract_status}\`. - -## Parameters - -| Parameter | Required | Accepted value | Description | Runtime default | + const normalizedBody = normalizePublicBody(entry.body, metadata.description, grouped); + const {guidance, examples: migratedExamples} = splitExamplesSection( + normalizedBody, + sectionHeading, + ); + const examples = [migratedExamples, fixtureExamples].filter(Boolean).join('\n\n'); + const examplesSection = examples ? `${sectionHeading} Examples\n\n${examples}` : ''; + const namespace = metadata.namespace || 'Root-level'; + const documentationGroup = entryGroup(entry); + const aliases = metadata.aliases.length ? + metadata.aliases.map((alias) => `\`${alias}\``).join(', ') : 'None'; + const lifecycleSuffix = metadata.status === 'deprecated' ? ' (Deprecated)' : + metadata.status === 'removed' ? ' (Removed)' : ''; + const displayTitle = `${metadata.title}${lifecycleSuffix}`; + const replacement = metadata.replaced_by ? entriesByKey.get(metadata.replaced_by) : null; + const replacementLink = replacement ? + `[\`${metadata.replaced_by}\`](${entryRoute(replacement)})` : + metadata.replaced_by ? `\`${metadata.replaced_by}\`` : ''; + const lifecycleNotice = metadata.status === 'deprecated' ? `:::warning Deprecated +This compatibility wrangle remains available for existing recipes. Use ${replacementLink} for new recipes. +::: + +` : metadata.status === 'removed' ? `:::danger Removed +This wrangle is retained only to document historical recipes.${replacementLink ? ` Use ${replacementLink} instead.` : ''} +::: + +` : ''; + const entryHeading = grouped ? + `## ${displayTitle} {#${entryAnchor(entry)}}\n\n` : ''; + return `${entryHeading}${lifecycleNotice}${escapeMdxText(metadata.description)} + +${guidance} + +${sectionHeading} Parameters + +
    + +| Name | Description | Accepted Values | Default | Required | | --- | --- | --- | --- | --- | ${parameterRows.map((row) => `| ${row.join(' | ')} |`).join('\n')} -## Verified examples +
    -${examples || '_No fixture-backed examples are currently available. See migrated examples under Guidance where present._'} +${examplesSection} -## Access +
    + +Access | Requirement | Value | | --- | --- | ${accessRows} -## Guidance +
    -${guidance || '_No additional guidance._'} +
    -## Provenance +Technical details + +| Field | Value | +| --- | --- | +| Recipe key | \`${metadata.wrangle_key}\` | +| Lifecycle status | ${metadata.status} | +${metadata.replaced_by ? `| Replaced by | ${replacementLink} |\n` : ''}| Namespace | ${namespace === 'Root-level' ? namespace : `\`${namespace}\``} | +| Documentation group | \`${documentationGroup}\` | +| Aliases | ${aliases} | +| Runtime symbol | \`${metadata.runtime.symbol}\` | + +**Sources** ${sourceRows} -## Registry metadata +
    +`; +} + +function renderEntryPage(entry, controls, entriesByKey) { + const metadata = entry.metadata; + const lifecycleSuffix = metadata.status === 'deprecated' ? ' (Deprecated)' : + metadata.status === 'removed' ? ' (Removed)' : ''; + const displayTitle = `${metadata.title}${lifecycleSuffix}`; + return `--- +title: "${displayTitle.replaceAll('"', '\\"')}" +description: "${metadata.description.replaceAll('"', '\\"')}" +sidebar_label: "${displayTitle.replaceAll('"', '\\"')}" +slug: "/${metadata.slug}" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# ${displayTitle} + +${renderEntryContent(entry, controls, {entriesByKey})}`; +} + +function groupedEntries(entries) { + const statusRank = {active: 0, draft: 1, deprecated: 2, removed: 3}; + const groupsByName = new Map(); + for (const entry of entries) { + const group = entryGroup(entry); + if (!groupsByName.has(group)) groupsByName.set(group, []); + groupsByName.get(group).push(entry); + } + + const orderOf = (group) => { + const index = GROUP_ORDER.indexOf(group); + return index < 0 ? Number.MAX_SAFE_INTEGER : index; + }; + return [...groupsByName.entries()] + .map(([group, groupItems]) => ({ + group, + entries: groupItems.sort((left, right) => + (statusRank[left.metadata.status] ?? 99) - (statusRank[right.metadata.status] ?? 99) || + left.metadata.wrangle_key.localeCompare(right.metadata.wrangle_key), + ), + })) + .sort((left, right) => + orderOf(left.group) - orderOf(right.group) || left.group.localeCompare(right.group), + ); +} + +function renderNamespacePage(grouped, controls, entriesByKey) { + const label = groupLabel(grouped.group); + const description = `${label} wrangles, with recipe examples, parameters, and behavior.`; + const sections = grouped.entries.map((entry) => + renderEntryContent(entry, controls, {grouped: true, entriesByKey}), + ).join('\n\n---\n\n'); + return `--- +title: "${label} Wrangles" +description: "${description}" +sidebar_label: "${label}" +slug: "/namespaces/${grouped.group}" +registry_entry: true +toc_min_heading_level: 2 +toc_max_heading_level: 3 +--- + +# ${label} Wrangles -- Registry ID: ${metadata.id ? `\`${metadata.id}\`` : 'pending database assignment'} -- Namespace: ${metadata.namespace ? `\`${metadata.namespace}\`` : 'root-level runtime key'} -- Recipe key: \`${metadata.wrangle_key}\` -- Aliases: ${metadata.aliases.length ? metadata.aliases.map((alias) => `\`${alias}\``).join(', ') : 'none'} -- Runtime symbol: \`${metadata.runtime.symbol}\` -- Status: \`${metadata.status}\` -- Registry version: \`${REGISTRY_VERSION}\` +${description} + +${sections}`; +} + +function renderRegistrySidebar(groups) { + const items = groups.map(({group}) => + ` {type: 'doc', id: 'namespaces/${group}', label: '${groupLabel(group)}'},`, + ).join('\n'); + return `/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */ +const sidebarsRegistry = { + registrySidebar: [ + {type: 'doc', id: 'index', label: 'Registry'}, +${items} + ], +}; + +export default sidebarsRegistry; `; } -function renderDocsIndex(entries) { - const rows = entries.map((entry) => - `| [\`${entry.metadata.wrangle_key}\`](${entryRoute(entry)}) | ${escapeCell(entry.metadata.description)} | \`${entry.metadata.runtime.contract_status}\` |`, +function renderDocsIndex(groups) { + const rows = groups.map(({group, entries}) => + `| [${groupLabel(group)}](${groupRoute(group)}) | ${entries.length} | ${groupLabel(group)} wrangles and compatibility entries. |`, ).join('\n'); return `--- -title: Wrangles Registry Pilot -description: Pilot of the versioned Wrangles recipe knowledge registry. +title: Wrangles Registry +description: Pre-production version of the Wrangles recipe knowledge registry. slug: / +registry_entry: true --- -# Wrangles Registry Pilot +# Wrangles Registry -This preview contains the first Registry records compiled from the new -Markdown contract. These pages are not yet the production replacement for the -existing wrangle reference. +These pages are compiled from the Registry Markdown contract. The Registry is +pre-production until the migration and production-readiness criteria in its +contract are complete. Wrangles are grouped using their Registry namespace; +root-level compatibility keys remain in their existing documentation group. -| Wrangle | Description | Runtime contract | -| --- | --- | --- | +| Namespace or group | Wrangles | Description | +| --- | ---: | --- | ${rows} `; } @@ -781,15 +1192,15 @@ function renderRawIndex(entries) { okf_version: "0.2" type: collection title: Wrangles Registry -description: Public pilot bundle for Wrangles recipe primitives. -status: pilot +description: Public pre-production bundle for Wrangles recipe primitives. +status: pre-production registry_version: ${REGISTRY_VERSION} --- # Wrangles Registry -This is the public, agent-readable pilot bundle. Use \`manifest.json\` for -structured discovery. +This is the public, agent-readable pre-production bundle. Use \`manifest.json\` +for structured discovery. ${rows} `; @@ -858,10 +1269,10 @@ function buildRecipeSchema(entries, controls) { return { $schema: 'http://json-schema.org/draft-07/schema#', - $id: `${PUBLIC_ORIGIN}/schemas/recipes/pilot/schema.json`, - title: 'Wrangles Recipe Schema - Registry Pilot', - description: 'Pilot schema containing only the initial Registry entries.', - $comment: 'Not the production recipe schema. Generated from Registry version 0.1.0-pilot.', + $id: `${PUBLIC_ORIGIN}/schemas/recipes/registry/schema.json`, + title: 'Wrangles Recipe Schema - Registry', + description: 'Pre-production schema generated from the Docs Registry.', + $comment: 'Pre-production artifact. See the Registry contract for production-readiness criteria.', type: 'object', additionalProperties: false, required: ['wrangles'], @@ -928,6 +1339,7 @@ function buildEntryContract(entry, controls) { title: metadata.title, description: metadata.description, status: metadata.status, + ...(metadata.replaced_by ? {replaced_by: metadata.replaced_by} : {}), visibility: metadata.visibility, tags: metadata.tags, runtime: metadata.runtime, @@ -1518,7 +1930,11 @@ async function readInputs() { throw new Error('registry/common/wrangle-controls.yaml is invalid'); } for (const [name, control] of Object.entries(commonDocument.controls)) { - if (!isObject(control) || typeof control.description !== 'string') { + if ( + !isObject(control) || + typeof control.description !== 'string' || + !PARAM_GROUP_ORDER.includes(control.param_group) + ) { throw new Error(`Common control ${name} is invalid`); } validateSchemaFragment(control.schema, commonControlsPath, `controls.${name}`); @@ -1553,6 +1969,11 @@ async function readInputs() { recipeKeys.add(recipeKey); } } + for (const entry of entries) { + if (entry.metadata.replaced_by && !recipeKeys.has(entry.metadata.replaced_by)) { + fail(entry.sourceFile, `replaced_by references unknown recipe key ${entry.metadata.replaced_by}`); + } + } return { entries, @@ -1571,11 +1992,21 @@ function buildOutputs(entries, controls, reconciliation, registrySchemas, runtim .filter((entry) => entry.metadata.visibility === 'public') .sort((left, right) => left.metadata.wrangle_key.localeCompare(right.metadata.wrangle_key)); - addGeneratedFile(path.join(docsOutputRoot, 'index.md'), renderDocsIndex(publicEntries)); + const groups = groupedEntries(publicEntries); + const entriesByKey = new Map(publicEntries.map((entry) => [entry.metadata.wrangle_key, entry])); + + addGeneratedFile(path.join(docsOutputRoot, 'index.md'), renderDocsIndex(groups)); + addGeneratedFile(registrySidebarPath, renderRegistrySidebar(groups)); + for (const group of groups) { + addGeneratedFile( + path.join(docsOutputRoot, 'namespaces', `${group.group}.md`), + renderNamespacePage(group, controls, entriesByKey), + ); + } for (const entry of publicEntries) { addGeneratedFile( path.join(docsOutputRoot, entryOutputRelativePath(entry)), - renderEntryPage(entry, controls), + renderEntryPage(entry, controls, entriesByKey), ); const rawRelative = path.relative(registryRoot, entry.sourceFile); addGeneratedFile(path.join(rawOutputRoot, rawRelative), entry.source); @@ -1600,7 +2031,7 @@ function buildOutputs(entries, controls, reconciliation, registrySchemas, runtim format: 'wrangles-registry', registry_version: REGISTRY_VERSION, contract_version: ENTRY_SCHEMA_VERSION, - status: 'pilot', + status: 'pre-production', entry_count: publicEntries.length, entries: publicEntries.map((entry) => ({ type: entry.metadata.type, @@ -1613,6 +2044,7 @@ function buildOutputs(entries, controls, reconciliation, registrySchemas, runtim title: entry.metadata.title, description: entry.metadata.description, status: entry.metadata.status, + ...(entry.metadata.replaced_by ? {replaced_by: entry.metadata.replaced_by} : {}), tags: entry.metadata.tags, route: entryRoute(entry), contract_json: `/registry/${posixPath(entryContractRelativePath(entry))}`, @@ -1623,7 +2055,7 @@ function buildOutputs(entries, controls, reconciliation, registrySchemas, runtim })), artifacts: { index: '/registry/index.md', - recipe_schema: '/schemas/recipes/pilot/schema.json', + recipe_schema: '/schemas/recipes/registry/schema.json', entry_schema: '/registry/schema/wrangle-entry.schema.json', runtime_manifest: '/registry/runtime/wranglespy.json', runtime_manifest_schema: '/registry/schema/wrangles-runtime-manifest.schema.json', diff --git a/wrangles-docs/sidebarsRegistry.js b/wrangles-docs/sidebarsRegistry.js index bba4c8a..1b01c1a 100644 --- a/wrangles-docs/sidebarsRegistry.js +++ b/wrangles-docs/sidebarsRegistry.js @@ -1,6 +1,25 @@ /** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */ const sidebarsRegistry = { - registrySidebar: [{type: 'autogenerated', dirName: '.'}], + registrySidebar: [ + {type: 'doc', id: 'index', label: 'Registry'}, + {type: 'doc', id: 'namespaces/convert', label: 'Convert'}, + {type: 'doc', id: 'namespaces/merge', label: 'Merge'}, + {type: 'doc', id: 'namespaces/split', label: 'Split'}, + {type: 'doc', id: 'namespaces/select', label: 'Select'}, + {type: 'doc', id: 'namespaces/format', label: 'Format'}, + {type: 'doc', id: 'namespaces/create', label: 'Create'}, + {type: 'doc', id: 'namespaces/extract', label: 'Extract'}, + {type: 'doc', id: 'namespaces/compare', label: 'Compare'}, + {type: 'doc', id: 'namespaces/compute', label: 'Compute'}, + {type: 'doc', id: 'namespaces/search', label: 'Search'}, + {type: 'doc', id: 'namespaces/ai', label: 'AI'}, + {type: 'doc', id: 'namespaces/generate', label: 'Generate'}, + {type: 'doc', id: 'namespaces/lookup', label: 'Lookup'}, + {type: 'doc', id: 'namespaces/standardize', label: 'Standardize'}, + {type: 'doc', id: 'namespaces/transform', label: 'Transform'}, + {type: 'doc', id: 'namespaces/utility', label: 'Utility'}, + {type: 'doc', id: 'namespaces/date', label: 'Date'}, + ], }; export default sidebarsRegistry; diff --git a/wrangles-docs/src/css/custom.css b/wrangles-docs/src/css/custom.css index 28bc29a..7265840 100644 --- a/wrangles-docs/src/css/custom.css +++ b/wrangles-docs/src/css/custom.css @@ -338,12 +338,12 @@ body { /* Wrangle pages use explicit panels to keep sample labels attached to their data. */ .ww-wrangle-doc-scope .theme-doc-markdown > h2:not([id^="metadata"]) { - background: #e9f3fb; - border: 1px solid #c7dceb; - border-bottom: 1px solid #c7dceb; - border-left: 6px solid #ecbf5d; + background: #4472c4; + border: 1px solid #4472c4; + border-bottom: 1px solid #4472c4; + border-left: 6px solid #4472c4; border-radius: 4px; - color: #124f78; + color: #ffffff; margin-top: 3.25rem; padding: 0.72rem 0.9rem; } @@ -354,45 +354,123 @@ body { .ww-wrangle-doc-scope .ww-sample-grid { align-items: stretch; + background: #ffffff; + border: 1px solid #aeb9c6; + border-radius: 5px; display: grid; - gap: 2.5rem; + gap: 0; grid-template-columns: repeat(2, minmax(0, 1fr)); margin: 1.25rem 0; + overflow: hidden; position: relative; } .ww-wrangle-doc-scope .ww-sample-grid::before { - background: #cbd5df; - bottom: 0; - content: ""; - left: 50%; - position: absolute; - top: 0; - transform: translateX(-50%); - width: 1px; + display: none; } .ww-wrangle-doc-scope .ww-sample-panel { - background: #f8fafc; - border: 1px solid #d6dde8; - border-radius: 4px; + background: #ffffff; + border: 0; + border-radius: 0; min-width: 0; overflow-x: auto; - padding: 1rem; + padding: 0; } -.ww-wrangle-doc-scope .ww-sample-panel > h5:first-child { - color: #234263; - margin: 0 0 0.8rem; +.ww-wrangle-doc-scope .ww-sample-panel + .ww-sample-panel { + border-left: 4px solid #8795a5; } .ww-wrangle-doc-scope .ww-sample-panel > table { + border-collapse: collapse; + border-spacing: 0; display: table; margin: 0; min-width: 100%; width: 100%; } +.ww-wrangle-doc-scope .ww-sample-panel > table th, +.ww-wrangle-doc-scope .ww-sample-panel > table td { + border: 1px solid #c8d0d9; + padding: 0.48rem 0.62rem; + text-align: left; + vertical-align: top; +} + +.ww-wrangle-doc-scope .ww-sample-panel > table thead th { + background: #294d6b; + color: #ffffff; + font-weight: 700; + white-space: nowrap; +} + +.ww-wrangle-doc-scope .ww-sample-panel + .ww-sample-panel > table thead th { + background: #548235; +} + +.ww-wrangle-doc-scope .ww-sample-panel > table tbody tr:nth-child(even) td { + background: #f3f6f9; +} + +.ww-wrangle-doc-scope .ww-sample-panel > table tbody tr:hover td { + background: #fff8d9; +} + +.ww-wrangle-doc-scope .ww-parameters-table { + overflow-x: auto; +} + +.ww-wrangle-doc-scope .ww-parameters-table > table { + display: table; + min-width: 100%; + width: 100%; +} + +.ww-wrangle-doc-scope .ww-parameters-table tbody tr:has(.ww-param-group-label) td { + background: #294d6b; + border-bottom-color: #294d6b; + border-top: 2px solid #1f3b53; + color: #ffffff; + font-size: 0.82rem; + font-weight: 800; + letter-spacing: 0.045em; + padding-bottom: 0.42rem; + padding-top: 0.42rem; + text-transform: uppercase; +} + +.ww-wrangle-doc-scope .ww-parameters-table tbody tr:has(.ww-param-group-label) td:not(:first-child) { + border-left-color: #294d6b; +} + +.ww-wrangle-doc-scope .ww-parameters-table tbody td:first-child code { + background: #dbeafe; + border: 1px solid #93c5fd; + color: #0b4f84; + font-weight: 700; + padding: 0.18rem 0.38rem; +} + +.ww-wrangle-doc-scope .ww-parameters-table tbody td:nth-child(4) code { + background: #f0f3f6; + border: 1px solid #d4dbe2; + color: #465563; + font-weight: 600; + padding: 0.14rem 0.32rem; + white-space: nowrap; +} + +.ww-wrangle-doc-scope .ww-param-enum-values { + margin: 0.32rem 0 0; + padding-left: 1.15rem; +} + +.ww-wrangle-doc-scope .ww-param-enum-values > li { + margin: 0.08rem 0; +} + .ww-wrangle-doc-scope .ww-editable-sample-cell { background: #fffdf8; box-shadow: inset 0 0 0 1px #d6dde8; @@ -430,12 +508,12 @@ body { @media (max-width: 760px) { .ww-wrangle-doc-scope .ww-sample-grid { - gap: 1rem; grid-template-columns: minmax(0, 1fr); } - .ww-wrangle-doc-scope .ww-sample-grid::before { - display: none; + .ww-wrangle-doc-scope .ww-sample-panel + .ww-sample-panel { + border-left: 0; + border-top: 4px solid #8795a5; } .ww-wrangle-doc-scope .theme-doc-markdown > h2:not([id^="metadata"]) { diff --git a/wrangles-docs/src/theme/CodeBlock/Buttons/index.js b/wrangles-docs/src/theme/CodeBlock/Buttons/index.js index 89552c8..3c81590 100644 --- a/wrangles-docs/src/theme/CodeBlock/Buttons/index.js +++ b/wrangles-docs/src/theme/CodeBlock/Buttons/index.js @@ -97,11 +97,20 @@ function findSampleGrid(control) { } function findSamplePanel(grid, label) { - return [...(grid?.querySelectorAll('.ww-sample-panel') ?? [])].find((panel) => { + const panels = [...(grid?.querySelectorAll('.ww-sample-panel') ?? [])]; + const role = label.startsWith('input') ? 'input' : 'output'; + const explicitPanel = grid?.querySelector( + `[data-sample-role="${role}"], .ww-sample-panel--${role}`, + ); + if (explicitPanel) return explicitPanel; + + const labelledPanel = panels.find((panel) => { const heading = panel.querySelector('h5'); const headingLabel = heading?.firstChild?.textContent.trim().toLowerCase(); return headingLabel === label; }); + if (labelledPanel) return labelledPanel; + return role === 'input' ? panels[0] : panels.at(-1); } function findInputTable(control, recipe) { @@ -162,7 +171,8 @@ function setRunError(control, message) { function replaceOutputTable(control, payload, documentedColumns) { const grid = findSampleGrid(control); const outputPanel = findSamplePanel(grid, 'output sample'); - const columns = payload.columns?.length ? payload.columns : documentedColumns; + const payloadColumns = Array.isArray(payload.columns) ? payload.columns : []; + const columns = documentedColumns.length ? documentedColumns : payloadColumns; const rows = Array.isArray(payload.rows) ? payload.rows : []; if (!outputPanel || !columns.length) { @@ -186,11 +196,15 @@ function replaceOutputTable(control, payload, documentedColumns) { thead.appendChild(headingRow); const tbody = document.createElement('tbody'); + const columnIndexes = columns.map((column, index) => { + const payloadIndex = payloadColumns.indexOf(column); + return payloadIndex >= 0 ? payloadIndex : index; + }); rows.forEach((row) => { const tableRow = document.createElement('tr'); - columns.forEach((_, columnIndex) => { + columnIndexes.forEach((sourceIndex) => { const cell = document.createElement('td'); - cell.textContent = String(row?.[columnIndex] ?? ''); + cell.textContent = String(row?.[sourceIndex] ?? ''); tableRow.appendChild(cell); }); tbody.appendChild(tableRow); diff --git a/wrangles-docs/src/theme/DocItem/Layout/index.js b/wrangles-docs/src/theme/DocItem/Layout/index.js index 5e9ef76..50e0713 100644 --- a/wrangles-docs/src/theme/DocItem/Layout/index.js +++ b/wrangles-docs/src/theme/DocItem/Layout/index.js @@ -26,6 +26,9 @@ function useDocTOC() { const windowSize = useWindowSize(); const hidden = frontMatter.hide_table_of_contents; const canRender = !hidden && toc.length > 0; + const isWrangleDoc = Boolean( + frontMatter.wrangle_type || frontMatter.registry_entry, + ); const mobile = canRender ? : undefined; const desktop = canRender && (windowSize === 'desktop' || windowSize === 'ssr') ? ( @@ -35,7 +38,7 @@ function useDocTOC() { hidden, mobile, desktop, - collapsible: Boolean(frontMatter.wrangle_type), + collapsible: isWrangleDoc, }; } function TocToggle({collapsed, onClick}) { diff --git a/wrangles-docs/src/theme/DocItem/TOC/Desktop/index.js b/wrangles-docs/src/theme/DocItem/TOC/Desktop/index.js index b1dfff5..d732ec6 100644 --- a/wrangles-docs/src/theme/DocItem/TOC/Desktop/index.js +++ b/wrangles-docs/src/theme/DocItem/TOC/Desktop/index.js @@ -124,7 +124,7 @@ function WrangleTOC({toc, minHeadingLevel, maxHeadingLevel}) { export default function DocItemTOCDesktop() { const {toc, frontMatter} = useDoc(); - if (!frontMatter.wrangle_type) { + if (!frontMatter.wrangle_type && !frontMatter.registry_entry) { return ( { const isWrangleMode = location.pathname === '/wrangle' || - location.pathname.startsWith('/wrangle/'); + location.pathname.startsWith('/wrangle/') || + location.pathname === '/wrangles' || + location.pathname.startsWith('/wrangles/'); document.body.classList.toggle('ww-wranglespy-mode', isWrangleMode); return () => document.body.classList.remove('ww-wranglespy-mode'); }, [location.pathname]); diff --git a/wrangles-docs/static/registry/contracts/accordion.json b/wrangles-docs/static/registry/contracts/accordion.json index 09eeea6..108d3c3 100644 --- a/wrangles-docs/static/registry/contracts/accordion.json +++ b/wrangles-docs/static/registry/contracts/accordion.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "wrangles", "description": "List of wrangles to apply.", "required": true, - "role": "nested-wrangles", + "param_group": "Execution", "schema": { "type": "array", "minItems": 1, @@ -51,7 +51,7 @@ "name": "input", "description": "The column(s) containing the list(s) that the wrangles will be applied to the elements of.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -64,7 +64,7 @@ "name": "output", "description": "Output of the wrangles to save back to the dataframe.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -78,7 +78,7 @@ "name": "propagate", "description": "Limit the column(s) that will be available to the wrangles and replicated for each element. If not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets.", "required": false, - "role": "option", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -92,31 +92,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/batch.json b/wrangles-docs/static/registry/contracts/batch.json index b3fdff6..2f8cbeb 100644 --- a/wrangles-docs/static/registry/contracts/batch.json +++ b/wrangles-docs/static/registry/contracts/batch.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "wrangles", "description": "The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size.", "required": true, - "role": "nested-wrangles", + "param_group": "Execution", "schema": { "type": "array", "minItems": 1, @@ -51,7 +51,7 @@ "name": "batch_size", "description": "The number of rows to split each batch into.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 1000, "schema": { "type": "integer" @@ -61,7 +61,7 @@ "name": "threads", "description": "The number of threads to use for parallel processing. Default 1.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 1, "schema": { "type": "integer" @@ -71,7 +71,7 @@ "name": "on_error", "description": "A dictionary of column_name: value to return if an error occurs while attempting to run a batch.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -84,7 +84,7 @@ "name": "timeout", "description": "The number of seconds to wait for a batch to complete before raising an error.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": null, "schema": { "type": [ @@ -97,7 +97,7 @@ "name": "use_multiprocessing", "description": "Use process-based workers instead of threads. This is an advanced runtime option.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": false, "schema": { "type": "boolean" @@ -107,31 +107,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/classify.json b/wrangles-docs/static/registry/contracts/classify.json index 3ec0acf..d776eaf 100644 --- a/wrangles-docs/static/registry/contracts/classify.json +++ b/wrangles-docs/static/registry/contracts/classify.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "06669ef8-cdd1-42f7-8078-98e0b7a42c30", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "model_id", "description": "ID of the classification model to be used.", "required": true, - "role": "model-reference", + "param_group": "Details", "schema": { "type": "string" } @@ -72,7 +72,7 @@ "name": "include_confidence", "description": "For models that support it, include the confidence level in the output.", "required": false, - "role": "option", + "param_group": "Formatting", "schema": { "type": "boolean" } @@ -81,31 +81,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/clean-whitespaces.json b/wrangles-docs/static/registry/contracts/clean-whitespaces.json index 7851366..02190cf 100644 --- a/wrangles-docs/static/registry/contracts/clean-whitespaces.json +++ b/wrangles-docs/static/registry/contracts/clean-whitespaces.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e36e15c4-f0ad-43f8-8555-ef683a8ab892", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "trim", "description": "Whether to trim leading and trailing spaces. Default True.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -75,7 +75,7 @@ "name": "remove_literals", "description": "Whether to remove special space characters such as new lines etc. Default True.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -85,31 +85,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/compare/lists.json b/wrangles-docs/static/registry/contracts/compare/lists.json index 2a2bace..c2bfcc6 100644 --- a/wrangles-docs/static/registry/contracts/compare/lists.json +++ b/wrangles-docs/static/registry/contracts/compare/lists.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "a393225e-7ccf-4708-83f8-d5abd6ba9b1e", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns containing lists to compare.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "method", "description": "Type of comparison to perform.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "intersection", "schema": { "type": "string", @@ -71,7 +71,7 @@ "name": "remove_duplicates", "description": "Remove duplicates from the result.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -81,7 +81,7 @@ "name": "ignore_case", "description": "Ignore case when comparing string items.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -91,31 +91,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/compare/text.json b/wrangles-docs/static/registry/contracts/compare/text.json index e4649be..bdecab8 100644 --- a/wrangles-docs/static/registry/contracts/compare/text.json +++ b/wrangles-docs/static/registry/contracts/compare/text.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "31905b74-ce58-45cd-8add-821cc04ab946", @@ -38,7 +38,7 @@ "name": "input", "description": "The columns to compare. First column is the base column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "The column to output the results to. Must be a list of two column names [mask_column, ratio_column] when method is overlap and include_ratio is true; otherwise a single column name.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -59,7 +59,7 @@ "name": "method", "description": "The type of comparison to perform (difference, intersection, overlap, similarity).", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "difference", "schema": { "type": "string", @@ -75,7 +75,7 @@ "name": "char", "description": "Character to split strings on for difference and intersection. Defaults to a space.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": " ", "schema": { "type": "string" @@ -85,7 +85,7 @@ "name": "non_match_char", "description": "Character to use for non-matching characters when using overlap.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": "*", "schema": { "type": "string" @@ -95,7 +95,7 @@ "name": "include_ratio", "description": "Include the ratio of matching characters when using overlap.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -105,7 +105,7 @@ "name": "decimal_places", "description": "Number of decimal places to round the overlap ratio to.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": 3, "schema": { "type": "integer" @@ -115,7 +115,7 @@ "name": "exact_match", "description": "Value to use for exact matches when using overlap.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -128,7 +128,7 @@ "name": "empty_a", "description": "Value to use when input A is empty when using overlap.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -141,7 +141,7 @@ "name": "empty_b", "description": "Value to use when input B is empty when using overlap.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -154,7 +154,7 @@ "name": "all_empty", "description": "Value to use when both inputs are empty when using overlap.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -167,7 +167,7 @@ "name": "case_sensitive", "description": "Whether the comparison is case sensitive. Defaults to true.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -177,7 +177,7 @@ "name": "metric", "description": "Metric value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "token_sort", "schema": { "type": "string" @@ -187,31 +187,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/compute/case-when.json b/wrangles-docs/static/registry/contracts/compute/case-when.json index 0aed7e3..809022a 100644 --- a/wrangles-docs/static/registry/contracts/compute/case-when.json +++ b/wrangles-docs/static/registry/contracts/compute/case-when.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "9a9662e4-53d1-4932-8adf-bc3e7aa364ad", @@ -38,7 +38,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "cases", "description": "List of conditions and corresponding values.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "array", "minItems": 1, @@ -77,7 +77,7 @@ "name": "default", "description": "Value to assign if no conditions are met. Default None.", "required": false, - "role": "fallback-value", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -93,31 +93,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/compute/score-search-results.json b/wrangles-docs/static/registry/contracts/compute/score-search-results.json index c82bb3f..935602f 100644 --- a/wrangles-docs/static/registry/contracts/compute/score-search-results.json +++ b/wrangles-docs/static/registry/contracts/compute/score-search-results.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "313a8ec0-cf13-4956-8d3b-5362b8641d0f", @@ -38,7 +38,7 @@ "name": "input", "description": "List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)].", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column].", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -59,7 +59,7 @@ "name": "must_match_part_code", "description": "If true, filters out results that don't satisfy the allowed match types.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -69,7 +69,7 @@ "name": "allow_mpn_exact", "description": "Treat exact MPN matches as valid part code matches.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -79,7 +79,7 @@ "name": "allow_mpn_partial", "description": "Treat partial MPN matches as valid part code matches.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -89,7 +89,7 @@ "name": "allow_other_exact", "description": "Treat exact other part code matches as valid part code matches.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -99,7 +99,7 @@ "name": "allow_other_partial", "description": "Treat partial other part code matches as valid part code matches.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -109,7 +109,7 @@ "name": "blacklist_keywords", "description": "Comma-separated list or array of keywords to filter out URLs containing them.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "", "schema": { "type": "string" @@ -119,7 +119,7 @@ "name": "mpn_exact_score", "description": "Mpn Exact Score value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 8, "schema": { "type": "number" @@ -129,7 +129,7 @@ "name": "mpn_partial_base", "description": "Mpn Partial Base value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 4, "schema": { "type": "number" @@ -139,7 +139,7 @@ "name": "part_code_exact_score", "description": "Part Code Exact Score value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 6, "schema": { "type": "number" @@ -149,7 +149,7 @@ "name": "part_code_partial_base", "description": "Part Code Partial Base value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 2, "schema": { "type": "number" @@ -159,7 +159,7 @@ "name": "supplier_exact_score", "description": "Supplier Exact Score value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 3, "schema": { "type": "number" @@ -169,7 +169,7 @@ "name": "supplier_partial_base", "description": "Supplier Partial Base value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 1, "schema": { "type": "number" @@ -179,7 +179,7 @@ "name": "context_match_base", "description": "Context Match Base value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 2, "schema": { "type": "number" @@ -189,7 +189,7 @@ "name": "fuzzy_match_threshold", "description": "Fuzzy Match Threshold value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 0.8, "schema": { "type": "number" @@ -199,31 +199,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/concurrent.json b/wrangles-docs/static/registry/contracts/concurrent.json index f90371d..41892f6 100644 --- a/wrangles-docs/static/registry/contracts/concurrent.json +++ b/wrangles-docs/static/registry/contracts/concurrent.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "wrangles", "description": "The wrangles section of a recipe to execute for each combination of variables.", "required": true, - "role": "nested-wrangles", + "param_group": "Execution", "schema": { "type": "array", "minItems": 1, @@ -51,7 +51,7 @@ "name": "max_concurrency", "description": "The maximum number of wrangles to execute in parallel.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 10, "schema": { "type": "integer", @@ -62,7 +62,7 @@ "name": "use_multiprocessing", "description": "Use process-based workers instead of threads. This is an advanced runtime option.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": false, "schema": { "type": "boolean" @@ -72,31 +72,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/convert/case.json b/wrangles-docs/static/registry/contracts/convert/case.json index b93d08a..28f1a37 100644 --- a/wrangles-docs/static/registry/contracts/convert/case.json +++ b/wrangles-docs/static/registry/contracts/convert/case.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "12ff4120-3613-4801-8653-99c793477fbc", @@ -39,7 +39,7 @@ "name": "input", "description": "Name, index, or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -58,7 +58,7 @@ "name": "output", "description": "Name or list of output columns. If omitted, each input column is overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "case", "description": "Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "lower", "schema": { "type": "string", @@ -90,31 +90,31 @@ "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } }, { "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [ diff --git a/wrangles-docs/static/registry/contracts/convert/data-type.json b/wrangles-docs/static/registry/contracts/convert/data-type.json index eee0757..4a98982 100644 --- a/wrangles-docs/static/registry/contracts/convert/data-type.json +++ b/wrangles-docs/static/registry/contracts/convert/data-type.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "52384f01-7164-404f-8615-063e7677a588", @@ -39,7 +39,7 @@ "name": "input", "description": "Name, index, or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -58,7 +58,7 @@ "name": "output", "description": "Name or list of output columns. If omitted, each input column is overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "data_type", "description": "Data type to produce.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "str", "schema": { "type": "string", @@ -91,7 +91,7 @@ "name": "default", "description": "Value returned when conversion fails. If omitted, the original value is retained.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -108,31 +108,31 @@ "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } }, { "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [ diff --git a/wrangles-docs/static/registry/contracts/convert/fraction-to-decimal.json b/wrangles-docs/static/registry/contracts/convert/fraction-to-decimal.json index d87cf34..2f36d3a 100644 --- a/wrangles-docs/static/registry/contracts/convert/fraction-to-decimal.json +++ b/wrangles-docs/static/registry/contracts/convert/fraction-to-decimal.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "7c55752e-70ab-4809-8298-c59436127457", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "decimals", "description": "Number of decimals to round fraction.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": 4, "schema": { "type": "integer" @@ -61,7 +61,7 @@ "name": "output", "description": "Name of the output colum.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/convert/from-json.json b/wrangles-docs/static/registry/contracts/convert/from-json.json index 3225df8..6ad730a 100644 --- a/wrangles-docs/static/registry/contracts/convert/from-json.json +++ b/wrangles-docs/static/registry/contracts/convert/from-json.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e370dfcf-b0fe-4c48-8a52-6f34c47e7978", @@ -39,7 +39,7 @@ "name": "input", "description": "Name, index, or list of columns containing valid JSON text.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -58,7 +58,7 @@ "name": "output", "description": "Name or list of output columns. If omitted, each input column is overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "default", "description": "Value returned for empty or invalid JSON. A list may supply one fallback per input column.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -91,31 +91,31 @@ "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } }, { "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [ diff --git a/wrangles-docs/static/registry/contracts/convert/from-yaml.json b/wrangles-docs/static/registry/contracts/convert/from-yaml.json index a9d57c8..f072d66 100644 --- a/wrangles-docs/static/registry/contracts/convert/from-yaml.json +++ b/wrangles-docs/static/registry/contracts/convert/from-yaml.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "b1c20004-5f80-41c2-84d2-2d4601a033b5", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column. If omitted, the input column will be overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "default", "description": "Value to return if the row is empty or fails to be parsed as YAML. If input is a list, default may also be a list - either a single value to apply to all columns, or one value per input column.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -82,31 +82,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/convert/to-json.json b/wrangles-docs/static/registry/contracts/convert/to-json.json index a43a62b..121aa4a 100644 --- a/wrangles-docs/static/registry/contracts/convert/to-json.json +++ b/wrangles-docs/static/registry/contracts/convert/to-json.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "94e54eb7-2b8c-4047-89d0-fb5d16baf396", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column. If omitted, the input column will be overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "ensure_ascii", "description": "If true, non-ASCII characters will be escaped. Default is false.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -75,7 +75,7 @@ "name": "indent", "description": "If indent is a non-negative integer or string, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, negative, or \"\" will only insert newlines. None (the default) selects the most compact representation. Using a positive integer indent indents that many spaces per level. If indent is a string (such as '\\t'), that string is used to indent each level.", "required": false, - "role": "option", + "param_group": "Formatting", "schema": { "type": [ "string", @@ -87,7 +87,7 @@ "name": "sort_keys", "description": "If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key.", "required": false, - "role": "option", + "param_group": "Formatting", "schema": { "type": "boolean" } @@ -96,31 +96,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/convert/to-yaml.json b/wrangles-docs/static/registry/contracts/convert/to-yaml.json index a7210b0..e081ed1 100644 --- a/wrangles-docs/static/registry/contracts/convert/to-yaml.json +++ b/wrangles-docs/static/registry/contracts/convert/to-yaml.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "4cd6252f-ce47-4a9d-8272-3d87e875b72a", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column. If omitted, the input column will be overwritten.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "sort_keys", "description": "If sort_keys is true (default: False), then the output of dictionaries will be sorted by key.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -75,7 +75,7 @@ "name": "allow_unicode", "description": "Allow Unicode value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": true, "schema": { "type": "boolean" @@ -85,7 +85,7 @@ "name": "indent", "description": "Specify the number of spaces for indentation to specify nested elements.", "required": false, - "role": "option", + "param_group": "Formatting", "schema": { "type": "integer" } @@ -94,31 +94,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/copy.json b/wrangles-docs/static/registry/contracts/copy.json index dd78b36..90020a6 100644 --- a/wrangles-docs/static/registry/contracts/copy.json +++ b/wrangles-docs/static/registry/contracts/copy.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "b002fbdc-92c0-4347-889d-0f4bfeec99fa", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input columns or columns.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -53,7 +53,7 @@ "name": "output", "description": "Name of the output columns or columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -67,31 +67,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/bins.json b/wrangles-docs/static/registry/contracts/create/bins.json index 1d7db10..91b449c 100644 --- a/wrangles-docs/static/registry/contracts/create/bins.json +++ b/wrangles-docs/static/registry/contracts/create/bins.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "232e3ba0-4735-4934-88aa-0163181abb3f", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of new column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "bins", "description": "Defines the number of equal-width bins in the range.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": [ "integer", @@ -75,7 +75,7 @@ "name": "labels", "description": "Labels for the returned bins.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -89,31 +89,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/column.json b/wrangles-docs/static/registry/contracts/create/column.json index 5317328..f27ea79 100644 --- a/wrangles-docs/static/registry/contracts/create/column.json +++ b/wrangles-docs/static/registry/contracts/create/column.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "5a18e2c8-ec7c-45f5-88fd-bb5c358a8b40", @@ -38,7 +38,7 @@ "name": "output", "description": "Name or list of names of new columns or column_name: value pairs.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,7 +50,7 @@ "name": "value", "description": "(Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -67,7 +67,7 @@ "name": "value_if_exists", "description": "Determines behaviour when the output column already exists. existing (default): leave the column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "existing", "schema": { "type": "string", @@ -82,7 +82,7 @@ "name": "coalesce_value", "description": "Only used when value_if_exists is coalesce. Determines which side is preferred when both the existing and new values are non-empty. existing (default): keep the existing value, only fill empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "existing", "schema": { "type": "string", @@ -96,31 +96,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/embeddings.json b/wrangles-docs/static/registry/contracts/create/embeddings.json index 56df279..448e485 100644 --- a/wrangles-docs/static/registry/contracts/create/embeddings.json +++ b/wrangles-docs/static/registry/contracts/create/embeddings.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e3518afd-a819-40ec-8b49-eb25690220c1", @@ -38,7 +38,7 @@ "name": "input", "description": "The column of text to create the embeddings for.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "api_key", "description": "The API key.", "required": true, - "role": "credential", + "param_group": "Details", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "output", "description": "The output column the embeddings will be saved as.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -69,7 +69,7 @@ "name": "batch_size", "description": "The number of rows to submit per individual request.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 100, "schema": { "type": "integer" @@ -79,7 +79,7 @@ "name": "threads", "description": "The number of requests to submit in parallel. Each request contains the number of rows set as batch_size.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 10, "schema": { "type": "integer" @@ -89,7 +89,7 @@ "name": "output_type", "description": "Output the embeddings as a numpy array or a python list Default - python list.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": "python list", "schema": { "type": "string", @@ -103,7 +103,7 @@ "name": "model", "description": "The specific model to use to generate the embeddings.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "text-embedding-3-small", "schema": { "type": "string" @@ -113,7 +113,7 @@ "name": "retries", "description": "The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting.", "required": false, - "role": "option", + "param_group": "Errors", "runtime_default": 0, "schema": { "type": "integer" @@ -123,7 +123,7 @@ "name": "url", "description": "The endpoint to send embedding requests to. Defaults to the standard endpoint for the resolved provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "https://api.openai.com/v1/embeddings", "schema": { "type": "string" @@ -133,7 +133,7 @@ "name": "precision", "description": "The precision of the embeddings. Default is float32. This should be used with output_type numpy array.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": "float32", "schema": { "type": "string", @@ -147,7 +147,7 @@ "name": "provider", "description": "Controls the request/response format for the embedding API. When omitted, inferred from url (jina.ai → jina, otherwise openai). Setting provider also sets the default url for that provider, so you only need one of provider or url for standard endpoints. Use both together only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy).", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": null, "schema": { "type": [ @@ -164,7 +164,7 @@ "name": "task", "description": "The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -184,31 +184,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/guid.json b/wrangles-docs/static/registry/contracts/create/guid.json index 6c38ef2..1957a20 100644 --- a/wrangles-docs/static/registry/contracts/create/guid.json +++ b/wrangles-docs/static/registry/contracts/create/guid.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "cf3aaab2-3d5b-4c9b-826e-7af7510521c9", @@ -38,7 +38,7 @@ "name": "output", "description": "Name or list of names of new columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,31 +50,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/hash.json b/wrangles-docs/static/registry/contracts/create/hash.json index 4a16e89..8271450 100644 --- a/wrangles-docs/static/registry/contracts/create/hash.json +++ b/wrangles-docs/static/registry/contracts/create/hash.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "11e8fc13-00d2-4779-8d87-6288b07de7e7", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of new column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "method", "description": "The method to use to hash the input (Default: md5).", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "md5", "schema": { "type": "string", @@ -79,31 +79,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/index.json b/wrangles-docs/static/registry/contracts/create/index.json index 596da93..cd12545 100644 --- a/wrangles-docs/static/registry/contracts/create/index.json +++ b/wrangles-docs/static/registry/contracts/create/index.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "0a4909ca-6e14-4da7-8a70-a7fd106d6944", @@ -38,7 +38,7 @@ "name": "output", "description": "Name or list of names of new columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,7 +50,7 @@ "name": "start", "description": "(Optional; default 1) Starting number for the index.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 1, "schema": { "type": "integer" @@ -60,7 +60,7 @@ "name": "step", "description": "(Optional; default 1) Step between successive rows.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 1, "schema": { "type": "integer" @@ -70,7 +70,7 @@ "name": "by", "description": "Optional. Cluster the created indexes by one or more columns.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -84,31 +84,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/jinja.json b/wrangles-docs/static/registry/contracts/create/jinja.json index c6ee0d6..b0fb39a 100644 --- a/wrangles-docs/static/registry/contracts/create/jinja.json +++ b/wrangles-docs/static/registry/contracts/create/jinja.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "10fc6709-16d4-4eab-8f56-6cb5d170ea66", @@ -38,7 +38,7 @@ "name": "template", "description": "A dictionary which defines the template/location as well as the form which the template is input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_).", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "object", "additionalProperties": false, @@ -59,7 +59,7 @@ "name": "output", "description": "Name of the column to be output to.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "array" } @@ -68,7 +68,7 @@ "name": "input", "description": "Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -81,31 +81,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/create/uuid.json b/wrangles-docs/static/registry/contracts/create/uuid.json index 7e89e6f..66712f9 100644 --- a/wrangles-docs/static/registry/contracts/create/uuid.json +++ b/wrangles-docs/static/registry/contracts/create/uuid.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "c376f3ff-2283-4c4c-8d7f-70db6f53ed19", @@ -38,7 +38,7 @@ "name": "output", "description": "Name or list of names of new columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,31 +50,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/date-calculator.json b/wrangles-docs/static/registry/contracts/date-calculator.json index 74e4668..8d33d4c 100644 --- a/wrangles-docs/static/registry/contracts/date-calculator.json +++ b/wrangles-docs/static/registry/contracts/date-calculator.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "19cfeb4f-02af-4ab2-895c-0ff2bb5cce19", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the dates column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "operation", "description": "Date operation.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "add", "schema": { "type": "string", @@ -61,7 +61,7 @@ "name": "output", "description": "Name of the output column of dates.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "time_unit", "description": "Time unit for operation.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -97,7 +97,7 @@ "name": "time_value", "description": "Time unit value for operation.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -110,31 +110,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/drop.json b/wrangles-docs/static/registry/contracts/drop.json index 150445e..e394392 100644 --- a/wrangles-docs/static/registry/contracts/drop.json +++ b/wrangles-docs/static/registry/contracts/drop.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "363bffbf-397e-4975-8382-e9efa5e9eed6", @@ -38,7 +38,7 @@ "name": "columns", "description": "Name of the column(s) to drop.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,10 +50,10 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/explode.json b/wrangles-docs/static/registry/contracts/explode.json index 6c488f4..97aa08c 100644 --- a/wrangles-docs/static/registry/contracts/explode.json +++ b/wrangles-docs/static/registry/contracts/explode.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "4e4b13ac-8d50-4b2c-85c8-2c31de1e817d", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "reset_index", "description": "Reset the index after exploding. Default True.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": true, "schema": { "type": "boolean" @@ -61,7 +61,7 @@ "name": "drop_empty", "description": "If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -71,31 +71,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/address.json b/wrangles-docs/static/registry/contracts/extract/address.json index 0b720d8..d9a7206 100644 --- a/wrangles-docs/static/registry/contracts/extract/address.json +++ b/wrangles-docs/static/registry/contracts/extract/address.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "44153f95-4581-4cee-898e-b3b4714045fd", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "dataType", "description": "Specific part of the address to extract.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -78,7 +78,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -96,7 +96,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -106,31 +106,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/ai.json b/wrangles-docs/static/registry/contracts/extract/ai.json index 54fb40a..889d5ce 100644 --- a/wrangles-docs/static/registry/contracts/extract/ai.json +++ b/wrangles-docs/static/registry/contracts/extract/ai.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "d9f89b00-fda3-4f4c-826c-6417b9390607", @@ -38,7 +38,7 @@ "name": "api_key", "description": "OpenAI API key used for this wrangle, normally supplied through a recipe variable.", "required": true, - "role": "credential", + "param_group": "Details", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "input", "description": "Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -66,7 +66,7 @@ "name": "output", "description": "Desired extraction. Use an object keyed by output column name for structured fields, a string for one prompted value, or an array of field names/definitions. Each field may use the schema options below.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -175,7 +175,7 @@ "name": "model_id", "description": "ID of a saved extract.ai definition. Use it instead of defining an output schema. When output is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields.", "required": false, - "role": "model-reference", + "param_group": "Details", "runtime_default": null, "schema": { "type": [ @@ -188,7 +188,7 @@ "name": "record_examples", "description": "Whole-record examples. Each example has a separate input value or record and the complete expected output record. Optional name and notes provide model-visible context. Use {name: ..., notes: ..., input: ..., output: ...}. Omitted nullable output fields are completed with null. Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -233,7 +233,7 @@ "name": "output_format", "description": "How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -251,7 +251,7 @@ "name": "char", "description": "Separator used only when output_format is concatenate. Defaults to comma-space.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -261,7 +261,7 @@ "name": "web_search", "description": "Enable OpenAI Responses web search; the model decides when searching helps. When true, every row also receives web_search_sources: a deduplicated list of {title, url} objects in source order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -271,7 +271,7 @@ "name": "instructions", "description": "Additional guidance applied to every input row. Use this for decision rules, evidence priorities, normalization requirements, or other behavior that applies to the complete extraction.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -288,7 +288,7 @@ "name": "model", "description": "OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "string" } @@ -297,7 +297,7 @@ "name": "threads", "description": "Maximum number of row-level requests sent in parallel. The configured default is 32.", "required": false, - "role": "option", + "param_group": "Execution", "schema": { "type": "integer", "minimum": 1 @@ -307,7 +307,7 @@ "name": "timeout", "description": "Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner.", "required": false, - "role": "option", + "param_group": "Execution", "schema": { "type": "number", "exclusiveMinimum": 0 @@ -317,7 +317,7 @@ "name": "retries", "description": "Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline.", "required": false, - "role": "option", + "param_group": "Errors", "schema": { "type": "integer", "minimum": 0 @@ -327,7 +327,7 @@ "name": "url", "description": "Override the endpoint for the selected protocol. A chat/completions URL selects the legacy protocol only when protocol is omitted; new recipes should use the configured Responses endpoint.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "string" } @@ -336,7 +336,7 @@ "name": "provider", "description": "AI service provider. Currently only OpenAI is supported.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "string", "enum": [ @@ -348,7 +348,7 @@ "name": "protocol", "description": "OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "string", "enum": [ @@ -361,7 +361,7 @@ "name": "deadline", "description": "Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15.", "required": false, - "role": "option", + "param_group": "Execution", "schema": { "type": "number", "exclusiveMinimum": 0 @@ -371,7 +371,7 @@ "name": "store", "description": "Whether OpenAI may store Responses API results. Defaults to false.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "boolean" } @@ -380,7 +380,7 @@ "name": "cache", "description": "Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "boolean" } @@ -389,7 +389,7 @@ "name": "cache_ttl", "description": "Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "number", "exclusiveMinimum": 0 @@ -399,7 +399,7 @@ "name": "strict", "description": "Require OpenAI structured-output strict mode. Defaults to true. Definitions with dynamic dictionary keys automatically switch to non-strict provider mode and are still validated locally.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "boolean" } @@ -408,7 +408,7 @@ "name": "reasoning", "description": "Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "object", "properties": { @@ -430,7 +430,7 @@ "name": "verbosity", "description": "Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models.", "required": false, - "role": "option", + "param_group": "Details", "schema": { "type": "string", "enum": [ @@ -444,31 +444,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/attributes.json b/wrangles-docs/static/registry/contracts/extract/attributes.json index 3692d90..42d1974 100644 --- a/wrangles-docs/static/registry/contracts/extract/attributes.json +++ b/wrangles-docs/static/registry/contracts/extract/attributes.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "03ccedef-c938-41f1-8980-280f1a91542e", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "responseContent", "description": "Span - returns the text found. object - returns an object with the value and unit.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "span", "schema": { "type": "string", @@ -77,7 +77,7 @@ "name": "attribute_type", "description": "Request only a specific type of attribute.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -117,7 +117,7 @@ "name": "desired_unit", "description": "Convert the extracted unit to the desired unit.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -130,7 +130,7 @@ "name": "bound", "description": "When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "mid", "schema": { "type": "string", @@ -145,7 +145,7 @@ "name": "first_element", "description": "Get the first element from results.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -155,7 +155,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -174,7 +174,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -184,31 +184,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/brackets.json b/wrangles-docs/static/registry/contracts/extract/brackets.json index 3e4d85d..b6d1ba5 100644 --- a/wrangles-docs/static/registry/contracts/extract/brackets.json +++ b/wrangles-docs/static/registry/contracts/extract/brackets.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "9b4c15fa-2aaa-40c8-8834-6e835760bee5", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "find", "description": "(Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "all", "schema": { "type": [ @@ -76,7 +76,7 @@ "name": "include_brackets", "description": "(Optional) Include the brackets in the output.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -86,7 +86,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -104,7 +104,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -114,31 +114,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/codes.json b/wrangles-docs/static/registry/contracts/extract/codes.json index 042c994..13b80b5 100644 --- a/wrangles-docs/static/registry/contracts/extract/codes.json +++ b/wrangles-docs/static/registry/contracts/extract/codes.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "da591387-0fca-4842-8bcb-d19f561f0292", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name or list of output columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "first_element", "description": "Get the first element from results.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -73,7 +73,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -91,7 +91,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -101,7 +101,7 @@ "name": "min_length", "description": "Minimum length of allowed results.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "integer", @@ -113,7 +113,7 @@ "name": "max_length", "description": "Maximum length of allowed results.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "integer", @@ -125,7 +125,7 @@ "name": "strategy", "description": "Controls filtering of likely false positives such as measurements. Lenient skips this filter; balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -139,7 +139,7 @@ "name": "sort_order", "description": "Default is input order. Also allows longest or shortest.", "required": false, - "role": "option", + "param_group": "Formatting", "schema": { "type": "string", "enum": [ @@ -153,7 +153,7 @@ "name": "disallowed_patterns", "description": "A pattern or JSON array of regex patterns to not include in the found codes.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -162,7 +162,7 @@ "name": "include_multi_part_tokens", "description": "Whether to include multi-part tokens that have a space. Default True.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "boolean" } @@ -171,7 +171,7 @@ "name": "extract_raw", "description": "Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "boolean" } @@ -180,31 +180,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/custom.json b/wrangles-docs/static/registry/contracts/extract/custom.json index 3ff4c63..e40a012 100644 --- a/wrangles-docs/static/registry/contracts/extract/custom.json +++ b/wrangles-docs/static/registry/contracts/extract/custom.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e8e96b76-86bf-41dc-8d16-825dcff9688b", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "model_id", "description": "The ID of the wrangle to use.", "required": true, - "role": "model-reference", + "param_group": "Details", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -77,7 +77,7 @@ "name": "use_labels", "description": "Use Labels in the extract output {label: value}.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -87,7 +87,7 @@ "name": "first_element", "description": "Get the first element from results.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -97,7 +97,7 @@ "name": "case_sensitive", "description": "Allows the wrangle to be case sensitive if set to True, default is False.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -107,7 +107,7 @@ "name": "extract_raw", "description": "Extract the raw data from the wrangle.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -117,7 +117,7 @@ "name": "use_spellcheck", "description": "Use spellcheck to also find minor mispellings compared to the reference data.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -127,7 +127,7 @@ "name": "include_empty_labels", "description": "Include labels with no found values in the output when using use_labels=True.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": true, "schema": { "type": "boolean" @@ -137,7 +137,7 @@ "name": "sort", "description": "Sort the results.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "training_order", "schema": { "type": "string", @@ -157,7 +157,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -176,7 +176,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -186,31 +186,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/date-properties.json b/wrangles-docs/static/registry/contracts/extract/date-properties.json index b095c5f..ae90505 100644 --- a/wrangles-docs/static/registry/contracts/extract/date-properties.json +++ b/wrangles-docs/static/registry/contracts/extract/date-properties.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "a346de62-93cd-44ba-8d30-a6305629c6d7", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "property", "description": "Property to extract from date.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -70,7 +70,7 @@ "name": "output", "description": "Name of the output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -83,31 +83,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/date-range.json b/wrangles-docs/static/registry/contracts/extract/date-range.json index 02d5b5c..5fa1fec 100644 --- a/wrangles-docs/static/registry/contracts/extract/date-range.json +++ b/wrangles-docs/static/registry/contracts/extract/date-range.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "9cebfa6f-a524-4aec-84a7-02d77b792843", @@ -38,7 +38,7 @@ "name": "start_time", "description": "Name of the start date column.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "end_time", "description": "Name of the end date column.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -65,7 +65,7 @@ "name": "range", "description": "Type of frequency to count.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "day", "schema": { "type": "string", @@ -94,31 +94,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/html.json b/wrangles-docs/static/registry/contracts/extract/html.json index 4086ce3..2265eec 100644 --- a/wrangles-docs/static/registry/contracts/extract/html.json +++ b/wrangles-docs/static/registry/contracts/extract/html.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "728fc87a-a20d-4efa-833a-612e0b5eadc3", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "data_type", "description": "The type of data to extract.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -64,7 +64,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -78,7 +78,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -96,7 +96,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -106,31 +106,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/properties.json b/wrangles-docs/static/registry/contracts/extract/properties.json index 807928d..99fd424 100644 --- a/wrangles-docs/static/registry/contracts/extract/properties.json +++ b/wrangles-docs/static/registry/contracts/extract/properties.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "23bbdb86-fd13-4b78-8fff-a7a76769ab63", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "property_type", "description": "The specific type of properties to extract.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -82,7 +82,7 @@ "name": "return_data_type", "description": "Legacy format option. Prefer output_format.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": "list", "schema": { "type": "string", @@ -96,7 +96,7 @@ "name": "first_element", "description": "Get the first element from results.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -106,7 +106,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -125,7 +125,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -135,31 +135,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/extract/regex.json b/wrangles-docs/static/registry/contracts/extract/regex.json index 42c01e2..aad877a 100644 --- a/wrangles-docs/static/registry/contracts/extract/regex.json +++ b/wrangles-docs/static/registry/contracts/extract/regex.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "9aa0253a-4b70-4737-832c-964e15967289", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column(s).", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "find", "description": "Pattern to find using regex.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output column(s).", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -72,7 +72,7 @@ "name": "output_pattern", "description": "Specifies the format to output matches and specific capture groups using backreferences (e.g., `\\1`, `\\2`). Default is to return entire matches. **Example**: For a regex pattern `r'(\\d+)\\s(\\w+)'` and `output_pattern = '\\2 \\1'`, with input `'120 volt'`, the output would be `'volt 120'`.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -85,7 +85,7 @@ "name": "first_element", "description": "Get the first element from results.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -95,7 +95,7 @@ "name": "output_format", "description": "Format of the extract output.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -113,7 +113,7 @@ "name": "char", "description": "Character to use when output_format is concatenate.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ", ", "schema": { "type": "string" @@ -123,31 +123,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/filter.json b/wrangles-docs/static/registry/contracts/filter.json index a622d44..98f277f 100644 --- a/wrangles-docs/static/registry/contracts/filter.json +++ b/wrangles-docs/static/registry/contracts/filter.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e3242acf-d204-433f-8373-205b77481131", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column to filter on. If multiple are provided, all must match the criteria.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": [], "schema": { "type": [ @@ -52,7 +52,7 @@ "name": "equal", "description": "Select rows where the values equal a given value.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -66,7 +66,7 @@ "name": "not_equal", "description": "Select rows where the values do not equal a given value.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -80,7 +80,7 @@ "name": "is_in", "description": "Select rows where the values are in a given list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -94,7 +94,7 @@ "name": "not_in", "description": "Select rows where the values are not in a given list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -108,7 +108,7 @@ "name": "greater_than", "description": "Select rows where the values are greater than a specified value. Does include the value itself.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -122,7 +122,7 @@ "name": "greater_than_equal_to", "description": "Select rows where the values are greater than a specified value. Does include the value itself.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -136,7 +136,7 @@ "name": "less_than", "description": "Select rows where the values are less than a specified value. Does not include the value itself.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -150,7 +150,7 @@ "name": "less_than_equal_to", "description": "Select rows where the values are less than a specified value. Does include the value itself.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -164,7 +164,7 @@ "name": "between", "description": "Value or list of values to filter that are in between two parameter values.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -177,7 +177,7 @@ "name": "contains", "description": "Select rows where the input contains the value. Allows regular expressions.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -190,7 +190,7 @@ "name": "not_contains", "description": "Select rows where the input does not contain the value. Allows regular expressions.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -203,7 +203,7 @@ "name": "is_null", "description": "If true, select all rows where the value is NULL. If false, where is not NULL.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -216,31 +216,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/dates.json b/wrangles-docs/static/registry/contracts/format/dates.json index 8df5030..da4f87f 100644 --- a/wrangles-docs/static/registry/contracts/format/dates.json +++ b/wrangles-docs/static/registry/contracts/format/dates.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "a3c15135-4f7a-4659-83fd-f657afa603c9", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "format", "description": "String pattern to format date.", "required": true, - "role": "option", + "param_group": "Formatting", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,31 +74,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/pad.json b/wrangles-docs/static/registry/contracts/format/pad.json index 7d3f1a0..a3158a2 100644 --- a/wrangles-docs/static/registry/contracts/format/pad.json +++ b/wrangles-docs/static/registry/contracts/format/pad.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "76c19378-38f4-45aa-85d1-3cdf8f8aae29", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "pad_length", "description": "Length for the output.", "required": true, - "role": "option", + "param_group": "Formatting", "schema": { "type": "integer" } @@ -60,7 +60,7 @@ "name": "side", "description": "Side from which to fill resulting string.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -69,7 +69,7 @@ "name": "char", "description": "The character to pad the input with.", "required": true, - "role": "option", + "param_group": "Formatting", "schema": { "type": "string" } @@ -78,7 +78,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -92,7 +92,7 @@ "name": "skip_empty", "description": "If true, skip padding for empty or whitespace-only values.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -102,31 +102,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/prefix.json b/wrangles-docs/static/registry/contracts/format/prefix.json index f156066..5528a93 100644 --- a/wrangles-docs/static/registry/contracts/format/prefix.json +++ b/wrangles-docs/static/registry/contracts/format/prefix.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "c12f99b9-2363-4da7-8405-7c73b87906e5", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "value", "description": "Prefix value to add.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -64,7 +64,7 @@ "name": "output", "description": "(Optional) Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -78,7 +78,7 @@ "name": "skip_empty", "description": "Whether to skip empty values.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -88,31 +88,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/price-breaks.json b/wrangles-docs/static/registry/contracts/format/price-breaks.json index fa41475..8142081 100644 --- a/wrangles-docs/static/registry/contracts/format/price-breaks.json +++ b/wrangles-docs/static/registry/contracts/format/price-breaks.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "input", "description": "Name, index, or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "categoryLabel", "description": "Prefix for output columns that identify the source price-break category.", "required": true, - "role": "option", + "param_group": "Formatting", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "valueLabel", "description": "Prefix for output columns that contain the corresponding price-break value.", "required": true, - "role": "option", + "param_group": "Formatting", "schema": { "type": "string" } @@ -65,35 +65,35 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], - "guidance": "## Behavior\n\nExpand non-empty price-break cells into paired category and value columns.\n\nThis first-pass guidance is derived from the callable signature.", + "guidance": "## Behavior\n\nExpand non-empty price-break cells into paired category and value columns.\n\nThis guidance was derived from the callable signature.", "sources": [ { "id": "runtime", diff --git a/wrangles-docs/static/registry/contracts/format/remove-duplicates.json b/wrangles-docs/static/registry/contracts/format/remove-duplicates.json index 4c0f4cb..19f7cfe 100644 --- a/wrangles-docs/static/registry/contracts/format/remove-duplicates.json +++ b/wrangles-docs/static/registry/contracts/format/remove-duplicates.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "283b9e78-b2b2-43d0-844f-9842c33120aa", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "ignore_case", "description": "Ignore case when removing duplicates.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/significant-figures.json b/wrangles-docs/static/registry/contracts/format/significant-figures.json index 6ad3522..d5a86bd 100644 --- a/wrangles-docs/static/registry/contracts/format/significant-figures.json +++ b/wrangles-docs/static/registry/contracts/format/significant-figures.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "a359f72a-5250-4dd8-84f6-8a8173bee0f6", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "significant_figures", "description": "Number of significant figures to format to. Default is 3.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": 3, "schema": { "type": "integer" @@ -61,7 +61,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/suffix.json b/wrangles-docs/static/registry/contracts/format/suffix.json index 6d2c67d..d75c82b 100644 --- a/wrangles-docs/static/registry/contracts/format/suffix.json +++ b/wrangles-docs/static/registry/contracts/format/suffix.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "8d127060-ba2d-4934-897f-07662e01e40b", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "value", "description": "Suffix value to add.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -65,7 +65,7 @@ "name": "output", "description": "(Optional) Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -78,7 +78,7 @@ "name": "skip_empty", "description": "Whether to skip empty values.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -88,31 +88,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/format/trim.json b/wrangles-docs/static/registry/contracts/format/trim.json index 306db7d..77c1638 100644 --- a/wrangles-docs/static/registry/contracts/format/trim.json +++ b/wrangles-docs/static/registry/contracts/format/trim.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "af16b3c1-c230-4868-8ebe-f574904a0c76", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,31 +65,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/generate/ai.json b/wrangles-docs/static/registry/contracts/generate/ai.json index 2335a76..9665a8a 100644 --- a/wrangles-docs/static/registry/contracts/generate/ai.json +++ b/wrangles-docs/static/registry/contracts/generate/ai.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "12c31c6c-cade-484d-84ba-7f302bf6af52", @@ -38,7 +38,7 @@ "name": "api_key", "description": "OpenAI-compatible API key.", "required": true, - "role": "credential", + "param_group": "Details", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "output", "description": "Target schema; string/array shorthands are expanded automatically.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -60,7 +60,7 @@ "name": "input", "description": "Column(s) to concatenate into the prompt (defaults to all columns).", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "model", "description": "Responses model name (e.g. gpt-5-mini).", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "gpt-5", "schema": { "type": "string" @@ -84,7 +84,7 @@ "name": "threads", "description": "Maximum concurrent requests (default 20).", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 20, "schema": { "type": "integer" @@ -94,7 +94,7 @@ "name": "timeout", "description": "Per-request timeout in seconds.", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 90, "schema": { "type": "integer" @@ -104,7 +104,7 @@ "name": "retries", "description": "Number of retry attempts on failure.", "required": false, - "role": "option", + "param_group": "Errors", "runtime_default": 0, "schema": { "type": "integer" @@ -114,7 +114,7 @@ "name": "messages", "description": "Optional extra messages forwarded to the inner generate helper.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -128,7 +128,7 @@ "name": "url", "description": "Override for the OpenAI-compatible endpoint.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "https://api.openai.com/v1/responses", "schema": { "type": "string" @@ -138,7 +138,7 @@ "name": "strict", "description": "Enforce JSON-schema validation on the response.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": false, "schema": { "type": "boolean" @@ -148,7 +148,7 @@ "name": "web_search", "description": "Enable DuckDuckGo context lookup per row.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -158,7 +158,7 @@ "name": "reasoning", "description": "Responses API reasoning options (forwarded verbatim).", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": { "effort": "low" }, @@ -173,7 +173,7 @@ "name": "previous_response", "description": "Chain responses by reusing previous_response_id for field-by-field calls.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": false, "schema": { "type": "boolean" @@ -183,7 +183,7 @@ "name": "summary", "description": "Request summary text to be merged into the output.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -193,31 +193,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/huggingface.json b/wrangles-docs/static/registry/contracts/huggingface.json index 68170e0..b37b70c 100644 --- a/wrangles-docs/static/registry/contracts/huggingface.json +++ b/wrangles-docs/static/registry/contracts/huggingface.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "ae12cf20-4934-428f-84a9-a6898cb7ffe0", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "api_token", "description": "Huggingface API Token.", "required": true, - "role": "option", + "param_group": "Details", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "model", "description": "Name of the model to use. e.g. facebook/bart-large-cnn.", "required": true, - "role": "option", + "param_group": "Details", "schema": { "type": "string" } @@ -69,7 +69,7 @@ "name": "output", "description": "Name of the output column. If not provided, will overwrite the input column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -83,7 +83,7 @@ "name": "parameters", "description": "Optionally, provide additional parameters to define the model behaviour.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -96,31 +96,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/log.json b/wrangles-docs/static/registry/contracts/log.json index 3fd2af6..fc6f6fd 100644 --- a/wrangles-docs/static/registry/contracts/log.json +++ b/wrangles-docs/static/registry/contracts/log.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "6177808e-aa2d-4d0b-8385-858b16948a5d", @@ -38,7 +38,7 @@ "name": "columns", "description": "(Optional, default all columns) List of specific columns to log.", "required": false, - "role": "option", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -51,7 +51,7 @@ "name": "write", "description": "(Optional) Allows for an intermediate output to a file/dataframe/database etc.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -68,7 +68,7 @@ "name": "warning", "description": "Log a warning to the console.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -81,7 +81,7 @@ "name": "info", "description": "Log info to the console.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -94,7 +94,7 @@ "name": "log_data", "description": "Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -107,7 +107,7 @@ "name": "error", "description": "Log an error to the console.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -116,31 +116,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/lookup.json b/wrangles-docs/static/registry/contracts/lookup.json index 109063d..afa7a7e 100644 --- a/wrangles-docs/static/registry/contracts/lookup.json +++ b/wrangles-docs/static/registry/contracts/lookup.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "b3339193-d1cc-4c89-8ed6-901efa6d81be", @@ -37,7 +37,7 @@ "name": "input", "description": "Name of the column(s) to lookup.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -46,7 +46,7 @@ "name": "output", "description": "Name of the output column(s). When n is provided and the output list length equals n, each output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. \"Top *\" with n: 3 becomes \"Top 1\", \"Top 2\", \"Top 3\".", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -60,7 +60,7 @@ "name": "model_id", "description": "The model_id to use lookup against.", "required": false, - "role": "model-reference", + "param_group": "Details", "runtime_default": null, "schema": { "type": [ @@ -73,7 +73,7 @@ "name": "lookup_mode", "description": "How to perform lookups. 'by_row' (default): lookup each row individually. 'by_dataframe': lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "by_row", "schema": { "type": "string", @@ -88,7 +88,7 @@ "name": "n", "description": "Number of matches to return per input value. When the output list length equals n, each output column receives the corresponding match. Otherwise all n matches are stored as a list in each output column.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -101,31 +101,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/math.json b/wrangles-docs/static/registry/contracts/math.json index 28fed35..6060694 100644 --- a/wrangles-docs/static/registry/contracts/math.json +++ b/wrangles-docs/static/registry/contracts/math.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "494b11ad-00c1-4748-8b93-6bec982f4fec", @@ -38,7 +38,7 @@ "name": "input", "description": "The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_).", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "output", "description": "The column to output the results to.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,31 +56,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/maths.json b/wrangles-docs/static/registry/contracts/maths.json index e76049d..5a94b46 100644 --- a/wrangles-docs/static/registry/contracts/maths.json +++ b/wrangles-docs/static/registry/contracts/maths.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -12,6 +12,7 @@ "title": "Maths", "description": "Deprecated alias for `math`; evaluate an expression and write its result to an output column.", "status": "deprecated", + "replaced_by": "math", "visibility": "public", "tags": [ "maths" @@ -37,7 +38,7 @@ "name": "input", "description": "Name, index, or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -46,7 +47,7 @@ "name": "output", "description": "Name or list of output columns.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -55,35 +56,35 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], - "guidance": "## Behavior\n\nDeprecated alias for `math`; evaluate an expression and write its result to an output column.\n\nThis first-pass guidance is derived from the callable signature.", + "guidance": "## Behavior\n\nDeprecated alias for `math`; evaluate an expression and write its result to an output column.\n\nThis guidance was derived from the callable signature.", "sources": [ { "id": "runtime", diff --git a/wrangles-docs/static/registry/contracts/matrix.json b/wrangles-docs/static/registry/contracts/matrix.json index 8c3eb7d..cb3d710 100644 --- a/wrangles-docs/static/registry/contracts/matrix.json +++ b/wrangles-docs/static/registry/contracts/matrix.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "variables", "description": "A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values.", "required": true, - "role": "variables", + "param_group": "Execution", "schema": { "type": "object" } @@ -47,7 +47,7 @@ "name": "wrangles", "description": "The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables.", "required": true, - "role": "nested-wrangles", + "param_group": "Execution", "schema": { "type": "array", "minItems": 1, @@ -60,7 +60,7 @@ "name": "strategy", "description": "Determines how to combine variables when there are multiple. loop (default) iterates over each set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "loop", "schema": { "type": "string", @@ -74,31 +74,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/coalesce.json b/wrangles-docs/static/registry/contracts/merge/coalesce.json index a0d56db..188b560 100644 --- a/wrangles-docs/static/registry/contracts/merge/coalesce.json +++ b/wrangles-docs/static/registry/contracts/merge/coalesce.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "25c1a60d-fa48-4b9a-8c03-0921d5b31049", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns or a single column containing lists.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output columns. This is required if multiple input columns are provided.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -60,31 +60,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/concatenate.json b/wrangles-docs/static/registry/contracts/merge/concatenate.json index 89ea18a..d32755b 100644 --- a/wrangles-docs/static/registry/contracts/merge/concatenate.json +++ b/wrangles-docs/static/registry/contracts/merge/concatenate.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "6ff76728-e1f6-4d3d-8946-6aa3b7524b3f", @@ -38,7 +38,7 @@ "name": "input", "description": "Either a single column name or list of columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "char", "description": "(Optional) Character to add between successive values.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": ",", "schema": { "type": "string" @@ -70,7 +70,7 @@ "name": "skip_empty", "description": "Whether to skip empty values, defaults to false.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean", @@ -81,31 +81,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/dictionaries.json b/wrangles-docs/static/registry/contracts/merge/dictionaries.json index d58b4fa..3e7a542 100644 --- a/wrangles-docs/static/registry/contracts/merge/dictionaries.json +++ b/wrangles-docs/static/registry/contracts/merge/dictionaries.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "93e27737-e966-4ba9-8777-4e96724ebfc4", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "skip_empty", "description": "Whether to skip empty dictionaries when merging.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -66,31 +66,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/key-value-pairs.json b/wrangles-docs/static/registry/contracts/merge/key-value-pairs.json index 6be0b41..f3b3c20 100644 --- a/wrangles-docs/static/registry/contracts/merge/key-value-pairs.json +++ b/wrangles-docs/static/registry/contracts/merge/key-value-pairs.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "8a2cd37c-8ef7-4b05-8264-36512f5dd837", @@ -38,7 +38,7 @@ "name": "input", "description": "Matched pairs of key and value columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "object" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "skip_empty", "description": "Whether to skip empty keys or values when creating the dictionary.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -66,31 +66,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/lists.json b/wrangles-docs/static/registry/contracts/merge/lists.json index a9a1c01..6ee7b84 100644 --- a/wrangles-docs/static/registry/contracts/merge/lists.json +++ b/wrangles-docs/static/registry/contracts/merge/lists.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "d9978f00-b3d4-4583-884e-a53b98a43e9a", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "remove_duplicates", "description": "Whether to remove duplicates from the created list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -66,7 +66,7 @@ "name": "ignore_case", "description": "Ignore case when removing duplicates.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -76,7 +76,7 @@ "name": "include_empty", "description": "Whether to include empty values in the created list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -86,31 +86,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/to-dict.json b/wrangles-docs/static/registry/contracts/merge/to-dict.json index fee8ae5..6e22bb4 100644 --- a/wrangles-docs/static/registry/contracts/merge/to-dict.json +++ b/wrangles-docs/static/registry/contracts/merge/to-dict.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "9b869210-0d89-403b-8409-7cecdb5f9c7c", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "include_empty", "description": "Whether to include empty columns in the created dictionary.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -66,31 +66,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/merge/to-list.json b/wrangles-docs/static/registry/contracts/merge/to-list.json index dd14ce4..be2f668 100644 --- a/wrangles-docs/static/registry/contracts/merge/to-list.json +++ b/wrangles-docs/static/registry/contracts/merge/to-list.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "d5300fe7-c8a4-4a41-8f12-f2c1698678cc", @@ -38,7 +38,7 @@ "name": "input", "description": "List of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "include_empty", "description": "Whether to include empty columns in the created list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -66,31 +66,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/python.json b/wrangles-docs/static/registry/contracts/python.json index 61a33af..9997177 100644 --- a/wrangles-docs/static/registry/contracts/python.json +++ b/wrangles-docs/static/registry/contracts/python.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "c0398a11-7731-4e47-8df0-b07eea0b1d6c", @@ -38,7 +38,7 @@ "name": "command", "description": "Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_).", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name or list of output column(s). To output multiple columns, return a list of the corresponding length.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -59,7 +59,7 @@ "name": "input", "description": "Name or list of input column(s) to filter the data available to the command. Useful in conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "except", "description": "Value to return for the row if an exception occurs during the evaluation. If not provided, an exception will be raised as normal. If multiple output columns are specified, this must match the length.", "required": false, - "role": "option", + "param_group": "Errors", "schema": { "type": [ "string", @@ -90,31 +90,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/recipe.json b/wrangles-docs/static/registry/contracts/recipe.json index c6a80ad..412c9cb 100644 --- a/wrangles-docs/static/registry/contracts/recipe.json +++ b/wrangles-docs/static/registry/contracts/recipe.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -37,7 +37,7 @@ "name": "input", "description": "Name, index, or list of input columns.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -52,7 +52,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -66,7 +66,7 @@ "name": "name", "description": "File name of the recipe.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -79,7 +79,7 @@ "name": "variables", "description": "A dictionary of variables to pass to the recipe.", "required": false, - "role": "variables", + "param_group": "Execution", "runtime_default": null, "schema": { "type": [ @@ -92,35 +92,35 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], - "guidance": "## Behavior\n\nRun another recipe as a wrangle against the current dataframe.\n\nThis first-pass guidance is derived from the callable signature and its embedded Python schema docstring.", + "guidance": "## Behavior\n\nRun another recipe as a wrangle against the current dataframe.\n\nThis guidance was derived from the callable signature and its embedded Python schema docstring.", "sources": [ { "id": "runtime", diff --git a/wrangles-docs/static/registry/contracts/reindex.json b/wrangles-docs/static/registry/contracts/reindex.json index a72ae61..6063277 100644 --- a/wrangles-docs/static/registry/contracts/reindex.json +++ b/wrangles-docs/static/registry/contracts/reindex.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "def87df8-72da-4e34-83c1-1fde25126257", @@ -38,7 +38,7 @@ "name": "labels", "description": "New labels / index to conform the axis specified by ‘axis’ to.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -51,7 +51,7 @@ "name": "index", "description": "New labels for the index. Preferably an Index object to avoid duplicating data.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -64,7 +64,7 @@ "name": "columns", "description": "New labels for the columns. Preferably an Index object to avoid duplicating data.", "required": false, - "role": "option", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -77,7 +77,7 @@ "name": "axis", "description": "Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1).", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -91,10 +91,10 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/remove-words.json b/wrangles-docs/static/registry/contracts/remove-words.json index 410135c..fc38e0a 100644 --- a/wrangles-docs/static/registry/contracts/remove-words.json +++ b/wrangles-docs/static/registry/contracts/remove-words.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "543b96c3-f354-48be-8046-bf0cb9fbaf56", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of column to remove words from.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "to_remove", "description": "Column or list of columns with a list of words to be removed.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "tokenize_to_remove", "description": "Tokenize all to_remove inputs.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -84,7 +84,7 @@ "name": "ignore_case", "description": "Ignore input and to_remove case.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -94,31 +94,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/rename.json b/wrangles-docs/static/registry/contracts/rename.json index 62280ef..f2710c5 100644 --- a/wrangles-docs/static/registry/contracts/rename.json +++ b/wrangles-docs/static/registry/contracts/rename.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "ab06898f-faf7-42e7-8275-5e3034a4d727", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -53,7 +53,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -67,7 +67,7 @@ "name": "wrangles", "description": "Use wrangles to transform the column names. The input is named 'columns' and the final result must also include the column named 'columns'. This can only be used instead of the standard rename.", "required": false, - "role": "nested-wrangles", + "param_group": "Execution", "runtime_default": null, "schema": { "type": [ @@ -84,10 +84,10 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/replace.json b/wrangles-docs/static/registry/contracts/replace.json index 72300e0..fed4e6c 100644 --- a/wrangles-docs/static/registry/contracts/replace.json +++ b/wrangles-docs/static/registry/contracts/replace.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "f0ab715e-9e0e-4614-83e4-5cd8ea08a09f", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "find", "description": "Pattern to find using regex.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "replace", "description": "Value to replace the pattern found.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -69,7 +69,7 @@ "name": "output", "description": "Name or list of output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -83,31 +83,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/round.json b/wrangles-docs/static/registry/contracts/round.json index f1e986c..6799f83 100644 --- a/wrangles-docs/static/registry/contracts/round.json +++ b/wrangles-docs/static/registry/contracts/round.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "12f3111b-8511-4e42-8d3f-b5302dc3b4e4", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column(s).", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "decimals", "description": "Number of decimal places to round column.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": 0, "schema": { "type": "integer" @@ -61,7 +61,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/search/find-links.json b/wrangles-docs/static/registry/contracts/search/find-links.json index 16002f4..dec7fcb 100644 --- a/wrangles-docs/static/registry/contracts/search/find-links.json +++ b/wrangles-docs/static/registry/contracts/search/find-links.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "ec1f661b-2cbc-4a47-88e3-646179104376", @@ -38,7 +38,7 @@ "name": "queries", "description": "Name or list of input columns containing search queries.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,7 +50,7 @@ "name": "id", "description": "Name of the column containing the row ID to append to each search result.", "required": true, - "role": "option", + "param_group": "I/O", "schema": { "type": "string" } @@ -59,7 +59,7 @@ "name": "output", "description": "Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column].", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -73,7 +73,7 @@ "name": "client", "description": "The search provider to use.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "serpapi", "schema": { "type": "string", @@ -86,7 +86,7 @@ "name": "api_key", "description": "API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY).", "required": false, - "role": "credential", + "param_group": "Details", "runtime_default": null, "schema": { "type": [ @@ -99,7 +99,7 @@ "name": "n_results", "description": "Number of search results to return per query (default 10, max 100).", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 10, "schema": { "type": "integer" @@ -109,7 +109,7 @@ "name": "threads", "description": "Number of concurrent threads for parallel processing (default 10).", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 10, "schema": { "type": "integer" @@ -119,7 +119,7 @@ "name": "country", "description": "Country code for search results (default 'us'). Alias: gl.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -128,7 +128,7 @@ "name": "language", "description": "Language code for search results (default 'en'). Alias: hl.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -137,7 +137,7 @@ "name": "location", "description": "Location for search results (e.g., 'Austin, Texas').", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -146,7 +146,7 @@ "name": "device", "description": "Device type for search results.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -160,31 +160,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/search/retrieve-link-content.json b/wrangles-docs/static/registry/contracts/search/retrieve-link-content.json index d9806f0..5ca53ad 100644 --- a/wrangles-docs/static/registry/contracts/search/retrieve-link-content.json +++ b/wrangles-docs/static/registry/contracts/search/retrieve-link-content.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "3d5faa87-9e96-48c0-8226-5ae566e8d76c", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns containing URLs or Scored Search Result dictionaries.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,7 +50,7 @@ "name": "output", "description": "Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -64,7 +64,7 @@ "name": "client", "description": "The retrieval provider to use.", "required": false, - "role": "option", + "param_group": "Details", "runtime_default": "google_url_context", "schema": { "type": "string", @@ -77,7 +77,7 @@ "name": "api_key", "description": "API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY).", "required": false, - "role": "credential", + "param_group": "Details", "runtime_default": null, "schema": { "type": [ @@ -90,7 +90,7 @@ "name": "prompt", "description": "Optional custom system prompt to guide the extraction behavior and output format.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -103,7 +103,7 @@ "name": "model_id", "description": "The specific model ID to use (default models/gemini-3-flash-preview).", "required": false, - "role": "model-reference", + "param_group": "Details", "runtime_default": "models/gemini-3-flash-preview", "schema": { "type": "string" @@ -113,7 +113,7 @@ "name": "output_format", "description": "The desired format for the extracted content.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": "json", "schema": { "type": "string", @@ -127,7 +127,7 @@ "name": "threads", "description": "Number of concurrent threads for parallel processing (default 10).", "required": false, - "role": "option", + "param_group": "Execution", "runtime_default": 10, "schema": { "type": "integer" @@ -137,31 +137,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/columns.json b/wrangles-docs/static/registry/contracts/select/columns.json index f756f6b..dd592c2 100644 --- a/wrangles-docs/static/registry/contracts/select/columns.json +++ b/wrangles-docs/static/registry/contracts/select/columns.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "bca12c60-3957-4dc5-83f7-5ee460df2a11", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column(s) to select.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,31 +51,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/dictionary-element.json b/wrangles-docs/static/registry/contracts/select/dictionary-element.json index f707de4..27098c2 100644 --- a/wrangles-docs/static/registry/contracts/select/dictionary-element.json +++ b/wrangles-docs/static/registry/contracts/select/dictionary-element.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "5adaeada-6da8-464f-84cb-9fb5ecc17e48", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "element", "description": "The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output column. If omitted, the input column will be replaced.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,7 +74,7 @@ "name": "default", "description": "Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": "", "schema": { "type": [ @@ -91,31 +91,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/element.json b/wrangles-docs/static/registry/contracts/select/element.json index 3925ab9..d887e4b 100644 --- a/wrangles-docs/static/registry/contracts/select/element.json +++ b/wrangles-docs/static/registry/contracts/select/element.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "223d2f4e-3247-4189-8b6c-e73fe44c4266", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{\"key\":\"val\"}] -> \"val\".", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "default", "description": "Set the default value to return if the specified element doesn't exist.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -82,31 +82,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/group-by.json b/wrangles-docs/static/registry/contracts/select/group-by.json index fd171a1..b85a731 100644 --- a/wrangles-docs/static/registry/contracts/select/group-by.json +++ b/wrangles-docs/static/registry/contracts/select/group-by.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "c0af10b1-423a-416c-8cb5-7e7fe1164964", @@ -38,7 +38,7 @@ "name": "by", "description": "List of the input columns to group on.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": [], "schema": { "type": [ @@ -51,7 +51,7 @@ "name": "auto_rename_columns", "description": "If true (default), aggregated column names include the operation as a suffix (e.g. Value.sum). If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total).", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": true, "schema": { "type": "boolean" @@ -61,7 +61,7 @@ "name": "list", "description": "Group and return all values for these column(s) as a list.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -73,7 +73,7 @@ "name": "first", "description": "The first value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -85,7 +85,7 @@ "name": "last", "description": "The last value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -97,7 +97,7 @@ "name": "min", "description": "The minimum value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -109,7 +109,7 @@ "name": "max", "description": "The maximum value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -121,7 +121,7 @@ "name": "mean", "description": "The mean (average) value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -133,7 +133,7 @@ "name": "median", "description": "The median value for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -145,7 +145,7 @@ "name": "nunique", "description": "The count of unique values for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -157,7 +157,7 @@ "name": "count", "description": "The count of values for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -169,7 +169,7 @@ "name": "counts", "description": "Return a dictionary containing the count of each distinct value for these column(s). Keys are converted to JSON-safe strings; missing values use the key \"null\" and booleans use lowercase \"true\"/\"false\".", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -181,7 +181,7 @@ "name": "std", "description": "The standard deviation of values for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -193,7 +193,7 @@ "name": "sum", "description": "The total of values for these column(s).", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -205,7 +205,7 @@ "name": "any", "description": "Return true if any of the values for these column(s) are true.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -217,7 +217,7 @@ "name": "all", "description": "Return true if all of the values for these column(s) are true.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -229,7 +229,7 @@ "name": "p75", "description": "Get a percentile. Note, you can use any integer here for the corresponding percentile.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -242,7 +242,7 @@ "name_pattern": "^custom\\.[A-Za-z_][A-Za-z0-9_]*$", "description": "Placeholder for custom functions. Replace 'placeholder' with the name of the function.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "string", @@ -254,31 +254,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/head.json b/wrangles-docs/static/registry/contracts/select/head.json index 4523ef2..3952e94 100644 --- a/wrangles-docs/static/registry/contracts/select/head.json +++ b/wrangles-docs/static/registry/contracts/select/head.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "237af1ec-db7d-415a-88b7-70586a2191fb", @@ -38,7 +38,7 @@ "name": "n", "description": "Number of rows to return.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "integer" } @@ -47,31 +47,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/highest-confidence.json b/wrangles-docs/static/registry/contracts/select/highest-confidence.json index f8dff3b..2c851fd 100644 --- a/wrangles-docs/static/registry/contracts/select/highest-confidence.json +++ b/wrangles-docs/static/registry/contracts/select/highest-confidence.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "00aad85d-8cc8-42e5-86f3-e4ff916e8ac2", @@ -38,7 +38,7 @@ "name": "input", "description": "List of the input columns to select from.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "If two columns; the result and confidence. If one column; [result, confidence].", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -59,31 +59,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/left.json b/wrangles-docs/static/registry/contracts/select/left.json index 9726fd1..d5f7f29 100644 --- a/wrangles-docs/static/registry/contracts/select/left.json +++ b/wrangles-docs/static/registry/contracts/select/left.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "5cdd9857-0c77-43bf-80d7-d0a8cb6f980b", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column(s) to edit.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "length", "description": "Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "integer" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,31 +74,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/length.json b/wrangles-docs/static/registry/contracts/select/length.json index b57b1ad..434186c 100644 --- a/wrangles-docs/static/registry/contracts/select/length.json +++ b/wrangles-docs/static/registry/contracts/select/length.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "4d7a5f66-0a4a-40e0-8298-d5c55754423d", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column(s).", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,31 +65,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/list-element.json b/wrangles-docs/static/registry/contracts/select/list-element.json index cab532f..9cea94a 100644 --- a/wrangles-docs/static/registry/contracts/select/list-element.json +++ b/wrangles-docs/static/registry/contracts/select/list-element.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "ec40495d-d29a-4f62-86dd-eafa43cf388a", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the input column.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "element", "description": "The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": 0, "schema": { "type": "integer" @@ -75,7 +75,7 @@ "name": "default", "description": "Set the default value to return if the specified element doesn't exist.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": "", "schema": { "type": [ @@ -92,31 +92,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/right.json b/wrangles-docs/static/registry/contracts/select/right.json index af432e2..61402ac 100644 --- a/wrangles-docs/static/registry/contracts/select/right.json +++ b/wrangles-docs/static/registry/contracts/select/right.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "89ee82ec-3bc5-4bfa-899b-7a1260ef9bdb", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column(s) to edit.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "length", "description": "Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "integer" } @@ -60,7 +60,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -74,31 +74,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/sample.json b/wrangles-docs/static/registry/contracts/select/sample.json index 23ab4ea..9dfb59c 100644 --- a/wrangles-docs/static/registry/contracts/select/sample.json +++ b/wrangles-docs/static/registry/contracts/select/sample.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "95a84ab6-a66e-450f-8a4a-7a87e3a77932", @@ -38,7 +38,7 @@ "name": "rows", "description": "If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": [ "integer", @@ -51,31 +51,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/substring.json b/wrangles-docs/static/registry/contracts/select/substring.json index f9f108b..fa382e4 100644 --- a/wrangles-docs/static/registry/contracts/select/substring.json +++ b/wrangles-docs/static/registry/contracts/select/substring.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "8befddf8-602e-4fa9-8f16-4c547210ebec", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column(s) to edit.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "start", "description": "The position of the first character to select. If ommited will start from the beginning and length must be provided.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "length", "description": "The length of the string to select. If ommited will select to the end of the string and start must be provided.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -79,7 +79,7 @@ "name": "output", "description": "Name of the output column(s).", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -93,31 +93,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/tail.json b/wrangles-docs/static/registry/contracts/select/tail.json index 1910283..cd52f92 100644 --- a/wrangles-docs/static/registry/contracts/select/tail.json +++ b/wrangles-docs/static/registry/contracts/select/tail.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "60853759-c160-49a4-87eb-036516a9d823", @@ -38,7 +38,7 @@ "name": "n", "description": "Number of rows to return.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "integer" } @@ -47,31 +47,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/select/threshold.json b/wrangles-docs/static/registry/contracts/select/threshold.json index fc190f4..3ada443 100644 --- a/wrangles-docs/static/registry/contracts/select/threshold.json +++ b/wrangles-docs/static/registry/contracts/select/threshold.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "af2a5dcc-0ec0-48d7-8fb4-f58d9c5391d2", @@ -38,7 +38,7 @@ "name": "input", "description": "List of the input columns to select from.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -56,7 +56,7 @@ "name": "threshold", "description": "Threshold above which to choose the first option, otherwise the second.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "number", "minimum": 0, @@ -67,31 +67,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/similarity.json b/wrangles-docs/static/registry/contracts/similarity.json index 670569d..a7a09f6 100644 --- a/wrangles-docs/static/registry/contracts/similarity.json +++ b/wrangles-docs/static/registry/contracts/similarity.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "7c733344-4cce-4938-8013-53742fb46a90", @@ -38,7 +38,7 @@ "name": "input", "description": "Two columns of vectors to compare the similarity of.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "array", "minItems": 2, @@ -49,7 +49,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": "string" } @@ -58,7 +58,7 @@ "name": "method", "description": "The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "cosine", "schema": { "type": "string", @@ -73,31 +73,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/sort.json b/wrangles-docs/static/registry/contracts/sort.json index 3b51278..e6ad03d 100644 --- a/wrangles-docs/static/registry/contracts/sort.json +++ b/wrangles-docs/static/registry/contracts/sort.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "1dcf06ad-898a-4d83-862c-4774be37a687", @@ -38,7 +38,7 @@ "name": "ignore_index", "description": "Ignore Index value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": true, "schema": { "type": "boolean" @@ -48,7 +48,7 @@ "name": "by", "description": "Name or list of the column(s) to sort by.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -60,7 +60,7 @@ "name": "ascending", "description": "Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by.", "required": false, - "role": "option", + "param_group": "Options", "schema": { "type": [ "boolean", @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/split/dictionary.json b/wrangles-docs/static/registry/contracts/split/dictionary.json index ec8191b..804aae0 100644 --- a/wrangles-docs/static/registry/contracts/split/dictionary.json +++ b/wrangles-docs/static/registry/contracts/split/dictionary.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "06ca98e4-d026-43f7-84eb-af246d401ba9", @@ -38,7 +38,7 @@ "name": "input", "description": "Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "In columns output_format, this is an optional subset of keys to extract from the dictionary. If not provided, all keys will be returned. Columns can be renamed with the following syntax: output: - key1: new_column_name1 - key2: new_column_name2 In to_lists output_format, this must be two output columns for the keys and values lists. If not provided, Keys and Values will be used.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "default", "description": "Provide a set of default headings and values if they are not found within the input.", "required": false, - "role": "fallback-value", + "param_group": "Errors", "runtime_default": null, "schema": { "type": [ @@ -78,7 +78,7 @@ "name": "output_format", "description": "How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values.", "required": false, - "role": "column-output", + "param_group": "Formatting", "runtime_default": "columns", "schema": { "type": "string", @@ -92,31 +92,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/split/list.json b/wrangles-docs/static/registry/contracts/split/list.json index 66cd084..94f1274 100644 --- a/wrangles-docs/static/registry/contracts/split/list.json +++ b/wrangles-docs/static/registry/contracts/split/list.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "3260b9f7-aae2-499f-8004-d211c2cf643e", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column to be split.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -50,7 +50,7 @@ "name": "output", "description": "Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -62,31 +62,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/split/text.json b/wrangles-docs/static/registry/contracts/split/text.json index 59103ce..c98fef7 100644 --- a/wrangles-docs/static/registry/contracts/split/text.json +++ b/wrangles-docs/static/registry/contracts/split/text.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "e76e43f7-d129-4bf8-87b4-a304a378b130", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column to be split.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "output", "description": "Name of the output column(s) If a single column is provided, the results will be returned as a list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -61,7 +61,7 @@ "name": "char", "description": "Set the character(s) to split on. Default comma (,) Can also prefix with \"regex:\" to split on a pattern.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": ",", "schema": { "type": "string" @@ -71,7 +71,7 @@ "name": "pad", "description": "Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": null, "schema": { "type": [ @@ -84,7 +84,7 @@ "name": "element", "description": "Select a specific element or range after splitting using slicing syntax. e.g. 0, \":5\", \"5:\", \"2:8:2\".", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -98,7 +98,7 @@ "name": "inclusive", "description": "If true, include the split character in the output. Default False.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -108,7 +108,7 @@ "name": "skip_empty", "description": "Whether to skip empty values.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -118,31 +118,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/split/tokenize.json b/wrangles-docs/static/registry/contracts/split/tokenize.json index 9c38c37..add5cee 100644 --- a/wrangles-docs/static/registry/contracts/split/tokenize.json +++ b/wrangles-docs/static/registry/contracts/split/tokenize.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "6cc88418-ae0c-43f6-84ee-31e0d5f838c3", @@ -38,7 +38,7 @@ "name": "input", "description": "Column(s) to be split into tokens.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -65,7 +65,7 @@ "name": "method", "description": "Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "space", "schema": { "anyOf": [ @@ -87,31 +87,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/sql.json b/wrangles-docs/static/registry/contracts/sql.json index 21e996d..d68dc3b 100644 --- a/wrangles-docs/static/registry/contracts/sql.json +++ b/wrangles-docs/static/registry/contracts/sql.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "467a06b1-a697-4d31-8061-7d83a719fd79", @@ -38,7 +38,7 @@ "name": "command", "description": "SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string" } @@ -47,7 +47,7 @@ "name": "params", "description": "Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name).", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -61,7 +61,7 @@ "name": "preserve_index", "description": "Preserve Index value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": false, "schema": { "type": "boolean" @@ -71,7 +71,7 @@ "name": "preserve_data_types", "description": "Preserve Data Types value accepted by the runtime.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -81,31 +81,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/standardize.json b/wrangles-docs/static/registry/contracts/standardize.json index d3fbb88..c8ae208 100644 --- a/wrangles-docs/static/registry/contracts/standardize.json +++ b/wrangles-docs/static/registry/contracts/standardize.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "53cd3fdd-24e2-4411-8655-6014b92a3f3a", @@ -10,8 +10,9 @@ "aliases": [], "slug": "standardize", "title": "Standardize", - "description": "Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription.", - "status": "active", + "description": "Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model.", + "status": "deprecated", + "replaced_by": "standardize.custom", "visibility": "public", "tags": [ "format", @@ -38,7 +39,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +52,7 @@ "name": "model_id", "description": "The ID of the wrangle to use (do not include 'find' and 'replace').", "required": true, - "role": "model-reference", + "param_group": "Details", "schema": { "type": [ "string", @@ -63,7 +64,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -77,7 +78,7 @@ "name": "case_sensitive", "description": "Allows the wrangle to be case sensitive if set to True, default is False.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -87,31 +88,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/standardize/clean.json b/wrangles-docs/static/registry/contracts/standardize/clean.json index 1c4828e..f64af0a 100644 --- a/wrangles-docs/static/registry/contracts/standardize/clean.json +++ b/wrangles-docs/static/registry/contracts/standardize/clean.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name or list of output columns. Defaults to overwriting input.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -66,7 +66,7 @@ "name": "fix_encoding", "description": "Repair mojibake and other reversible encoding errors.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -76,7 +76,7 @@ "name": "unescape_html", "description": "Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "auto", "schema": { "anyOf": [ @@ -96,7 +96,7 @@ "name": "normalization", "description": "Unicode normalization form.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "NFC", "schema": { "type": "string", @@ -113,7 +113,7 @@ "name": "fix_character_width", "description": "Normalize fullwidth and halfwidth characters.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -123,7 +123,7 @@ "name": "uncurl_quotes", "description": "Replace typographic quotes with straight quotes.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -133,7 +133,7 @@ "name": "remove_control_chars", "description": "Remove C0 and C1 control characters.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -143,7 +143,7 @@ "name": "collapse_whitespace", "description": "Collapse runs of Unicode whitespace.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -153,7 +153,7 @@ "name": "preserve_line_breaks", "description": "Preserve line breaks while collapsing other whitespace.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -163,7 +163,7 @@ "name": "trim", "description": "Remove leading and trailing whitespace.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": true, "schema": { "type": "boolean" @@ -173,7 +173,7 @@ "name": "separator", "description": "Text used to join multiple input columns into one output.", "required": false, - "role": "option", + "param_group": "Formatting", "runtime_default": " ", "schema": { "type": "string" @@ -183,35 +183,35 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], - "guidance": "## Behavior\n\nRepair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally.\n\nThis first-pass guidance is derived from the callable signature and its embedded Python schema docstring.", + "guidance": "## Behavior\n\nRepair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally.\n\nThis guidance was derived from the callable signature and its embedded Python schema docstring.", "sources": [ { "id": "runtime", diff --git a/wrangles-docs/static/registry/contracts/standardize/custom.json b/wrangles-docs/static/registry/contracts/standardize/custom.json index 114751c..58af7d3 100644 --- a/wrangles-docs/static/registry/contracts/standardize/custom.json +++ b/wrangles-docs/static/registry/contracts/standardize/custom.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "input", "description": "Name or list of input columns.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "model_id", "description": "The ID of the wrangle to use (do not include 'find' and 'replace').", "required": true, - "role": "model-reference", + "param_group": "Details", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "output", "description": "Name or list of output columns.", "required": false, - "role": "column-output", + "param_group": "I/O", "runtime_default": null, "schema": { "type": [ @@ -77,7 +77,7 @@ "name": "case_sensitive", "description": "Allows the wrangle to be case sensitive if set to True, default is False.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": false, "schema": { "type": "boolean" @@ -87,35 +87,35 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], - "guidance": "## Behavior\n\nStandardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription.\n\nThis first-pass guidance is derived from the callable signature and its embedded Python schema docstring.", + "guidance": "## Behavior\n\nStandardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription.\n\nThis guidance was derived from the callable signature and its embedded Python schema docstring.", "sources": [ { "id": "runtime", diff --git a/wrangles-docs/static/registry/contracts/translate.json b/wrangles-docs/static/registry/contracts/translate.json index b169e2b..d6c0d22 100644 --- a/wrangles-docs/static/registry/contracts/translate.json +++ b/wrangles-docs/static/registry/contracts/translate.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "73c3ceb6-ffd8-4d74-8389-c83b99d33bb0", @@ -38,7 +38,7 @@ "name": "input", "description": "Name of the column to translate.", "required": true, - "role": "column-selector", + "param_group": "I/O", "schema": { "type": [ "string", @@ -51,7 +51,7 @@ "name": "output", "description": "Name of the output column.", "required": true, - "role": "column-output", + "param_group": "I/O", "schema": { "type": [ "string", @@ -63,7 +63,7 @@ "name": "target_language", "description": "Code of the language to translate to.", "required": true, - "role": "option", + "param_group": "Options", "schema": { "type": "string", "enum": [ @@ -100,7 +100,7 @@ "name": "source_language", "description": "Code of the language to translate from. If omitted, automatically detects the input language.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": "AUTO", "schema": { "type": "string", @@ -137,7 +137,7 @@ "name": "case", "description": "Allow changing the case of the input prior to translation.", "required": false, - "role": "option", + "param_group": "Options", "runtime_default": null, "schema": { "type": [ @@ -150,31 +150,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/transpose.json b/wrangles-docs/static/registry/contracts/transpose.json index dc1deb4..a9140f7 100644 --- a/wrangles-docs/static/registry/contracts/transpose.json +++ b/wrangles-docs/static/registry/contracts/transpose.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": "8716347f-f286-49b2-8a0b-cb73292e7475", @@ -38,7 +38,7 @@ "name": "header_column", "description": "Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header.", "required": false, - "role": "column-selector", + "param_group": "I/O", "runtime_default": 0, "schema": { "type": [ @@ -52,31 +52,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/contracts/try.json b/wrangles-docs/static/registry/contracts/try.json index 4b4f1bd..fe368fc 100644 --- a/wrangles-docs/static/registry/contracts/try.json +++ b/wrangles-docs/static/registry/contracts/try.json @@ -1,6 +1,6 @@ { "format": "wrangles-registry-entry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "schema_version": "0.1", "type": "wrangle", "id": null, @@ -38,7 +38,7 @@ "name": "wrangles", "description": "List of wrangles to apply.", "required": true, - "role": "nested-wrangles", + "param_group": "Execution", "schema": { "type": "array", "minItems": 1, @@ -51,7 +51,7 @@ "name": "retries", "description": "Number of times to retry the wrangles if an error occurs. Default 0.", "required": false, - "role": "option", + "param_group": "Errors", "runtime_default": 0, "schema": { "type": "integer", @@ -62,7 +62,7 @@ "name": "except", "description": "An action to take if the wrangles encounter an error. This can contain a list of wrangles or a dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue.", "required": false, - "role": "option", + "param_group": "Errors", "schema": { "type": "object", "minItems": 1, @@ -75,31 +75,31 @@ "name": "if", "description": "Condition that determines whether the wrangle runs as a whole. Recipe variables may be referenced with `${variable}`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where", "description": "Filter rows before applying the wrangle using SQL-like criteria, such as `column1 = 123 OR column2 = 'abc'`.", "required": false, + "param_group": "Conditions", "schema": { "type": "string" - }, - "role": "common-control" + } }, { "name": "where_params", "description": "Values used with `where` for parameterized criteria. Uses SQLite placeholder syntax such as `?` or `:name`.", "required": false, + "param_group": "Conditions", "schema": { "type": [ "array", "object" ] - }, - "role": "common-control" + } } ], "examples": [], diff --git a/wrangles-docs/static/registry/index.md b/wrangles-docs/static/registry/index.md index 61f435c..f7e7ad1 100644 --- a/wrangles-docs/static/registry/index.md +++ b/wrangles-docs/static/registry/index.md @@ -2,15 +2,15 @@ okf_version: "0.2" type: collection title: Wrangles Registry -description: Public pilot bundle for Wrangles recipe primitives. -status: pilot -registry_version: 0.1.0-pilot +description: Public pre-production bundle for Wrangles recipe primitives. +status: pre-production +registry_version: 0.1.0 --- # Wrangles Registry -This is the public, agent-readable pilot bundle. Use `manifest.json` for -structured discovery. +This is the public, agent-readable pre-production bundle. Use `manifest.json` +for structured discovery. - [`accordion`](wrangles/_root/accordion.md): Apply a series of wrangles to column(s) containing lists. The wrangles will be applied to each element in the list and the results will be returned back as a list. - [`batch`](wrangles/_root/batch.md): Split the data into batches for executing a list of wrangles. Use this in situations such as where the intermediate data is too large to fit in memory. @@ -104,7 +104,7 @@ structured discovery. - [`split.text`](wrangles/split/text.md): Split a string to multiple columns or a list. - [`split.tokenize`](wrangles/split/tokenize.md): Split text into tokens. A variety of methods are available. The default method is to split on spaces. - [`sql`](wrangles/_root/sql.md): Apply a SQL command to the current dataframe. Only SELECT statements are supported - the result will be the output. -- [`standardize`](wrangles/_root/standardize.md): Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. +- [`standardize`](wrangles/_root/standardize.md): Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model. - [`standardize.clean`](wrangles/standardize/clean.md): Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. - [`standardize.custom`](wrangles/standardize/custom.md): Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. - [`translate`](wrangles/_root/translate.md): Translate the input to a different language. Requires WrangleWorks Account and DeepL API Key (A free account for up to 500,000 characters per month is available). diff --git a/wrangles-docs/static/registry/manifest.json b/wrangles-docs/static/registry/manifest.json index ab425e4..a72bc28 100644 --- a/wrangles-docs/static/registry/manifest.json +++ b/wrangles-docs/static/registry/manifest.json @@ -1,8 +1,8 @@ { "format": "wrangles-registry", - "registry_version": "0.1.0-pilot", + "registry_version": "0.1.0", "contract_version": "0.1", - "status": "pilot", + "status": "pre-production", "entry_count": 98, "entries": [ { @@ -20,7 +20,7 @@ "utility", "accordion" ], - "route": "/wrangles/accordion", + "route": "/wrangles/namespaces/utility#accordion", "contract_json": "/registry/contracts/accordion.json", "raw_markdown": "/registry/wrangles/_root/accordion.md", "runtime_symbol": "wrangles.recipe_wrangles.main.accordion", @@ -42,7 +42,7 @@ "utility", "batch" ], - "route": "/wrangles/batch", + "route": "/wrangles/namespaces/utility#batch", "contract_json": "/registry/contracts/batch.json", "raw_markdown": "/registry/wrangles/_root/batch.md", "runtime_symbol": "wrangles.recipe_wrangles.main.batch", @@ -64,7 +64,7 @@ "ai", "classify" ], - "route": "/wrangles/classify", + "route": "/wrangles/namespaces/ai#classify", "contract_json": "/registry/contracts/classify.json", "raw_markdown": "/registry/wrangles/_root/classify.md", "runtime_symbol": "wrangles.recipe_wrangles.main.classify", @@ -86,7 +86,7 @@ "format", "clean-whitespaces" ], - "route": "/wrangles/clean-whitespaces", + "route": "/wrangles/namespaces/format#clean-whitespaces", "contract_json": "/registry/contracts/clean-whitespaces.json", "raw_markdown": "/registry/wrangles/_root/clean-whitespaces.md", "runtime_symbol": "wrangles.recipe_wrangles.main.clean_whitespaces", @@ -108,7 +108,7 @@ "compare", "lists" ], - "route": "/wrangles/compare/lists", + "route": "/wrangles/namespaces/compare#lists", "contract_json": "/registry/contracts/compare/lists.json", "raw_markdown": "/registry/wrangles/compare/lists.md", "runtime_symbol": "wrangles.recipe_wrangles.compare.lists", @@ -130,7 +130,7 @@ "compare", "text" ], - "route": "/wrangles/compare/text", + "route": "/wrangles/namespaces/compare#text", "contract_json": "/registry/contracts/compare/text.json", "raw_markdown": "/registry/wrangles/compare/text.md", "runtime_symbol": "wrangles.recipe_wrangles.compare.text", @@ -152,7 +152,7 @@ "compute", "case-when" ], - "route": "/wrangles/compute/case-when", + "route": "/wrangles/namespaces/compute#case-when", "contract_json": "/registry/contracts/compute/case-when.json", "raw_markdown": "/registry/wrangles/compute/case-when.md", "runtime_symbol": "wrangles.recipe_wrangles.compute.case_when", @@ -174,7 +174,7 @@ "compute", "score-search-results" ], - "route": "/wrangles/compute/score-search-results", + "route": "/wrangles/namespaces/compute#score-search-results", "contract_json": "/registry/contracts/compute/score-search-results.json", "raw_markdown": "/registry/wrangles/compute/score-search-results.md", "runtime_symbol": "wrangles.recipe_wrangles.compute.score_search_results", @@ -196,7 +196,7 @@ "utility", "concurrent" ], - "route": "/wrangles/concurrent", + "route": "/wrangles/namespaces/utility#concurrent", "contract_json": "/registry/contracts/concurrent.json", "raw_markdown": "/registry/wrangles/_root/concurrent.md", "runtime_symbol": "wrangles.recipe_wrangles.main.concurrent", @@ -219,7 +219,7 @@ "text", "case" ], - "route": "/wrangles/convert/case", + "route": "/wrangles/namespaces/convert#case", "contract_json": "/registry/contracts/convert/case.json", "raw_markdown": "/registry/wrangles/convert/case.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.case", @@ -242,7 +242,7 @@ "data-type", "validation" ], - "route": "/wrangles/convert/data-type", + "route": "/wrangles/namespaces/convert#data-type", "contract_json": "/registry/contracts/convert/data-type.json", "raw_markdown": "/registry/wrangles/convert/data-type.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.data_type", @@ -264,7 +264,7 @@ "convert", "fraction-to-decimal" ], - "route": "/wrangles/convert/fraction-to-decimal", + "route": "/wrangles/namespaces/convert#fraction-to-decimal", "contract_json": "/registry/contracts/convert/fraction-to-decimal.json", "raw_markdown": "/registry/wrangles/convert/fraction-to-decimal.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.fraction_to_decimal", @@ -287,7 +287,7 @@ "json", "structured-data" ], - "route": "/wrangles/convert/from-json", + "route": "/wrangles/namespaces/convert#from-json", "contract_json": "/registry/contracts/convert/from-json.json", "raw_markdown": "/registry/wrangles/convert/from-json.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.from_json", @@ -309,7 +309,7 @@ "convert", "from-yaml" ], - "route": "/wrangles/convert/from-yaml", + "route": "/wrangles/namespaces/convert#from-yaml", "contract_json": "/registry/contracts/convert/from-yaml.json", "raw_markdown": "/registry/wrangles/convert/from-yaml.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.from_yaml", @@ -331,7 +331,7 @@ "convert", "to-json" ], - "route": "/wrangles/convert/to-json", + "route": "/wrangles/namespaces/convert#to-json", "contract_json": "/registry/contracts/convert/to-json.json", "raw_markdown": "/registry/wrangles/convert/to-json.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.to_json", @@ -353,7 +353,7 @@ "convert", "to-yaml" ], - "route": "/wrangles/convert/to-yaml", + "route": "/wrangles/namespaces/convert#to-yaml", "contract_json": "/registry/contracts/convert/to-yaml.json", "raw_markdown": "/registry/wrangles/convert/to-yaml.md", "runtime_symbol": "wrangles.recipe_wrangles.convert.to_yaml", @@ -375,7 +375,7 @@ "transform", "copy" ], - "route": "/wrangles/copy", + "route": "/wrangles/namespaces/transform#copy", "contract_json": "/registry/contracts/copy.json", "raw_markdown": "/registry/wrangles/_root/copy.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.copy", @@ -397,7 +397,7 @@ "create", "bins" ], - "route": "/wrangles/create/bins", + "route": "/wrangles/namespaces/create#bins", "contract_json": "/registry/contracts/create/bins.json", "raw_markdown": "/registry/wrangles/create/bins.md", "runtime_symbol": "wrangles.recipe_wrangles.create.bins", @@ -419,7 +419,7 @@ "create", "column" ], - "route": "/wrangles/create/column", + "route": "/wrangles/namespaces/create#column", "contract_json": "/registry/contracts/create/column.json", "raw_markdown": "/registry/wrangles/create/column.md", "runtime_symbol": "wrangles.recipe_wrangles.create.column", @@ -441,7 +441,7 @@ "create", "embeddings" ], - "route": "/wrangles/create/embeddings", + "route": "/wrangles/namespaces/create#embeddings", "contract_json": "/registry/contracts/create/embeddings.json", "raw_markdown": "/registry/wrangles/create/embeddings.md", "runtime_symbol": "wrangles.recipe_wrangles.create.embeddings", @@ -463,7 +463,7 @@ "create", "guid" ], - "route": "/wrangles/create/guid", + "route": "/wrangles/namespaces/create#guid", "contract_json": "/registry/contracts/create/guid.json", "raw_markdown": "/registry/wrangles/create/guid.md", "runtime_symbol": "wrangles.recipe_wrangles.create.guid", @@ -485,7 +485,7 @@ "create", "hash" ], - "route": "/wrangles/create/hash", + "route": "/wrangles/namespaces/create#hash", "contract_json": "/registry/contracts/create/hash.json", "raw_markdown": "/registry/wrangles/create/hash.md", "runtime_symbol": "wrangles.recipe_wrangles.create.hash", @@ -507,7 +507,7 @@ "create", "index" ], - "route": "/wrangles/create/index", + "route": "/wrangles/namespaces/create#index", "contract_json": "/registry/contracts/create/index.json", "raw_markdown": "/registry/wrangles/create/index.md", "runtime_symbol": "wrangles.recipe_wrangles.create.index", @@ -529,7 +529,7 @@ "create", "jinja" ], - "route": "/wrangles/create/jinja", + "route": "/wrangles/namespaces/create#jinja", "contract_json": "/registry/contracts/create/jinja.json", "raw_markdown": "/registry/wrangles/create/jinja.md", "runtime_symbol": "wrangles.recipe_wrangles.create.jinja", @@ -551,7 +551,7 @@ "create", "uuid" ], - "route": "/wrangles/create/uuid", + "route": "/wrangles/namespaces/create#uuid", "contract_json": "/registry/contracts/create/uuid.json", "raw_markdown": "/registry/wrangles/create/uuid.md", "runtime_symbol": "wrangles.recipe_wrangles.create.uuid", @@ -573,7 +573,7 @@ "date", "date-calculator" ], - "route": "/wrangles/date-calculator", + "route": "/wrangles/namespaces/date#date-calculator", "contract_json": "/registry/contracts/date-calculator.json", "raw_markdown": "/registry/wrangles/_root/date-calculator.md", "runtime_symbol": "wrangles.recipe_wrangles.main.date_calculator", @@ -595,7 +595,7 @@ "select", "drop" ], - "route": "/wrangles/drop", + "route": "/wrangles/namespaces/select#drop", "contract_json": "/registry/contracts/drop.json", "raw_markdown": "/registry/wrangles/_root/drop.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.drop", @@ -617,7 +617,7 @@ "split", "explode" ], - "route": "/wrangles/explode", + "route": "/wrangles/namespaces/split#explode", "contract_json": "/registry/contracts/explode.json", "raw_markdown": "/registry/wrangles/_root/explode.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.explode", @@ -639,7 +639,7 @@ "extract", "address" ], - "route": "/wrangles/extract/address", + "route": "/wrangles/namespaces/extract#address", "contract_json": "/registry/contracts/extract/address.json", "raw_markdown": "/registry/wrangles/extract/address.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.address", @@ -661,7 +661,7 @@ "ai", "extract" ], - "route": "/wrangles/extract/ai", + "route": "/wrangles/namespaces/extract#ai", "contract_json": "/registry/contracts/extract/ai.json", "raw_markdown": "/registry/wrangles/extract/ai.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.ai", @@ -683,7 +683,7 @@ "extract", "attributes" ], - "route": "/wrangles/extract/attributes", + "route": "/wrangles/namespaces/extract#attributes", "contract_json": "/registry/contracts/extract/attributes.json", "raw_markdown": "/registry/wrangles/extract/attributes.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.attributes", @@ -705,7 +705,7 @@ "extract", "brackets" ], - "route": "/wrangles/extract/brackets", + "route": "/wrangles/namespaces/extract#brackets", "contract_json": "/registry/contracts/extract/brackets.json", "raw_markdown": "/registry/wrangles/extract/brackets.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.brackets", @@ -727,7 +727,7 @@ "extract", "codes" ], - "route": "/wrangles/extract/codes", + "route": "/wrangles/namespaces/extract#codes", "contract_json": "/registry/contracts/extract/codes.json", "raw_markdown": "/registry/wrangles/extract/codes.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.codes", @@ -749,7 +749,7 @@ "extract", "custom" ], - "route": "/wrangles/extract/custom", + "route": "/wrangles/namespaces/extract#custom", "contract_json": "/registry/contracts/extract/custom.json", "raw_markdown": "/registry/wrangles/extract/custom.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.custom", @@ -771,7 +771,7 @@ "extract", "date-properties" ], - "route": "/wrangles/extract/date-properties", + "route": "/wrangles/namespaces/extract#date-properties", "contract_json": "/registry/contracts/extract/date-properties.json", "raw_markdown": "/registry/wrangles/extract/date-properties.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.date_properties", @@ -793,7 +793,7 @@ "extract", "date-range" ], - "route": "/wrangles/extract/date-range", + "route": "/wrangles/namespaces/extract#date-range", "contract_json": "/registry/contracts/extract/date-range.json", "raw_markdown": "/registry/wrangles/extract/date-range.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.date_range", @@ -815,7 +815,7 @@ "extract", "html" ], - "route": "/wrangles/extract/html", + "route": "/wrangles/namespaces/extract#html", "contract_json": "/registry/contracts/extract/html.json", "raw_markdown": "/registry/wrangles/extract/html.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.html", @@ -837,7 +837,7 @@ "extract", "properties" ], - "route": "/wrangles/extract/properties", + "route": "/wrangles/namespaces/extract#properties", "contract_json": "/registry/contracts/extract/properties.json", "raw_markdown": "/registry/wrangles/extract/properties.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.properties", @@ -859,7 +859,7 @@ "extract", "regex" ], - "route": "/wrangles/extract/regex", + "route": "/wrangles/namespaces/extract#regex", "contract_json": "/registry/contracts/extract/regex.json", "raw_markdown": "/registry/wrangles/extract/regex.md", "runtime_symbol": "wrangles.recipe_wrangles.extract.regex", @@ -881,7 +881,7 @@ "select", "filter" ], - "route": "/wrangles/filter", + "route": "/wrangles/namespaces/select#filter", "contract_json": "/registry/contracts/filter.json", "raw_markdown": "/registry/wrangles/_root/filter.md", "runtime_symbol": "wrangles.recipe_wrangles.main.filter", @@ -903,7 +903,7 @@ "format", "dates" ], - "route": "/wrangles/format/dates", + "route": "/wrangles/namespaces/format#dates", "contract_json": "/registry/contracts/format/dates.json", "raw_markdown": "/registry/wrangles/format/dates.md", "runtime_symbol": "wrangles.recipe_wrangles.format.dates", @@ -925,7 +925,7 @@ "format", "pad" ], - "route": "/wrangles/format/pad", + "route": "/wrangles/namespaces/format#pad", "contract_json": "/registry/contracts/format/pad.json", "raw_markdown": "/registry/wrangles/format/pad.md", "runtime_symbol": "wrangles.recipe_wrangles.format.pad", @@ -947,7 +947,7 @@ "format", "prefix" ], - "route": "/wrangles/format/prefix", + "route": "/wrangles/namespaces/format#prefix", "contract_json": "/registry/contracts/format/prefix.json", "raw_markdown": "/registry/wrangles/format/prefix.md", "runtime_symbol": "wrangles.recipe_wrangles.format.prefix", @@ -969,7 +969,7 @@ "format", "price-breaks" ], - "route": "/wrangles/format/price-breaks", + "route": "/wrangles/namespaces/format#price-breaks", "contract_json": "/registry/contracts/format/price-breaks.json", "raw_markdown": "/registry/wrangles/format/price-breaks.md", "runtime_symbol": "wrangles.recipe_wrangles.format.price_breaks", @@ -991,7 +991,7 @@ "format", "remove-duplicates" ], - "route": "/wrangles/format/remove-duplicates", + "route": "/wrangles/namespaces/format#remove-duplicates", "contract_json": "/registry/contracts/format/remove-duplicates.json", "raw_markdown": "/registry/wrangles/format/remove-duplicates.md", "runtime_symbol": "wrangles.recipe_wrangles.format.remove_duplicates", @@ -1013,7 +1013,7 @@ "format", "significant-figures" ], - "route": "/wrangles/format/significant-figures", + "route": "/wrangles/namespaces/format#significant-figures", "contract_json": "/registry/contracts/format/significant-figures.json", "raw_markdown": "/registry/wrangles/format/significant-figures.md", "runtime_symbol": "wrangles.recipe_wrangles.format.significant_figures", @@ -1035,7 +1035,7 @@ "format", "suffix" ], - "route": "/wrangles/format/suffix", + "route": "/wrangles/namespaces/format#suffix", "contract_json": "/registry/contracts/format/suffix.json", "raw_markdown": "/registry/wrangles/format/suffix.md", "runtime_symbol": "wrangles.recipe_wrangles.format.suffix", @@ -1057,7 +1057,7 @@ "format", "trim" ], - "route": "/wrangles/format/trim", + "route": "/wrangles/namespaces/format#trim", "contract_json": "/registry/contracts/format/trim.json", "raw_markdown": "/registry/wrangles/format/trim.md", "runtime_symbol": "wrangles.recipe_wrangles.format.trim", @@ -1079,7 +1079,7 @@ "ai", "generate" ], - "route": "/wrangles/generate/ai", + "route": "/wrangles/namespaces/generate#ai", "contract_json": "/registry/contracts/generate/ai.json", "raw_markdown": "/registry/wrangles/generate/ai.md", "runtime_symbol": "wrangles.recipe_wrangles.generate.ai", @@ -1101,7 +1101,7 @@ "ai", "huggingface" ], - "route": "/wrangles/huggingface", + "route": "/wrangles/namespaces/ai#huggingface", "contract_json": "/registry/contracts/huggingface.json", "raw_markdown": "/registry/wrangles/_root/huggingface.md", "runtime_symbol": "wrangles.recipe_wrangles.main.huggingface", @@ -1123,7 +1123,7 @@ "utility", "log" ], - "route": "/wrangles/log", + "route": "/wrangles/namespaces/utility#log", "contract_json": "/registry/contracts/log.json", "raw_markdown": "/registry/wrangles/_root/log.md", "runtime_symbol": "wrangles.recipe_wrangles.main.log", @@ -1144,7 +1144,7 @@ "tags": [ "lookup" ], - "route": "/wrangles/lookup", + "route": "/wrangles/namespaces/lookup#lookup", "contract_json": "/registry/contracts/lookup.json", "raw_markdown": "/registry/wrangles/_root/lookup.md", "runtime_symbol": "wrangles.recipe_wrangles.main.lookup", @@ -1166,7 +1166,7 @@ "compute", "math" ], - "route": "/wrangles/math", + "route": "/wrangles/namespaces/compute#math", "contract_json": "/registry/contracts/math.json", "raw_markdown": "/registry/wrangles/_root/math.md", "runtime_symbol": "wrangles.recipe_wrangles.main.math", @@ -1184,10 +1184,11 @@ "title": "Maths", "description": "Deprecated alias for `math`; evaluate an expression and write its result to an output column.", "status": "deprecated", + "replaced_by": "math", "tags": [ "maths" ], - "route": "/wrangles/maths", + "route": "/wrangles/namespaces/compute#maths", "contract_json": "/registry/contracts/maths.json", "raw_markdown": "/registry/wrangles/_root/maths.md", "runtime_symbol": "wrangles.recipe_wrangles.main.maths", @@ -1209,7 +1210,7 @@ "utility", "matrix" ], - "route": "/wrangles/matrix", + "route": "/wrangles/namespaces/utility#matrix", "contract_json": "/registry/contracts/matrix.json", "raw_markdown": "/registry/wrangles/_root/matrix.md", "runtime_symbol": "wrangles.recipe_wrangles.main.matrix", @@ -1231,7 +1232,7 @@ "merge", "coalesce" ], - "route": "/wrangles/merge/coalesce", + "route": "/wrangles/namespaces/merge#coalesce", "contract_json": "/registry/contracts/merge/coalesce.json", "raw_markdown": "/registry/wrangles/merge/coalesce.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.coalesce", @@ -1253,7 +1254,7 @@ "merge", "concatenate" ], - "route": "/wrangles/merge/concatenate", + "route": "/wrangles/namespaces/merge#concatenate", "contract_json": "/registry/contracts/merge/concatenate.json", "raw_markdown": "/registry/wrangles/merge/concatenate.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.concatenate", @@ -1275,7 +1276,7 @@ "merge", "dictionaries" ], - "route": "/wrangles/merge/dictionaries", + "route": "/wrangles/namespaces/merge#dictionaries", "contract_json": "/registry/contracts/merge/dictionaries.json", "raw_markdown": "/registry/wrangles/merge/dictionaries.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.dictionaries", @@ -1297,7 +1298,7 @@ "merge", "key-value-pairs" ], - "route": "/wrangles/merge/key-value-pairs", + "route": "/wrangles/namespaces/merge#key-value-pairs", "contract_json": "/registry/contracts/merge/key-value-pairs.json", "raw_markdown": "/registry/wrangles/merge/key-value-pairs.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.key_value_pairs", @@ -1319,7 +1320,7 @@ "merge", "lists" ], - "route": "/wrangles/merge/lists", + "route": "/wrangles/namespaces/merge#lists", "contract_json": "/registry/contracts/merge/lists.json", "raw_markdown": "/registry/wrangles/merge/lists.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.lists", @@ -1341,7 +1342,7 @@ "merge", "to-dict" ], - "route": "/wrangles/merge/to-dict", + "route": "/wrangles/namespaces/merge#to-dict", "contract_json": "/registry/contracts/merge/to-dict.json", "raw_markdown": "/registry/wrangles/merge/to-dict.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.to_dict", @@ -1363,7 +1364,7 @@ "merge", "to-list" ], - "route": "/wrangles/merge/to-list", + "route": "/wrangles/namespaces/merge#to-list", "contract_json": "/registry/contracts/merge/to-list.json", "raw_markdown": "/registry/wrangles/merge/to-list.md", "runtime_symbol": "wrangles.recipe_wrangles.merge.to_list", @@ -1385,7 +1386,7 @@ "compute", "python" ], - "route": "/wrangles/python", + "route": "/wrangles/namespaces/compute#python", "contract_json": "/registry/contracts/python.json", "raw_markdown": "/registry/wrangles/_root/python.md", "runtime_symbol": "wrangles.recipe_wrangles.main.python", @@ -1406,7 +1407,7 @@ "tags": [ "recipe" ], - "route": "/wrangles/recipe", + "route": "/wrangles/namespaces/utility#recipe", "contract_json": "/registry/contracts/recipe.json", "raw_markdown": "/registry/wrangles/_root/recipe.md", "runtime_symbol": "wrangles.recipe_wrangles.main.recipe", @@ -1428,7 +1429,7 @@ "transform", "reindex" ], - "route": "/wrangles/reindex", + "route": "/wrangles/namespaces/transform#reindex", "contract_json": "/registry/contracts/reindex.json", "raw_markdown": "/registry/wrangles/_root/reindex.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.reindex", @@ -1450,7 +1451,7 @@ "format", "remove-words" ], - "route": "/wrangles/remove-words", + "route": "/wrangles/namespaces/format#remove-words", "contract_json": "/registry/contracts/remove-words.json", "raw_markdown": "/registry/wrangles/_root/remove-words.md", "runtime_symbol": "wrangles.recipe_wrangles.main.remove_words", @@ -1472,7 +1473,7 @@ "transform", "rename" ], - "route": "/wrangles/rename", + "route": "/wrangles/namespaces/transform#rename", "contract_json": "/registry/contracts/rename.json", "raw_markdown": "/registry/wrangles/_root/rename.md", "runtime_symbol": "wrangles.recipe_wrangles.main.rename", @@ -1494,7 +1495,7 @@ "format", "replace" ], - "route": "/wrangles/replace", + "route": "/wrangles/namespaces/format#replace", "contract_json": "/registry/contracts/replace.json", "raw_markdown": "/registry/wrangles/_root/replace.md", "runtime_symbol": "wrangles.recipe_wrangles.main.replace", @@ -1516,7 +1517,7 @@ "format", "round" ], - "route": "/wrangles/round", + "route": "/wrangles/namespaces/format#round", "contract_json": "/registry/contracts/round.json", "raw_markdown": "/registry/wrangles/_root/round.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.round", @@ -1538,7 +1539,7 @@ "search", "find-links" ], - "route": "/wrangles/search/find-links", + "route": "/wrangles/namespaces/search#find-links", "contract_json": "/registry/contracts/search/find-links.json", "raw_markdown": "/registry/wrangles/search/find-links.md", "runtime_symbol": "wrangles.recipe_wrangles.search.find_links", @@ -1560,7 +1561,7 @@ "search", "retrieve-link-content" ], - "route": "/wrangles/search/retrieve-link-content", + "route": "/wrangles/namespaces/search#retrieve-link-content", "contract_json": "/registry/contracts/search/retrieve-link-content.json", "raw_markdown": "/registry/wrangles/search/retrieve-link-content.md", "runtime_symbol": "wrangles.recipe_wrangles.search.retrieve_link_content", @@ -1582,7 +1583,7 @@ "select", "columns" ], - "route": "/wrangles/select/columns", + "route": "/wrangles/namespaces/select#columns", "contract_json": "/registry/contracts/select/columns.json", "raw_markdown": "/registry/wrangles/select/columns.md", "runtime_symbol": "wrangles.recipe_wrangles.select.columns", @@ -1604,7 +1605,7 @@ "select", "dictionary-element" ], - "route": "/wrangles/select/dictionary-element", + "route": "/wrangles/namespaces/select#dictionary-element", "contract_json": "/registry/contracts/select/dictionary-element.json", "raw_markdown": "/registry/wrangles/select/dictionary-element.md", "runtime_symbol": "wrangles.recipe_wrangles.select.dictionary_element", @@ -1626,7 +1627,7 @@ "select", "element" ], - "route": "/wrangles/select/element", + "route": "/wrangles/namespaces/select#element", "contract_json": "/registry/contracts/select/element.json", "raw_markdown": "/registry/wrangles/select/element.md", "runtime_symbol": "wrangles.recipe_wrangles.select.element", @@ -1648,7 +1649,7 @@ "select", "group-by" ], - "route": "/wrangles/select/group-by", + "route": "/wrangles/namespaces/select#group-by", "contract_json": "/registry/contracts/select/group-by.json", "raw_markdown": "/registry/wrangles/select/group-by.md", "runtime_symbol": "wrangles.recipe_wrangles.select.group_by", @@ -1670,7 +1671,7 @@ "select", "head" ], - "route": "/wrangles/select/head", + "route": "/wrangles/namespaces/select#head", "contract_json": "/registry/contracts/select/head.json", "raw_markdown": "/registry/wrangles/select/head.md", "runtime_symbol": "wrangles.recipe_wrangles.select.head", @@ -1692,7 +1693,7 @@ "select", "highest-confidence" ], - "route": "/wrangles/select/highest-confidence", + "route": "/wrangles/namespaces/select#highest-confidence", "contract_json": "/registry/contracts/select/highest-confidence.json", "raw_markdown": "/registry/wrangles/select/highest-confidence.md", "runtime_symbol": "wrangles.recipe_wrangles.select.highest_confidence", @@ -1714,7 +1715,7 @@ "select", "left" ], - "route": "/wrangles/select/left", + "route": "/wrangles/namespaces/select#left", "contract_json": "/registry/contracts/select/left.json", "raw_markdown": "/registry/wrangles/select/left.md", "runtime_symbol": "wrangles.recipe_wrangles.select.left", @@ -1736,7 +1737,7 @@ "select", "length" ], - "route": "/wrangles/select/length", + "route": "/wrangles/namespaces/select#length", "contract_json": "/registry/contracts/select/length.json", "raw_markdown": "/registry/wrangles/select/length.md", "runtime_symbol": "wrangles.recipe_wrangles.select.length", @@ -1758,7 +1759,7 @@ "select", "list-element" ], - "route": "/wrangles/select/list-element", + "route": "/wrangles/namespaces/select#list-element", "contract_json": "/registry/contracts/select/list-element.json", "raw_markdown": "/registry/wrangles/select/list-element.md", "runtime_symbol": "wrangles.recipe_wrangles.select.list_element", @@ -1780,7 +1781,7 @@ "select", "right" ], - "route": "/wrangles/select/right", + "route": "/wrangles/namespaces/select#right", "contract_json": "/registry/contracts/select/right.json", "raw_markdown": "/registry/wrangles/select/right.md", "runtime_symbol": "wrangles.recipe_wrangles.select.right", @@ -1802,7 +1803,7 @@ "select", "sample" ], - "route": "/wrangles/select/sample", + "route": "/wrangles/namespaces/select#sample", "contract_json": "/registry/contracts/select/sample.json", "raw_markdown": "/registry/wrangles/select/sample.md", "runtime_symbol": "wrangles.recipe_wrangles.select.sample", @@ -1824,7 +1825,7 @@ "select", "substring" ], - "route": "/wrangles/select/substring", + "route": "/wrangles/namespaces/select#substring", "contract_json": "/registry/contracts/select/substring.json", "raw_markdown": "/registry/wrangles/select/substring.md", "runtime_symbol": "wrangles.recipe_wrangles.select.substring", @@ -1846,7 +1847,7 @@ "select", "tail" ], - "route": "/wrangles/select/tail", + "route": "/wrangles/namespaces/select#tail", "contract_json": "/registry/contracts/select/tail.json", "raw_markdown": "/registry/wrangles/select/tail.md", "runtime_symbol": "wrangles.recipe_wrangles.select.tail", @@ -1868,7 +1869,7 @@ "select", "threshold" ], - "route": "/wrangles/select/threshold", + "route": "/wrangles/namespaces/select#threshold", "contract_json": "/registry/contracts/select/threshold.json", "raw_markdown": "/registry/wrangles/select/threshold.md", "runtime_symbol": "wrangles.recipe_wrangles.select.threshold", @@ -1890,7 +1891,7 @@ "compare", "similarity" ], - "route": "/wrangles/similarity", + "route": "/wrangles/namespaces/compare#similarity", "contract_json": "/registry/contracts/similarity.json", "raw_markdown": "/registry/wrangles/_root/similarity.md", "runtime_symbol": "wrangles.recipe_wrangles.main.similarity", @@ -1912,7 +1913,7 @@ "select", "sort" ], - "route": "/wrangles/sort", + "route": "/wrangles/namespaces/select#sort", "contract_json": "/registry/contracts/sort.json", "raw_markdown": "/registry/wrangles/_root/sort.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.sort", @@ -1934,7 +1935,7 @@ "split", "dictionary" ], - "route": "/wrangles/split/dictionary", + "route": "/wrangles/namespaces/split#dictionary", "contract_json": "/registry/contracts/split/dictionary.json", "raw_markdown": "/registry/wrangles/split/dictionary.md", "runtime_symbol": "wrangles.recipe_wrangles.split.dictionary", @@ -1956,7 +1957,7 @@ "split", "list" ], - "route": "/wrangles/split/list", + "route": "/wrangles/namespaces/split#list", "contract_json": "/registry/contracts/split/list.json", "raw_markdown": "/registry/wrangles/split/list.md", "runtime_symbol": "wrangles.recipe_wrangles.split.list", @@ -1978,7 +1979,7 @@ "split", "text" ], - "route": "/wrangles/split/text", + "route": "/wrangles/namespaces/split#text", "contract_json": "/registry/contracts/split/text.json", "raw_markdown": "/registry/wrangles/split/text.md", "runtime_symbol": "wrangles.recipe_wrangles.split.text", @@ -2000,7 +2001,7 @@ "split", "tokenize" ], - "route": "/wrangles/split/tokenize", + "route": "/wrangles/namespaces/split#tokenize", "contract_json": "/registry/contracts/split/tokenize.json", "raw_markdown": "/registry/wrangles/split/tokenize.md", "runtime_symbol": "wrangles.recipe_wrangles.split.tokenize", @@ -2022,7 +2023,7 @@ "compute", "sql" ], - "route": "/wrangles/sql", + "route": "/wrangles/namespaces/compute#sql", "contract_json": "/registry/contracts/sql.json", "raw_markdown": "/registry/wrangles/_root/sql.md", "runtime_symbol": "wrangles.recipe_wrangles.main.sql", @@ -2038,13 +2039,14 @@ "aliases": [], "slug": "standardize", "title": "Standardize", - "description": "Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription.", - "status": "active", + "description": "Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model.", + "status": "deprecated", + "replaced_by": "standardize.custom", "tags": [ "format", "standardize" ], - "route": "/wrangles/standardize", + "route": "/wrangles/namespaces/format#standardize", "contract_json": "/registry/contracts/standardize.json", "raw_markdown": "/registry/wrangles/_root/standardize.md", "runtime_symbol": "wrangles.recipe_wrangles.main.standardize", @@ -2066,7 +2068,7 @@ "standardize", "clean" ], - "route": "/wrangles/standardize/clean", + "route": "/wrangles/namespaces/standardize#clean", "contract_json": "/registry/contracts/standardize/clean.json", "raw_markdown": "/registry/wrangles/standardize/clean.md", "runtime_symbol": "wrangles.recipe_wrangles.standardize.clean", @@ -2088,7 +2090,7 @@ "standardize", "custom" ], - "route": "/wrangles/standardize/custom", + "route": "/wrangles/namespaces/standardize#custom", "contract_json": "/registry/contracts/standardize/custom.json", "raw_markdown": "/registry/wrangles/standardize/custom.md", "runtime_symbol": "wrangles.recipe_wrangles.standardize.custom", @@ -2110,7 +2112,7 @@ "format", "translate" ], - "route": "/wrangles/translate", + "route": "/wrangles/namespaces/format#translate", "contract_json": "/registry/contracts/translate.json", "raw_markdown": "/registry/wrangles/_root/translate.md", "runtime_symbol": "wrangles.recipe_wrangles.main.translate", @@ -2132,7 +2134,7 @@ "transform", "transpose" ], - "route": "/wrangles/transpose", + "route": "/wrangles/namespaces/transform#transpose", "contract_json": "/registry/contracts/transpose.json", "raw_markdown": "/registry/wrangles/_root/transpose.md", "runtime_symbol": "wrangles.recipe_wrangles.pandas.transpose", @@ -2154,7 +2156,7 @@ "utility", "try" ], - "route": "/wrangles/try", + "route": "/wrangles/namespaces/utility#try", "contract_json": "/registry/contracts/try.json", "raw_markdown": "/registry/wrangles/_root/try.md", "runtime_symbol": "wrangles.recipe_wrangles.main.Try", @@ -2164,7 +2166,7 @@ ], "artifacts": { "index": "/registry/index.md", - "recipe_schema": "/schemas/recipes/pilot/schema.json", + "recipe_schema": "/schemas/recipes/registry/schema.json", "entry_schema": "/registry/schema/wrangle-entry.schema.json", "runtime_manifest": "/registry/runtime/wranglespy.json", "runtime_manifest_schema": "/registry/schema/wrangles-runtime-manifest.schema.json" diff --git a/wrangles-docs/static/registry/schema/wrangle-entry.schema.json b/wrangles-docs/static/registry/schema/wrangle-entry.schema.json index 9ded481..8aafdd4 100644 --- a/wrangles-docs/static/registry/schema/wrangle-entry.schema.json +++ b/wrangles-docs/static/registry/schema/wrangle-entry.schema.json @@ -84,6 +84,11 @@ "removed" ] }, + "replaced_by": { + "type": "string", + "pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)*$", + "description": "Canonical wrangle_key that replaces a deprecated or removed entry." + }, "visibility": { "enum": [ "public", @@ -174,6 +179,25 @@ } } }, + "allOf": [ + { + "if": { + "properties": { + "status": { + "const": "deprecated" + } + }, + "required": [ + "status" + ] + }, + "then": { + "required": [ + "replaced_by" + ] + } + } + ], "$defs": { "parameter": { "type": "object", @@ -182,7 +206,7 @@ "name", "description", "required", - "role", + "param_group", "schema" ], "properties": { @@ -202,9 +226,17 @@ "required": { "type": "boolean" }, - "role": { + "param_group": { "type": "string", - "minLength": 1 + "enum": [ + "I/O", + "Options", + "Formatting", + "Conditions", + "Execution", + "Errors", + "Details" + ] }, "runtime_default": {}, "schema": { diff --git a/wrangles-docs/static/registry/wrangles/_root/accordion.md b/wrangles-docs/static/registry/wrangles/_root/accordion.md index 73e57e8..163025b 100644 --- a/wrangles-docs/static/registry/wrangles/_root/accordion.md +++ b/wrangles-docs/static/registry/wrangles/_root/accordion.md @@ -33,7 +33,7 @@ parameters: - name: wrangles description: List of wrangles to apply. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -42,7 +42,7 @@ parameters: - name: input description: The column(s) containing the list(s) that the wrangles will be applied to the elements of. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -51,7 +51,7 @@ parameters: - name: output description: Output of the wrangles to save back to the dataframe. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -64,7 +64,7 @@ parameters: not specified, all columns will be propogated. This may be useful to limit the memory use for large datasets. required: false - role: option + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/batch.md b/wrangles-docs/static/registry/wrangles/_root/batch.md index f17bbec..7794310 100644 --- a/wrangles-docs/static/registry/wrangles/_root/batch.md +++ b/wrangles-docs/static/registry/wrangles/_root/batch.md @@ -35,7 +35,7 @@ parameters: The wrangles to execute on the data. Each series of wrangles will be run against the data in batches of the size defined by batch_size. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -44,14 +44,14 @@ parameters: - name: batch_size description: The number of rows to split each batch into. required: false - role: option + param_group: Execution runtime_default: 1000 schema: type: integer - name: threads description: The number of threads to use for parallel processing. Default 1. required: false - role: option + param_group: Execution runtime_default: 1 schema: type: integer @@ -60,7 +60,7 @@ parameters: A dictionary of column_name: value to return if an error occurs while attempting to run a batch. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: @@ -69,7 +69,7 @@ parameters: - name: timeout description: The number of seconds to wait for a batch to complete before raising an error. required: false - role: option + param_group: Execution runtime_default: null schema: type: @@ -78,7 +78,7 @@ parameters: - name: use_multiprocessing description: Use process-based workers instead of threads. This is an advanced runtime option. required: false - role: option + param_group: Execution runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/classify.md b/wrangles-docs/static/registry/wrangles/_root/classify.md index 9dffa3c..498ab61 100644 --- a/wrangles-docs/static/registry/wrangles/_root/classify.md +++ b/wrangles-docs/static/registry/wrangles/_root/classify.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,13 +48,13 @@ parameters: - name: model_id description: ID of the classification model to be used. required: true - role: model-reference + param_group: Details schema: type: string - name: include_confidence description: For models that support it, include the confidence level in the output. required: false - role: option + param_group: Formatting schema: type: boolean examples: [] diff --git a/wrangles-docs/static/registry/wrangles/_root/clean-whitespaces.md b/wrangles-docs/static/registry/wrangles/_root/clean-whitespaces.md index 5be4c43..0a76469 100644 --- a/wrangles-docs/static/registry/wrangles/_root/clean-whitespaces.md +++ b/wrangles-docs/static/registry/wrangles/_root/clean-whitespaces.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,14 +52,14 @@ parameters: - name: trim description: Whether to trim leading and trailing spaces. Default True. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: remove_literals description: Whether to remove special space characters such as new lines etc. Default True. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/concurrent.md b/wrangles-docs/static/registry/wrangles/_root/concurrent.md index 05a1f18..ce5187f 100644 --- a/wrangles-docs/static/registry/wrangles/_root/concurrent.md +++ b/wrangles-docs/static/registry/wrangles/_root/concurrent.md @@ -34,7 +34,7 @@ parameters: - name: wrangles description: The wrangles section of a recipe to execute for each combination of variables. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -43,7 +43,7 @@ parameters: - name: max_concurrency description: The maximum number of wrangles to execute in parallel. required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer @@ -51,7 +51,7 @@ parameters: - name: use_multiprocessing description: Use process-based workers instead of threads. This is an advanced runtime option. required: false - role: option + param_group: Execution runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/copy.md b/wrangles-docs/static/registry/wrangles/_root/copy.md index f9d8764..34c320a 100644 --- a/wrangles-docs/static/registry/wrangles/_root/copy.md +++ b/wrangles-docs/static/registry/wrangles/_root/copy.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input columns or columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output columns or columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/date-calculator.md b/wrangles-docs/static/registry/wrangles/_root/date-calculator.md index d0d7987..ba25a44 100644 --- a/wrangles-docs/static/registry/wrangles/_root/date-calculator.md +++ b/wrangles-docs/static/registry/wrangles/_root/date-calculator.md @@ -31,13 +31,13 @@ parameters: - name: input description: Name of the dates column. required: true - role: column-selector + param_group: I/O schema: type: string - name: operation description: Date operation. required: false - role: option + param_group: Options runtime_default: add schema: type: string @@ -47,7 +47,7 @@ parameters: - name: output description: Name of the output column of dates. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -56,7 +56,7 @@ parameters: - name: time_unit description: Time unit for operation. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -74,7 +74,7 @@ parameters: - name: time_value description: Time unit value for operation. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/drop.md b/wrangles-docs/static/registry/wrangles/_root/drop.md index f27d9ec..f24ac35 100644 --- a/wrangles-docs/static/registry/wrangles/_root/drop.md +++ b/wrangles-docs/static/registry/wrangles/_root/drop.md @@ -31,7 +31,7 @@ parameters: - name: columns description: Name of the column(s) to drop. required: true - role: option + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/_root/explode.md b/wrangles-docs/static/registry/wrangles/_root/explode.md index f743d1c..2b18fc3 100644 --- a/wrangles-docs/static/registry/wrangles/_root/explode.md +++ b/wrangles-docs/static/registry/wrangles/_root/explode.md @@ -33,7 +33,7 @@ parameters: Name of the column(s) to explode. If multiple columns are included they must contain lists of the same length. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: reset_index description: Reset the index after exploding. Default True. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean @@ -51,7 +51,7 @@ parameters: If true, any rows that contain an empty list will be dropped. If false, rows that contain empty lists will keep 1 row with an empty value. Default False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/filter.md b/wrangles-docs/static/registry/wrangles/_root/filter.md index 453b311..d95698d 100644 --- a/wrangles-docs/static/registry/wrangles/_root/filter.md +++ b/wrangles-docs/static/registry/wrangles/_root/filter.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column to filter on. If multiple are provided, all must match the criteria. required: false - role: column-selector + param_group: I/O runtime_default: [] schema: type: @@ -43,7 +43,7 @@ parameters: - name: equal description: Select rows where the values equal a given value. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: not_equal description: Select rows where the values do not equal a given value. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -63,7 +63,7 @@ parameters: - name: is_in description: Select rows where the values are in a given list. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -73,7 +73,7 @@ parameters: - name: not_in description: Select rows where the values are not in a given list. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -85,7 +85,7 @@ parameters: Select rows where the values are greater than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -97,7 +97,7 @@ parameters: Select rows where the values are greater than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -109,7 +109,7 @@ parameters: Select rows where the values are less than a specified value. Does not include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -119,7 +119,7 @@ parameters: - name: less_than_equal_to description: Select rows where the values are less than a specified value. Does include the value itself. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -129,7 +129,7 @@ parameters: - name: between description: Value or list of values to filter that are in between two parameter values. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -138,7 +138,7 @@ parameters: - name: contains description: Select rows where the input contains the value. Allows regular expressions. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -147,7 +147,7 @@ parameters: - name: not_contains description: Select rows where the input does not contain the value. Allows regular expressions. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -156,7 +156,7 @@ parameters: - name: is_null description: If true, select all rows where the value is NULL. If false, where is not NULL. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/huggingface.md b/wrangles-docs/static/registry/wrangles/_root/huggingface.md index f684cc6..1d058ce 100644 --- a/wrangles-docs/static/registry/wrangles/_root/huggingface.md +++ b/wrangles-docs/static/registry/wrangles/_root/huggingface.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,19 +40,19 @@ parameters: - name: api_token description: Huggingface API Token. required: true - role: option + param_group: Details schema: type: string - name: model description: Name of the model to use. e.g. facebook/bart-large-cnn. required: true - role: option + param_group: Details schema: type: string - name: output description: Name of the output column. If not provided, will overwrite the input column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -62,7 +62,7 @@ parameters: - name: parameters description: Optionally, provide additional parameters to define the model behaviour. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/log.md b/wrangles-docs/static/registry/wrangles/_root/log.md index de9c3af..55f78ab 100644 --- a/wrangles-docs/static/registry/wrangles/_root/log.md +++ b/wrangles-docs/static/registry/wrangles/_root/log.md @@ -31,7 +31,7 @@ parameters: - name: columns description: (Optional, default all columns) List of specific columns to log. required: false - role: option + param_group: I/O runtime_default: null schema: type: @@ -40,7 +40,7 @@ parameters: - name: write description: (Optional) Allows for an intermediate output to a file/dataframe/database etc. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: warning description: Log a warning to the console. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -61,7 +61,7 @@ parameters: - name: info description: Log info to the console. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: Whether to log a sample of the contents of the dataframe. Default True if not logging to a write, error, warning or info. Default False otherwise. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -81,7 +81,7 @@ parameters: - name: error description: Log an error to the console. required: false - role: option + param_group: Options schema: type: string examples: [] diff --git a/wrangles-docs/static/registry/wrangles/_root/lookup.md b/wrangles-docs/static/registry/wrangles/_root/lookup.md index d219a3d..13652cb 100644 --- a/wrangles-docs/static/registry/wrangles/_root/lookup.md +++ b/wrangles-docs/static/registry/wrangles/_root/lookup.md @@ -30,7 +30,7 @@ parameters: - name: input description: Name of the column(s) to lookup. required: true - role: column-selector + param_group: I/O schema: type: string - name: output @@ -39,7 +39,7 @@ parameters: output column receives the corresponding match. A single output containing a wildcard (*) is expanded into n columns, e.g. "Top *" with n: 3 becomes "Top 1", "Top 2", "Top 3". required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -49,7 +49,7 @@ parameters: - name: model_id description: The model_id to use lookup against. required: false - role: model-reference + param_group: Details runtime_default: null schema: type: @@ -61,7 +61,7 @@ parameters: lookup unique values once, copy results to all rows. 'by_matrix': lookup once per matrix permutation. required: false - role: option + param_group: Options runtime_default: by_row schema: type: string @@ -75,7 +75,7 @@ parameters: column receives the corresponding match. Otherwise all n matches are stored as a list in each output column. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/math.md b/wrangles-docs/static/registry/wrangles/_root/math.md index 14bb1de..9e6df49 100644 --- a/wrangles-docs/static/registry/wrangles/_root/math.md +++ b/wrangles-docs/static/registry/wrangles/_root/math.md @@ -33,13 +33,13 @@ parameters: The mathematical expression using column names. e.g. column1 * column2 + column3. Note: spaces within column names are replaced by underscores (_). required: true - role: column-selector + param_group: I/O schema: type: string - name: output description: The column to output the results to. required: true - role: column-output + param_group: I/O schema: type: string examples: [] diff --git a/wrangles-docs/static/registry/wrangles/_root/maths.md b/wrangles-docs/static/registry/wrangles/_root/maths.md index 320743b..188e429 100644 --- a/wrangles-docs/static/registry/wrangles/_root/maths.md +++ b/wrangles-docs/static/registry/wrangles/_root/maths.md @@ -10,6 +10,7 @@ wrangle_key: maths aliases: [] slug: maths status: deprecated +replaced_by: math visibility: public tags: - maths @@ -30,13 +31,13 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: string - name: output description: Name or list of output columns. required: true - role: column-output + param_group: I/O schema: type: string examples: [] @@ -53,4 +54,4 @@ sources: Deprecated alias for `math`; evaluate an expression and write its result to an output column. -This first-pass guidance is derived from the callable signature. +This guidance was derived from the callable signature. diff --git a/wrangles-docs/static/registry/wrangles/_root/matrix.md b/wrangles-docs/static/registry/wrangles/_root/matrix.md index efe654b..5394e36 100644 --- a/wrangles-docs/static/registry/wrangles/_root/matrix.md +++ b/wrangles-docs/static/registry/wrangles/_root/matrix.md @@ -35,7 +35,7 @@ parameters: A dictionary of variables to pass to the wrangle. The key is the variable name and the value is a list of values. required: true - role: variables + param_group: Execution schema: type: object - name: wrangles @@ -43,7 +43,7 @@ parameters: The wrangles to apply to the dataframe. Each wrangle will be run for each combination of the variables. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -55,7 +55,7 @@ parameters: set of variables, repeating shorter lists until the longest is completed. permutations uses the combination of all variables against all other variables. required: false - role: option + param_group: Options runtime_default: loop schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/_root/python.md b/wrangles-docs/static/registry/wrangles/_root/python.md index 8e4ce10..d6fc9a1 100644 --- a/wrangles-docs/static/registry/wrangles/_root/python.md +++ b/wrangles-docs/static/registry/wrangles/_root/python.md @@ -39,7 +39,7 @@ parameters: Python command. This must return a value. Note: any non-alphanumeric characters in variable names are replaced by underscores (_). required: true - role: option + param_group: Options schema: type: string - name: output @@ -47,7 +47,7 @@ parameters: Name or list of output column(s). To output multiple columns, return a list of the corresponding length. required: true - role: column-output + param_group: I/O schema: type: - string @@ -58,7 +58,7 @@ parameters: conjunction with kwargs to target a variable range of columns. If not specified, all columns will be available. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: exception will be raised as normal. If multiple output columns are specified, this must match the length. required: false - role: option + param_group: Errors schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/_root/recipe.md b/wrangles-docs/static/registry/wrangles/_root/recipe.md index 296f36f..8e794a1 100644 --- a/wrangles-docs/static/registry/wrangles/_root/recipe.md +++ b/wrangles-docs/static/registry/wrangles/_root/recipe.md @@ -30,7 +30,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -41,7 +41,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -51,7 +51,7 @@ parameters: - name: name description: File name of the recipe. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: - name: variables description: A dictionary of variables to pass to the recipe. required: false - role: variables + param_group: Execution runtime_default: null schema: type: @@ -80,4 +80,4 @@ sources: Run another recipe as a wrangle against the current dataframe. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/wrangles-docs/static/registry/wrangles/_root/reindex.md b/wrangles-docs/static/registry/wrangles/_root/reindex.md index 4191c70..0a0fff9 100644 --- a/wrangles-docs/static/registry/wrangles/_root/reindex.md +++ b/wrangles-docs/static/registry/wrangles/_root/reindex.md @@ -31,7 +31,7 @@ parameters: - name: labels description: New labels / index to conform the axis specified by ‘axis’ to. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -40,7 +40,7 @@ parameters: - name: index description: New labels for the index. Preferably an Index object to avoid duplicating data. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -49,7 +49,7 @@ parameters: - name: columns description: New labels for the columns. Preferably an Index object to avoid duplicating data. required: false - role: option + param_group: I/O runtime_default: null schema: type: @@ -58,7 +58,7 @@ parameters: - name: axis description: Axis to target. Can be either the axis name (‘index’, ‘columns’) or number (0, 1). required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/remove-words.md b/wrangles-docs/static/registry/wrangles/_root/remove-words.md index d84baf3..d93dc0c 100644 --- a/wrangles-docs/static/registry/wrangles/_root/remove-words.md +++ b/wrangles-docs/static/registry/wrangles/_root/remove-words.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of column to remove words from. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: to_remove description: Column or list of columns with a list of words to be removed. required: true - role: option + param_group: I/O schema: type: string - name: output description: Name of the output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -56,14 +56,14 @@ parameters: - name: tokenize_to_remove description: Tokenize all to_remove inputs. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore input and to_remove case. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/rename.md b/wrangles-docs/static/registry/wrangles/_root/rename.md index 882075d..3235df0 100644 --- a/wrangles-docs/static/registry/wrangles/_root/rename.md +++ b/wrangles-docs/static/registry/wrangles/_root/rename.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -55,7 +55,7 @@ parameters: must also include the column named 'columns'. This can only be used instead of the standard rename. required: false - role: nested-wrangles + param_group: Execution runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/replace.md b/wrangles-docs/static/registry/wrangles/_root/replace.md index 452a7fa..53a8f31 100644 --- a/wrangles-docs/static/registry/wrangles/_root/replace.md +++ b/wrangles-docs/static/registry/wrangles/_root/replace.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,19 +42,19 @@ parameters: - name: find description: Pattern to find using regex. required: true - role: option + param_group: Options schema: type: string - name: replace description: Value to replace the pattern found. required: true - role: option + param_group: Options schema: type: string - name: output description: Name or list of output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/round.md b/wrangles-docs/static/registry/wrangles/_root/round.md index 9f29e10..a2aca9d 100644 --- a/wrangles-docs/static/registry/wrangles/_root/round.md +++ b/wrangles-docs/static/registry/wrangles/_root/round.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: decimals description: Number of decimal places to round column. required: false - role: option + param_group: Formatting runtime_default: 0 schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/similarity.md b/wrangles-docs/static/registry/wrangles/_root/similarity.md index 03f5f5e..7502f6f 100644 --- a/wrangles-docs/static/registry/wrangles/_root/similarity.md +++ b/wrangles-docs/static/registry/wrangles/_root/similarity.md @@ -31,7 +31,7 @@ parameters: - name: input description: Two columns of vectors to compare the similarity of. required: true - role: column-selector + param_group: I/O schema: type: array minItems: 2 @@ -39,7 +39,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: method @@ -47,7 +47,7 @@ parameters: The type of similarity to calculate (cosine or euclidean). Adjusted cosine adjusts the default cosine calculation to cover a range of 0-1 for typical comparisons. required: false - role: option + param_group: Options runtime_default: cosine schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/_root/sort.md b/wrangles-docs/static/registry/wrangles/_root/sort.md index 211ec33..7f53055 100644 --- a/wrangles-docs/static/registry/wrangles/_root/sort.md +++ b/wrangles-docs/static/registry/wrangles/_root/sort.md @@ -31,14 +31,14 @@ parameters: - name: ignore_index description: Ignore Index value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: by description: Name or list of the column(s) to sort by. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: Sort ascending vs. descending. Specify a list to sort multiple columns in different orders. If this is a list of bools then it must match the length of the by. required: false - role: option + param_group: Options schema: type: - boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/sql.md b/wrangles-docs/static/registry/wrangles/_root/sql.md index 4c3178d..c75242f 100644 --- a/wrangles-docs/static/registry/wrangles/_root/sql.md +++ b/wrangles-docs/static/registry/wrangles/_root/sql.md @@ -33,7 +33,7 @@ parameters: - name: command description: SQL Command. The table is called df. For specific SQL syntax, this uses the SQLite dialect. required: true - role: option + param_group: Options schema: type: string - name: params @@ -41,7 +41,7 @@ parameters: Variables to use in conjunctions with query. This allows the query to be parameterized. This uses sqlite syntax (? or :name). required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -51,14 +51,14 @@ parameters: - name: preserve_index description: Preserve Index value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: preserve_data_types description: Preserve Data Types value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/standardize.md b/wrangles-docs/static/registry/wrangles/_root/standardize.md index e414d26..a6b4555 100644 --- a/wrangles-docs/static/registry/wrangles/_root/standardize.md +++ b/wrangles-docs/static/registry/wrangles/_root/standardize.md @@ -6,12 +6,13 @@ wrangle_name: standardize namespace: null title: Standardize description: >- - Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and - Subscription. + Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY + or bespoke model. wrangle_key: standardize aliases: [] slug: standardize -status: active +status: deprecated +replaced_by: standardize.custom visibility: public tags: - format @@ -33,7 +34,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +43,7 @@ parameters: - name: model_id description: The ID of the wrangle to use (do not include 'find' and 'replace'). required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +51,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +61,7 @@ parameters: - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/_root/translate.md b/wrangles-docs/static/registry/wrangles/_root/translate.md index ddba2cc..2339235 100644 --- a/wrangles-docs/static/registry/wrangles/_root/translate.md +++ b/wrangles-docs/static/registry/wrangles/_root/translate.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column to translate. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: target_language description: Code of the language to translate to. required: true - role: option + param_group: Options schema: type: string enum: @@ -83,7 +83,7 @@ parameters: - name: source_language description: Code of the language to translate from. If omitted, automatically detects the input language. required: false - role: option + param_group: Options runtime_default: AUTO schema: type: string @@ -116,7 +116,7 @@ parameters: - name: case description: Allow changing the case of the input prior to translation. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/transpose.md b/wrangles-docs/static/registry/wrangles/_root/transpose.md index b9fa957..f116141 100644 --- a/wrangles-docs/static/registry/wrangles/_root/transpose.md +++ b/wrangles-docs/static/registry/wrangles/_root/transpose.md @@ -33,7 +33,7 @@ parameters: Name or position of the column that will be used as the column headings for the transposed DataFrame. Default 0 (first column). Use header_column = null to not use any column as header. required: false - role: column-selector + param_group: I/O runtime_default: 0 schema: type: diff --git a/wrangles-docs/static/registry/wrangles/_root/try.md b/wrangles-docs/static/registry/wrangles/_root/try.md index 34daeb8..ce46640 100644 --- a/wrangles-docs/static/registry/wrangles/_root/try.md +++ b/wrangles-docs/static/registry/wrangles/_root/try.md @@ -31,7 +31,7 @@ parameters: - name: wrangles description: List of wrangles to apply. required: true - role: nested-wrangles + param_group: Execution schema: type: array minItems: 1 @@ -40,7 +40,7 @@ parameters: - name: retries description: Number of times to retry the wrangles if an error occurs. Default 0. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer @@ -51,7 +51,7 @@ parameters: dictionary of column names and values. If except is not provided, the error will be logged and the recipe will continue. required: false - role: option + param_group: Errors schema: type: object minItems: 1 diff --git a/wrangles-docs/static/registry/wrangles/compare/lists.md b/wrangles-docs/static/registry/wrangles/compare/lists.md index 8e7f258..78ec5c4 100644 --- a/wrangles-docs/static/registry/wrangles/compare/lists.md +++ b/wrangles-docs/static/registry/wrangles/compare/lists.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns containing lists to compare. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: method description: Type of comparison to perform. required: false - role: option + param_group: Options runtime_default: intersection schema: type: string @@ -54,14 +54,14 @@ parameters: - name: remove_duplicates description: Remove duplicates from the result. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore case when comparing string items. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/compare/text.md b/wrangles-docs/static/registry/wrangles/compare/text.md index c726f75..1dc9781 100644 --- a/wrangles-docs/static/registry/wrangles/compare/text.md +++ b/wrangles-docs/static/registry/wrangles/compare/text.md @@ -33,7 +33,7 @@ parameters: - name: input description: The columns to compare. First column is the base column. required: true - role: column-selector + param_group: I/O schema: type: array - name: output @@ -42,7 +42,7 @@ parameters: ratio_column] when method is overlap and include_ratio is true; otherwise a single column name. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: method description: The type of comparison to perform (difference, intersection, overlap, similarity). required: false - role: option + param_group: Options runtime_default: difference schema: type: string @@ -62,35 +62,35 @@ parameters: - name: char description: Character to split strings on for difference and intersection. Defaults to a space. required: false - role: option + param_group: Options runtime_default: ' ' schema: type: string - name: non_match_char description: Character to use for non-matching characters when using overlap. required: false - role: option + param_group: Formatting runtime_default: '*' schema: type: string - name: include_ratio description: Include the ratio of matching characters when using overlap. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: decimal_places description: Number of decimal places to round the overlap ratio to. required: false - role: option + param_group: Formatting runtime_default: 3 schema: type: integer - name: exact_match description: Value to use for exact matches when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -99,7 +99,7 @@ parameters: - name: empty_a description: Value to use when input A is empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -108,7 +108,7 @@ parameters: - name: empty_b description: Value to use when input B is empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -117,7 +117,7 @@ parameters: - name: all_empty description: Value to use when both inputs are empty when using overlap. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -126,14 +126,14 @@ parameters: - name: case_sensitive description: Whether the comparison is case sensitive. Defaults to true. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: metric description: Metric value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: token_sort schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/compute/case-when.md b/wrangles-docs/static/registry/wrangles/compute/case-when.md index c93bf26..d0be1cf 100644 --- a/wrangles-docs/static/registry/wrangles/compute/case-when.md +++ b/wrangles-docs/static/registry/wrangles/compute/case-when.md @@ -31,13 +31,13 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: cases description: List of conditions and corresponding values. required: true - role: option + param_group: Options schema: type: array minItems: 1 @@ -58,7 +58,7 @@ parameters: - name: default description: Value to assign if no conditions are met. Default None. required: false - role: fallback-value + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/compute/score-search-results.md b/wrangles-docs/static/registry/wrangles/compute/score-search-results.md index 68dbc5d..e730a00 100644 --- a/wrangles-docs/static/registry/wrangles/compute/score-search-results.md +++ b/wrangles-docs/static/registry/wrangles/compute/score-search-results.md @@ -35,7 +35,7 @@ parameters: List of 3 to 5 columns -> [results, suppliers, part_codes, mpns (optional), descriptions (optional)]. required: true - role: column-selector + param_group: I/O schema: type: array - name: output @@ -43,7 +43,7 @@ parameters: Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. required: true - role: column-output + param_group: I/O schema: type: - string @@ -51,98 +51,98 @@ parameters: - name: must_match_part_code description: If true, filters out results that don't satisfy the allowed match types. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_mpn_exact description: Treat exact MPN matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_mpn_partial description: Treat partial MPN matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_other_exact description: Treat exact other part code matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: allow_other_partial description: Treat partial other part code matches as valid part code matches. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: blacklist_keywords description: Comma-separated list or array of keywords to filter out URLs containing them. required: false - role: option + param_group: Options runtime_default: '' schema: type: string - name: mpn_exact_score description: Mpn Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 8 schema: type: number - name: mpn_partial_base description: Mpn Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 4 schema: type: number - name: part_code_exact_score description: Part Code Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 6 schema: type: number - name: part_code_partial_base description: Part Code Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 2 schema: type: number - name: supplier_exact_score description: Supplier Exact Score value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 3 schema: type: number - name: supplier_partial_base description: Supplier Partial Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 1 schema: type: number - name: context_match_base description: Context Match Base value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 2 schema: type: number - name: fuzzy_match_threshold description: Fuzzy Match Threshold value accepted by the runtime. required: false - role: option + param_group: Options runtime_default: 0.8 schema: type: number diff --git a/wrangles-docs/static/registry/wrangles/convert/case.md b/wrangles-docs/static/registry/wrangles/convert/case.md index 2709ff6..986490c 100644 --- a/wrangles-docs/static/registry/wrangles/convert/case.md +++ b/wrangles-docs/static/registry/wrangles/convert/case.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: case description: Letter case to apply. Sentence case lowercases the value and capitalizes sentence starts. required: false - role: option + param_group: Options runtime_default: lower schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/convert/data-type.md b/wrangles-docs/static/registry/wrangles/convert/data-type.md index c74839c..51543fe 100644 --- a/wrangles-docs/static/registry/wrangles/convert/data-type.md +++ b/wrangles-docs/static/registry/wrangles/convert/data-type.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: data_type description: Data type to produce. required: false - role: option + param_group: Options runtime_default: str schema: type: string @@ -57,7 +57,7 @@ parameters: - name: default description: Value returned when conversion fails. If omitted, the original value is retained. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: [string, number, array, object, boolean, "null"] diff --git a/wrangles-docs/static/registry/wrangles/convert/fraction-to-decimal.md b/wrangles-docs/static/registry/wrangles/convert/fraction-to-decimal.md index 69ef4a5..74c50db 100644 --- a/wrangles-docs/static/registry/wrangles/convert/fraction-to-decimal.md +++ b/wrangles-docs/static/registry/wrangles/convert/fraction-to-decimal.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: decimals description: Number of decimals to round fraction. required: false - role: option + param_group: Formatting runtime_default: 4 schema: type: integer - name: output description: Name of the output colum. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/convert/from-json.md b/wrangles-docs/static/registry/wrangles/convert/from-json.md index bed4ad6..3e55d3b 100644 --- a/wrangles-docs/static/registry/wrangles/convert/from-json.md +++ b/wrangles-docs/static/registry/wrangles/convert/from-json.md @@ -32,7 +32,7 @@ parameters: - name: input description: Name, index, or list of columns containing valid JSON text. required: true - role: column-selector + param_group: I/O schema: type: [string, integer, array] items: @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. If omitted, each input column is overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: [string, array] @@ -49,7 +49,7 @@ parameters: - name: default description: Value returned for empty or invalid JSON. A list may supply one fallback per input column. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: [string, number, array, object, boolean, "null"] diff --git a/wrangles-docs/static/registry/wrangles/convert/from-yaml.md b/wrangles-docs/static/registry/wrangles/convert/from-yaml.md index 8e56142..a8370a7 100644 --- a/wrangles-docs/static/registry/wrangles/convert/from-yaml.md +++ b/wrangles-docs/static/registry/wrangles/convert/from-yaml.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: may also be a list - either a single value to apply to all columns, or one value per input column. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/convert/to-json.md b/wrangles-docs/static/registry/wrangles/convert/to-json.md index 856dc5c..c9d4e5e 100644 --- a/wrangles-docs/static/registry/wrangles/convert/to-json.md +++ b/wrangles-docs/static/registry/wrangles/convert/to-json.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,7 +50,7 @@ parameters: - name: ensure_ascii description: If true, non-ASCII characters will be escaped. Default is false. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean @@ -62,7 +62,7 @@ parameters: integer indent indents that many spaces per level. If indent is a string (such as '\t'), that string is used to indent each level. required: false - role: option + param_group: Formatting schema: type: - string @@ -72,7 +72,7 @@ parameters: If sort_keys is true (defaults to False), then the output of dictionaries will be sorted by key. required: false - role: option + param_group: Formatting schema: type: boolean examples: [] diff --git a/wrangles-docs/static/registry/wrangles/convert/to-yaml.md b/wrangles-docs/static/registry/wrangles/convert/to-yaml.md index 003a7e9..d837118 100644 --- a/wrangles-docs/static/registry/wrangles/convert/to-yaml.md +++ b/wrangles-docs/static/registry/wrangles/convert/to-yaml.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. If omitted, the input column will be overwritten. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,21 +50,21 @@ parameters: - name: sort_keys description: 'If sort_keys is true (default: False), then the output of dictionaries will be sorted by key.' required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: allow_unicode description: Allow Unicode value accepted by the runtime. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: indent description: Specify the number of spaces for indentation to specify nested elements. required: false - role: option + param_group: Formatting schema: type: integer examples: [] diff --git a/wrangles-docs/static/registry/wrangles/create/bins.md b/wrangles-docs/static/registry/wrangles/create/bins.md index d4fce38..cd1bd69 100644 --- a/wrangles-docs/static/registry/wrangles/create/bins.md +++ b/wrangles-docs/static/registry/wrangles/create/bins.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of new column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: bins description: Defines the number of equal-width bins in the range. required: true - role: option + param_group: Options schema: type: - integer @@ -56,7 +56,7 @@ parameters: - name: labels description: Labels for the returned bins. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/create/column.md b/wrangles-docs/static/registry/wrangles/create/column.md index 396dc5c..bb0677b 100644 --- a/wrangles-docs/static/registry/wrangles/create/column.md +++ b/wrangles-docs/static/registry/wrangles/create/column.md @@ -31,7 +31,7 @@ parameters: - name: output description: 'Name or list of names of new columns or column_name: value pairs.' required: true - role: column-output + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: (Optional) Value(s) to add in the new column(s). If using a dictionary in output, value can only be a string. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -57,7 +57,7 @@ parameters: column unchanged. coalesce: fill empty/null cells with the new value, keeping non-null cells. new: overwrite the entire column with the new value. required: false - role: option + param_group: Options runtime_default: existing schema: type: string @@ -72,7 +72,7 @@ parameters: empty/null cells with the new value. new: keep the new value, only fall back to the existing value where the new value is empty/null. required: false - role: option + param_group: Options runtime_default: existing schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/create/embeddings.md b/wrangles-docs/static/registry/wrangles/create/embeddings.md index da187e7..719f6c9 100644 --- a/wrangles-docs/static/registry/wrangles/create/embeddings.md +++ b/wrangles-docs/static/registry/wrangles/create/embeddings.md @@ -31,19 +31,19 @@ parameters: - name: input description: The column of text to create the embeddings for. required: true - role: column-selector + param_group: I/O schema: type: string - name: api_key description: The API key. required: true - role: credential + param_group: Details schema: type: string - name: output description: The output column the embeddings will be saved as. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: batch_size description: The number of rows to submit per individual request. required: false - role: option + param_group: Execution runtime_default: 100 schema: type: integer @@ -61,14 +61,14 @@ parameters: The number of requests to submit in parallel. Each request contains the number of rows set as batch_size. required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer - name: output_type description: Output the embeddings as a numpy array or a python list Default - python list. required: false - role: column-output + param_group: Formatting runtime_default: python list schema: type: string @@ -78,7 +78,7 @@ parameters: - name: model description: The specific model to use to generate the embeddings. required: false - role: option + param_group: Details runtime_default: text-embedding-3-small schema: type: string @@ -87,7 +87,7 @@ parameters: The number of times to retry if the request fails. This will apply exponential backoff to help with rate limiting. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer @@ -97,7 +97,7 @@ parameters: provider. Setting a Jina URL without an explicit provider will automatically use Jina's request/response format. required: false - role: option + param_group: Details runtime_default: https://api.openai.com/v1/embeddings schema: type: string @@ -106,7 +106,7 @@ parameters: The precision of the embeddings. Default is float32. This should be used with output_type numpy array. required: false - role: option + param_group: Formatting runtime_default: float32 schema: type: string @@ -121,7 +121,7 @@ parameters: only when pointing to a custom endpoint that uses a non-default provider's API format (e.g. a Jina-compatible proxy). required: false - role: option + param_group: Details runtime_default: null schema: type: @@ -135,7 +135,7 @@ parameters: The task type for the embedding model. Only applicable for the Jina provider. Selects the appropriate task-specific adapter. required: false - role: option + param_group: Options runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/create/guid.md b/wrangles-docs/static/registry/wrangles/create/guid.md index 59d51d7..960ab60 100644 --- a/wrangles-docs/static/registry/wrangles/create/guid.md +++ b/wrangles-docs/static/registry/wrangles/create/guid.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/create/hash.md b/wrangles-docs/static/registry/wrangles/create/hash.md index 8dd0da6..f575a66 100644 --- a/wrangles-docs/static/registry/wrangles/create/hash.md +++ b/wrangles-docs/static/registry/wrangles/create/hash.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of new column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: method description: 'The method to use to hash the input (Default: md5).' required: false - role: option + param_group: Options runtime_default: md5 schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/create/index.md b/wrangles-docs/static/registry/wrangles/create/index.md index 53f05c1..8fef5b5 100644 --- a/wrangles-docs/static/registry/wrangles/create/index.md +++ b/wrangles-docs/static/registry/wrangles/create/index.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -39,21 +39,21 @@ parameters: - name: start description: (Optional; default 1) Starting number for the index. required: false - role: option + param_group: Options runtime_default: 1 schema: type: integer - name: step description: (Optional; default 1) Step between successive rows. required: false - role: option + param_group: Options runtime_default: 1 schema: type: integer - name: by description: Optional. Cluster the created indexes by one or more columns. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/create/jinja.md b/wrangles-docs/static/registry/wrangles/create/jinja.md index 219cac8..8a3b507 100644 --- a/wrangles-docs/static/registry/wrangles/create/jinja.md +++ b/wrangles-docs/static/registry/wrangles/create/jinja.md @@ -34,7 +34,7 @@ parameters: input. If any keys use a space, they must be replaced with an underscore. Note: spaces within column names are replaced by underscores (_). required: true - role: option + param_group: Options schema: type: object additionalProperties: false @@ -48,7 +48,7 @@ parameters: - name: output description: Name of the column to be output to. required: true - role: column-output + param_group: I/O schema: type: array - name: input @@ -56,7 +56,7 @@ parameters: Specify a name of column containing a dictionary of elements to be used in jinja template. Otherwise, the column headers will be used as keys. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/create/uuid.md b/wrangles-docs/static/registry/wrangles/create/uuid.md index 0e64405..56bc6fe 100644 --- a/wrangles-docs/static/registry/wrangles/create/uuid.md +++ b/wrangles-docs/static/registry/wrangles/create/uuid.md @@ -31,7 +31,7 @@ parameters: - name: output description: Name or list of names of new columns. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/extract/address.md b/wrangles-docs/static/registry/wrangles/extract/address.md index 89b1da8..e8e3c38 100644 --- a/wrangles-docs/static/registry/wrangles/extract/address.md +++ b/wrangles-docs/static/registry/wrangles/extract/address.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: dataType description: Specific part of the address to extract. required: true - role: option + param_group: Options schema: type: string enum: @@ -59,7 +59,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/ai.md b/wrangles-docs/static/registry/wrangles/extract/ai.md index 41ef1a3..e116587 100644 --- a/wrangles-docs/static/registry/wrangles/extract/ai.md +++ b/wrangles-docs/static/registry/wrangles/extract/ai.md @@ -33,7 +33,7 @@ parameters: - name: api_key description: OpenAI API key used for this wrangle, normally supplied through a recipe variable. required: true - role: credential + param_group: Details schema: type: string - name: input @@ -41,7 +41,7 @@ parameters: Input column name, column index, or list of columns supplied together as DATA for each row. If omitted, all dataframe columns are supplied. required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -57,7 +57,7 @@ parameters: for one prompted value, or an array of field names/definitions. Each field may use the schema options below. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -139,7 +139,7 @@ parameters: is also supplied with model_id in a recipe, output names the destination column or columns for the saved fields. required: false - role: model-reference + param_group: Details runtime_default: null schema: type: @@ -153,7 +153,7 @@ parameters: Required non-null nested properties must be supplied. This differs from examples nested under one output field, which teach only that field. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -187,7 +187,7 @@ parameters: How extracted fields are written. columns writes one dataframe column per field (default); dictionary keeps one object; concatenate joins fields into one string using char. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -200,7 +200,7 @@ parameters: - name: char description: Separator used only when output_format is concatenate. Defaults to comma-space. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string @@ -211,7 +211,7 @@ parameters: order, or an empty list when no source was used. This reserved column is automatic. Requires protocol responses. Defaults to false. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean @@ -221,7 +221,7 @@ parameters: priorities, normalization requirements, or other behavior that applies to the complete extraction. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -235,13 +235,13 @@ parameters: OpenAI model ID for this call. If omitted, uses the configured extract.ai default; a saved model definition may supply its own model. required: false - role: option + param_group: Details schema: type: string - name: threads description: Maximum number of row-level requests sent in parallel. The configured default is 32. required: false - role: option + param_group: Execution schema: type: integer minimum: 1 @@ -250,7 +250,7 @@ parameters: Maximum seconds for one HTTP attempt. The configured default is 12; deadline can end the overall call sooner. required: false - role: option + param_group: Execution schema: type: number exclusiveMinimum: 0 @@ -259,7 +259,7 @@ parameters: Number of additional attempts after a retryable failure. The configured default is 1. Backoff and request timeouts remain bounded by deadline. required: false - role: option + param_group: Errors schema: type: integer minimum: 0 @@ -269,13 +269,13 @@ parameters: protocol only when protocol is omitted; new recipes should use the configured Responses endpoint. required: false - role: option + param_group: Details schema: type: string - name: provider description: AI service provider. Currently only OpenAI is supported. required: false - role: option + param_group: Details schema: type: string enum: @@ -285,7 +285,7 @@ parameters: OpenAI API protocol. Responses is the configured default and is required for web_search; chat_completions remains available for legacy definitions. required: false - role: option + param_group: Details schema: type: string enum: @@ -296,14 +296,14 @@ parameters: Total seconds allowed for the entire wrangle call, including queued work, retries, and backoff. The configured default is 15. required: false - role: option + param_group: Execution schema: type: number exclusiveMinimum: 0 - name: store description: Whether OpenAI may store Responses API results. Defaults to false. required: false - role: option + param_group: Details schema: type: boolean - name: cache @@ -311,7 +311,7 @@ parameters: Reuse identical successful results from the bounded warm-instance cache. Defaults to true. Set false when fresh model or web results are required. required: false - role: option + param_group: Details schema: type: boolean - name: cache_ttl @@ -319,7 +319,7 @@ parameters: Maximum age in seconds for a cached result used by this call. Applies to extracted values and web_search_sources together. required: false - role: option + param_group: Details schema: type: number exclusiveMinimum: 0 @@ -329,7 +329,7 @@ parameters: dictionary keys automatically switch to non-strict provider mode and are still validated locally. required: false - role: option + param_group: Details schema: type: boolean - name: reasoning @@ -337,7 +337,7 @@ parameters: Responses API reasoning controls. Set effort for reasoning-capable models. The configured default is none when that model supports it; otherwise the provider default applies. required: false - role: option + param_group: Details schema: type: object properties: @@ -355,7 +355,7 @@ parameters: Responses API text verbosity for compatible models. Defaults to low when supported; ignored with a warning for incompatible models. required: false - role: option + param_group: Details schema: type: string enum: diff --git a/wrangles-docs/static/registry/wrangles/extract/attributes.md b/wrangles-docs/static/registry/wrangles/extract/attributes.md index 3bb2db0..be13e4a 100644 --- a/wrangles-docs/static/registry/wrangles/extract/attributes.md +++ b/wrangles-docs/static/registry/wrangles/extract/attributes.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: responseContent description: Span - returns the text found. object - returns an object with the value and unit. required: false - role: option + param_group: Options runtime_default: span schema: type: string @@ -60,7 +60,7 @@ parameters: - name: attribute_type description: Request only a specific type of attribute. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -95,7 +95,7 @@ parameters: - name: desired_unit description: Convert the extracted unit to the desired unit. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -106,7 +106,7 @@ parameters: When returning an object, if the input is a range (e.g. 10-20mm) set the value to return. min, mid or max. Default mid. required: false - role: option + param_group: Options runtime_default: mid schema: type: string @@ -117,14 +117,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -138,7 +138,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/brackets.md b/wrangles-docs/static/registry/wrangles/extract/brackets.md index 03b9f40..a3e8b6c 100644 --- a/wrangles-docs/static/registry/wrangles/extract/brackets.md +++ b/wrangles-docs/static/registry/wrangles/extract/brackets.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -50,7 +50,7 @@ parameters: (Optional) The type of brackets to find (round '()', square '[]', curly '{}', angled '<>'). Default is all brackets. required: false - role: option + param_group: Options runtime_default: all schema: type: @@ -59,14 +59,14 @@ parameters: - name: include_brackets description: (Optional) Include the brackets in the output. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -79,7 +79,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/codes.md b/wrangles-docs/static/registry/wrangles/extract/codes.md index 7a8fb3b..e293cd9 100644 --- a/wrangles-docs/static/registry/wrangles/extract/codes.md +++ b/wrangles-docs/static/registry/wrangles/extract/codes.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name or list of output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,14 +48,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -68,14 +68,14 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string - name: min_length description: Minimum length of allowed results. required: false - role: option + param_group: Options schema: type: - integer @@ -83,7 +83,7 @@ parameters: - name: max_length description: Maximum length of allowed results. required: false - role: option + param_group: Options schema: type: - integer @@ -94,7 +94,7 @@ parameters: balanced and strict currently apply the same filter. Default is balanced. Unless min_length is provided, minimum lengths default to 3 for lenient, 4 for balanced, and 5 for strict. required: false - role: option + param_group: Options schema: type: string enum: @@ -104,7 +104,7 @@ parameters: - name: sort_order description: Default is input order. Also allows longest or shortest. required: false - role: option + param_group: Formatting schema: type: string enum: @@ -114,13 +114,13 @@ parameters: - name: disallowed_patterns description: A pattern or JSON array of regex patterns to not include in the found codes. required: false - role: option + param_group: Options schema: type: string - name: include_multi_part_tokens description: Whether to include multi-part tokens that have a space. Default True. required: false - role: option + param_group: Options schema: type: boolean - name: extract_raw @@ -128,7 +128,7 @@ parameters: Whether to return tokens with their adjacent non-whitespace characters included, rather than the cleaned token. Default False. required: false - role: option + param_group: Options schema: type: boolean examples: [] diff --git a/wrangles-docs/static/registry/wrangles/extract/custom.md b/wrangles-docs/static/registry/wrangles/extract/custom.md index 05dc9f9..a9d612b 100644 --- a/wrangles-docs/static/registry/wrangles/extract/custom.md +++ b/wrangles-docs/static/registry/wrangles/extract/custom.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: model_id description: The ID of the wrangle to use. required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,49 +60,49 @@ parameters: - name: use_labels description: 'Use Labels in the extract output {label: value}.' required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: extract_raw description: Extract the raw data from the wrangle. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: use_spellcheck description: Use spellcheck to also find minor mispellings compared to the reference data. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: include_empty_labels description: Include labels with no found values in the output when using use_labels=True. required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: sort description: Sort the results. required: false - role: option + param_group: Options runtime_default: training_order schema: type: string @@ -118,7 +118,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -132,7 +132,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/date-properties.md b/wrangles-docs/static/registry/wrangles/extract/date-properties.md index 22d1a39..e094780 100644 --- a/wrangles-docs/static/registry/wrangles/extract/date-properties.md +++ b/wrangles-docs/static/registry/wrangles/extract/date-properties.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: property description: Property to extract from date. required: true - role: option + param_group: Options schema: type: string enum: @@ -55,7 +55,7 @@ parameters: - name: output description: Name of the output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/extract/date-range.md b/wrangles-docs/static/registry/wrangles/extract/date-range.md index 89811d8..e131fae 100644 --- a/wrangles-docs/static/registry/wrangles/extract/date-range.md +++ b/wrangles-docs/static/registry/wrangles/extract/date-range.md @@ -31,25 +31,25 @@ parameters: - name: start_time description: Name of the start date column. required: true - role: option + param_group: I/O schema: type: string - name: end_time description: Name of the end date column. required: true - role: option + param_group: I/O schema: type: string - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: range description: Type of frequency to count. required: false - role: option + param_group: Options runtime_default: day schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/html.md b/wrangles-docs/static/registry/wrangles/extract/html.md index 938610c..4f07693 100644 --- a/wrangles-docs/static/registry/wrangles/extract/html.md +++ b/wrangles-docs/static/registry/wrangles/extract/html.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: data_type description: The type of data to extract. required: true - role: option + param_group: Options schema: type: string enum: @@ -49,7 +49,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/properties.md b/wrangles-docs/static/registry/wrangles/extract/properties.md index 4c4198c..22ad0c9 100644 --- a/wrangles-docs/static/registry/wrangles/extract/properties.md +++ b/wrangles-docs/static/registry/wrangles/extract/properties.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output columns. required: true - role: column-output + param_group: I/O schema: type: - string @@ -48,7 +48,7 @@ parameters: - name: property_type description: The specific type of properties to extract. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -62,7 +62,7 @@ parameters: - name: return_data_type description: Legacy format option. Prefer output_format. required: false - role: option + param_group: Formatting runtime_default: list schema: type: string @@ -72,14 +72,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -93,7 +93,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/extract/regex.md b/wrangles-docs/static/registry/wrangles/extract/regex.md index 8529f1a..d546de0 100644 --- a/wrangles-docs/static/registry/wrangles/extract/regex.md +++ b/wrangles-docs/static/registry/wrangles/extract/regex.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: find description: Pattern to find using regex. required: true - role: option + param_group: Options schema: type: string - name: output description: Name of the output column(s). required: true - role: column-output + param_group: I/O schema: type: - string @@ -58,7 +58,7 @@ parameters: `r'(\d+)\s(\w+)'` and `output_pattern = '\2 \1'`, with input `'120 volt'`, the output would be `'volt 120'`. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -67,14 +67,14 @@ parameters: - name: first_element description: Get the first element from results. required: false - role: option + param_group: Formatting runtime_default: false schema: type: boolean - name: output_format description: Format of the extract output. required: false - role: column-output + param_group: Formatting runtime_default: null schema: type: @@ -87,7 +87,7 @@ parameters: - name: char description: Character to use when output_format is concatenate. required: false - role: option + param_group: Formatting runtime_default: ', ' schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/format/dates.md b/wrangles-docs/static/registry/wrangles/format/dates.md index 519bb66..a5ed72d 100644 --- a/wrangles-docs/static/registry/wrangles/format/dates.md +++ b/wrangles-docs/static/registry/wrangles/format/dates.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,13 +40,13 @@ parameters: - name: format description: String pattern to format date. required: true - role: option + param_group: Formatting schema: type: string - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/format/pad.md b/wrangles-docs/static/registry/wrangles/format/pad.md index 84de5da..337795e 100644 --- a/wrangles-docs/static/registry/wrangles/format/pad.md +++ b/wrangles-docs/static/registry/wrangles/format/pad.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,25 +40,25 @@ parameters: - name: pad_length description: Length for the output. required: true - role: option + param_group: Formatting schema: type: integer - name: side description: Side from which to fill resulting string. required: true - role: option + param_group: Options schema: type: string - name: char description: The character to pad the input with. required: true - role: option + param_group: Formatting schema: type: string - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -68,7 +68,7 @@ parameters: - name: skip_empty description: If true, skip padding for empty or whitespace-only values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/format/prefix.md b/wrangles-docs/static/registry/wrangles/format/prefix.md index 6035fd6..3e6e96f 100644 --- a/wrangles-docs/static/registry/wrangles/format/prefix.md +++ b/wrangles-docs/static/registry/wrangles/format/prefix.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: value description: Prefix value to add. required: true - role: option + param_group: Options schema: type: - string @@ -49,7 +49,7 @@ parameters: - name: output description: (Optional) Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/format/price-breaks.md b/wrangles-docs/static/registry/wrangles/format/price-breaks.md index 3ca2e39..3908183 100644 --- a/wrangles-docs/static/registry/wrangles/format/price-breaks.md +++ b/wrangles-docs/static/registry/wrangles/format/price-breaks.md @@ -31,19 +31,19 @@ parameters: - name: input description: Name, index, or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: categoryLabel description: Prefix for output columns that identify the source price-break category. required: true - role: option + param_group: Formatting schema: type: string - name: valueLabel description: Prefix for output columns that contain the corresponding price-break value. required: true - role: option + param_group: Formatting schema: type: string examples: [] @@ -60,4 +60,4 @@ sources: Expand non-empty price-break cells into paired category and value columns. -This first-pass guidance is derived from the callable signature. +This guidance was derived from the callable signature. diff --git a/wrangles-docs/static/registry/wrangles/format/remove-duplicates.md b/wrangles-docs/static/registry/wrangles/format/remove-duplicates.md index 2729093..897c6cf 100644 --- a/wrangles-docs/static/registry/wrangles/format/remove-duplicates.md +++ b/wrangles-docs/static/registry/wrangles/format/remove-duplicates.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -50,7 +50,7 @@ parameters: - name: ignore_case description: Ignore case when removing duplicates. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/format/significant-figures.md b/wrangles-docs/static/registry/wrangles/format/significant-figures.md index aead2ba..68c6587 100644 --- a/wrangles-docs/static/registry/wrangles/format/significant-figures.md +++ b/wrangles-docs/static/registry/wrangles/format/significant-figures.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,14 +40,14 @@ parameters: - name: significant_figures description: Number of significant figures to format to. Default is 3. required: false - role: option + param_group: Formatting runtime_default: 3 schema: type: integer - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/format/suffix.md b/wrangles-docs/static/registry/wrangles/format/suffix.md index 2134406..e53346a 100644 --- a/wrangles-docs/static/registry/wrangles/format/suffix.md +++ b/wrangles-docs/static/registry/wrangles/format/suffix.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: value description: Suffix value to add. required: true - role: option + param_group: Options schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: (Optional) Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/format/trim.md b/wrangles-docs/static/registry/wrangles/format/trim.md index fcc36bc..a05c2d0 100644 --- a/wrangles-docs/static/registry/wrangles/format/trim.md +++ b/wrangles-docs/static/registry/wrangles/format/trim.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/generate/ai.md b/wrangles-docs/static/registry/wrangles/generate/ai.md index 9690616..50fdeef 100644 --- a/wrangles-docs/static/registry/wrangles/generate/ai.md +++ b/wrangles-docs/static/registry/wrangles/generate/ai.md @@ -31,13 +31,13 @@ parameters: - name: api_key description: OpenAI-compatible API key. required: true - role: credential + param_group: Details schema: type: string - name: output description: Target schema; string/array shorthands are expanded automatically. required: true - role: column-output + param_group: I/O schema: type: - string @@ -46,7 +46,7 @@ parameters: - name: input description: Column(s) to concatenate into the prompt (defaults to all columns). required: false - role: column-selector + param_group: I/O runtime_default: null schema: type: @@ -56,35 +56,35 @@ parameters: - name: model description: Responses model name (e.g. gpt-5-mini). required: false - role: option + param_group: Details runtime_default: gpt-5 schema: type: string - name: threads description: Maximum concurrent requests (default 20). required: false - role: option + param_group: Execution runtime_default: 20 schema: type: integer - name: timeout description: Per-request timeout in seconds. required: false - role: option + param_group: Execution runtime_default: 90 schema: type: integer - name: retries description: Number of retry attempts on failure. required: false - role: option + param_group: Errors runtime_default: 0 schema: type: integer - name: messages description: Optional extra messages forwarded to the inner generate helper. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -94,28 +94,28 @@ parameters: - name: url description: Override for the OpenAI-compatible endpoint. required: false - role: option + param_group: Details runtime_default: https://api.openai.com/v1/responses schema: type: string - name: strict description: Enforce JSON-schema validation on the response. required: false - role: option + param_group: Details runtime_default: false schema: type: boolean - name: web_search description: Enable DuckDuckGo context lookup per row. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: reasoning description: Responses API reasoning options (forwarded verbatim). required: false - role: option + param_group: Details runtime_default: effort: low schema: @@ -125,14 +125,14 @@ parameters: - name: previous_response description: Chain responses by reusing previous_response_id for field-by-field calls. required: false - role: option + param_group: Details runtime_default: false schema: type: boolean - name: summary description: Request summary text to be merged into the output. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/coalesce.md b/wrangles-docs/static/registry/wrangles/merge/coalesce.md index 5178b47..276ee34 100644 --- a/wrangles-docs/static/registry/wrangles/merge/coalesce.md +++ b/wrangles-docs/static/registry/wrangles/merge/coalesce.md @@ -31,13 +31,13 @@ parameters: - name: input description: List of input columns or a single column containing lists. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output columns. This is required if multiple input columns are provided. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/merge/concatenate.md b/wrangles-docs/static/registry/wrangles/merge/concatenate.md index fa69003..cd5efec 100644 --- a/wrangles-docs/static/registry/wrangles/merge/concatenate.md +++ b/wrangles-docs/static/registry/wrangles/merge/concatenate.md @@ -31,7 +31,7 @@ parameters: - name: input description: Either a single column name or list of columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,20 +40,20 @@ parameters: - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: char description: (Optional) Character to add between successive values. required: false - role: option + param_group: Formatting runtime_default: ',' schema: type: string - name: skip_empty description: Whether to skip empty values, defaults to false. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/dictionaries.md b/wrangles-docs/static/registry/wrangles/merge/dictionaries.md index 97e424f..e68412a 100644 --- a/wrangles-docs/static/registry/wrangles/merge/dictionaries.md +++ b/wrangles-docs/static/registry/wrangles/merge/dictionaries.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: skip_empty description: Whether to skip empty dictionaries when merging. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/key-value-pairs.md b/wrangles-docs/static/registry/wrangles/merge/key-value-pairs.md index 167aa94..896566f 100644 --- a/wrangles-docs/static/registry/wrangles/merge/key-value-pairs.md +++ b/wrangles-docs/static/registry/wrangles/merge/key-value-pairs.md @@ -33,19 +33,19 @@ parameters: - name: input description: Matched pairs of key and value columns. required: true - role: column-selector + param_group: I/O schema: type: object - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: skip_empty description: Whether to skip empty keys or values when creating the dictionary. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/lists.md b/wrangles-docs/static/registry/wrangles/merge/lists.md index b20fbaf..04a8c50 100644 --- a/wrangles-docs/static/registry/wrangles/merge/lists.md +++ b/wrangles-docs/static/registry/wrangles/merge/lists.md @@ -31,33 +31,33 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: remove_duplicates description: Whether to remove duplicates from the created list. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: ignore_case description: Ignore case when removing duplicates. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: include_empty description: Whether to include empty values in the created list. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/to-dict.md b/wrangles-docs/static/registry/wrangles/merge/to-dict.md index 509779b..137a8bb 100644 --- a/wrangles-docs/static/registry/wrangles/merge/to-dict.md +++ b/wrangles-docs/static/registry/wrangles/merge/to-dict.md @@ -33,19 +33,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: include_empty description: Whether to include empty columns in the created dictionary. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/merge/to-list.md b/wrangles-docs/static/registry/wrangles/merge/to-list.md index 62ebd22..3566655 100644 --- a/wrangles-docs/static/registry/wrangles/merge/to-list.md +++ b/wrangles-docs/static/registry/wrangles/merge/to-list.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of input columns. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: include_empty description: Whether to include empty columns in the created list. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/search/find-links.md b/wrangles-docs/static/registry/wrangles/search/find-links.md index 2cde605..11561e7 100644 --- a/wrangles-docs/static/registry/wrangles/search/find-links.md +++ b/wrangles-docs/static/registry/wrangles/search/find-links.md @@ -33,7 +33,7 @@ parameters: - name: queries description: Name or list of input columns containing search queries. required: true - role: option + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: - name: id description: Name of the column containing the row ID to append to each search result. required: true - role: option + param_group: I/O schema: type: string - name: output @@ -49,7 +49,7 @@ parameters: Output column for the dictionaries. If a list of 2 is provided, outputs [dicts_column, pretty_strings_column]. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: client description: The search provider to use. required: false - role: option + param_group: Details runtime_default: serpapi schema: type: string @@ -70,7 +70,7 @@ parameters: API key for the search client. Can also be set as an environment variable (e.g., SERPAPI_API_KEY). required: false - role: credential + param_group: Details runtime_default: null schema: type: @@ -79,39 +79,39 @@ parameters: - name: n_results description: Number of search results to return per query (default 10, max 100). required: false - role: option + param_group: Options runtime_default: 10 schema: type: integer - name: threads description: Number of concurrent threads for parallel processing (default 10). required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer - name: country description: 'Country code for search results (default ''us''). Alias: gl.' required: false - role: option + param_group: Options schema: type: string - name: language description: 'Language code for search results (default ''en''). Alias: hl.' required: false - role: option + param_group: Options schema: type: string - name: location description: Location for search results (e.g., 'Austin, Texas'). required: false - role: option + param_group: Options schema: type: string - name: device description: Device type for search results. required: false - role: option + param_group: Options schema: type: string enum: diff --git a/wrangles-docs/static/registry/wrangles/search/retrieve-link-content.md b/wrangles-docs/static/registry/wrangles/search/retrieve-link-content.md index c83f25c..c8a87ee 100644 --- a/wrangles-docs/static/registry/wrangles/search/retrieve-link-content.md +++ b/wrangles-docs/static/registry/wrangles/search/retrieve-link-content.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns containing URLs or Scored Search Result dictionaries. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -43,7 +43,7 @@ parameters: Name of the output column for the raw dictionaries. To output BOTH the raw dictionaries and the formatted text, provide a list of exactly two column names (e.g., [page_data, page_text]). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: client description: The retrieval provider to use. required: false - role: option + param_group: Details runtime_default: google_url_context schema: type: string @@ -62,7 +62,7 @@ parameters: - name: api_key description: API key for the provider. Can also be set as an environment variable (e.g., GOOGLE_API_KEY). required: false - role: credential + param_group: Details runtime_default: null schema: type: @@ -71,7 +71,7 @@ parameters: - name: prompt description: Optional custom system prompt to guide the extraction behavior and output format. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -80,14 +80,14 @@ parameters: - name: model_id description: The specific model ID to use (default models/gemini-3-flash-preview). required: false - role: model-reference + param_group: Details runtime_default: models/gemini-3-flash-preview schema: type: string - name: output_format description: The desired format for the extracted content. required: false - role: column-output + param_group: Formatting runtime_default: json schema: type: string @@ -97,7 +97,7 @@ parameters: - name: threads description: Number of concurrent threads for parallel processing (default 10). required: false - role: option + param_group: Execution runtime_default: 10 schema: type: integer diff --git a/wrangles-docs/static/registry/wrangles/select/columns.md b/wrangles-docs/static/registry/wrangles/select/columns.md index eaaa709..5e43303 100644 --- a/wrangles-docs/static/registry/wrangles/select/columns.md +++ b/wrangles-docs/static/registry/wrangles/select/columns.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column(s) to select. required: true - role: column-selector + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/select/dictionary-element.md b/wrangles-docs/static/registry/wrangles/select/dictionary-element.md index 577ffe5..de37e3f 100644 --- a/wrangles-docs/static/registry/wrangles/select/dictionary-element.md +++ b/wrangles-docs/static/registry/wrangles/select/dictionary-element.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,13 +42,13 @@ parameters: The key or keys from the dictionary to select. If a single key is provided, the value will be returned If a lists of keys are selected, the result will be a new dictionary. required: true - role: option + param_group: Options schema: type: string - name: output description: Name of the output column. If omitted, the input column will be replaced. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: Set the default value to return if the specified element doesn't exist. If selecting multiple elements, a dict of defaults can be set. required: false - role: fallback-value + param_group: Errors runtime_default: '' schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/element.md b/wrangles-docs/static/registry/wrangles/select/element.md index 9cc5232..e8321b7 100644 --- a/wrangles-docs/static/registry/wrangles/select/element.md +++ b/wrangles-docs/static/registry/wrangles/select/element.md @@ -33,7 +33,7 @@ parameters: Name of the input column and sub elements This permits by index for lists or dict and by key for dicts e.g. col[0]['key'] // [{"key":"val"}] -> "val". required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: - name: default description: Set the default value to return if the specified element doesn't exist. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/group-by.md b/wrangles-docs/static/registry/wrangles/select/group-by.md index d1cbb50..af00a5b 100644 --- a/wrangles-docs/static/registry/wrangles/select/group-by.md +++ b/wrangles-docs/static/registry/wrangles/select/group-by.md @@ -31,7 +31,7 @@ parameters: - name: by description: List of the input columns to group on. required: false - role: column-selector + param_group: I/O runtime_default: [] schema: type: @@ -43,14 +43,14 @@ parameters: If false, column names are left as-is; use a dictionary entry to supply a custom output name (e.g. - Value: Total). required: false - role: option + param_group: Formatting runtime_default: true schema: type: boolean - name: list description: Group and return all values for these column(s) as a list. required: false - role: option + param_group: Options schema: type: - string @@ -58,7 +58,7 @@ parameters: - name: first description: The first value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -66,7 +66,7 @@ parameters: - name: last description: The last value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -74,7 +74,7 @@ parameters: - name: min description: The minimum value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -82,7 +82,7 @@ parameters: - name: max description: The maximum value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -90,7 +90,7 @@ parameters: - name: mean description: The mean (average) value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -98,7 +98,7 @@ parameters: - name: median description: The median value for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -106,7 +106,7 @@ parameters: - name: nunique description: The count of unique values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -114,7 +114,7 @@ parameters: - name: count description: The count of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -125,7 +125,7 @@ parameters: converted to JSON-safe strings; missing values use the key "null" and booleans use lowercase "true"/"false". required: false - role: option + param_group: Options schema: type: - string @@ -133,7 +133,7 @@ parameters: - name: std description: The standard deviation of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -141,7 +141,7 @@ parameters: - name: sum description: The total of values for these column(s). required: false - role: option + param_group: Options schema: type: - string @@ -149,7 +149,7 @@ parameters: - name: any description: Return true if any of the values for these column(s) are true. required: false - role: option + param_group: Options schema: type: - string @@ -157,7 +157,7 @@ parameters: - name: all description: Return true if all of the values for these column(s) are true. required: false - role: option + param_group: Options schema: type: - string @@ -165,7 +165,7 @@ parameters: - name: p75 description: Get a percentile. Note, you can use any integer here for the corresponding percentile. required: false - role: option + param_group: Options schema: type: - string @@ -174,7 +174,7 @@ parameters: name_pattern: ^custom\.[A-Za-z_][A-Za-z0-9_]*$ description: Placeholder for custom functions. Replace 'placeholder' with the name of the function. required: false - role: option + param_group: Options schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/select/head.md b/wrangles-docs/static/registry/wrangles/select/head.md index 7a1c41b..e72e0ca 100644 --- a/wrangles-docs/static/registry/wrangles/select/head.md +++ b/wrangles-docs/static/registry/wrangles/select/head.md @@ -31,7 +31,7 @@ parameters: - name: n description: Number of rows to return. required: true - role: option + param_group: Options schema: type: integer examples: [] diff --git a/wrangles-docs/static/registry/wrangles/select/highest-confidence.md b/wrangles-docs/static/registry/wrangles/select/highest-confidence.md index 57a8b0b..20958b2 100644 --- a/wrangles-docs/static/registry/wrangles/select/highest-confidence.md +++ b/wrangles-docs/static/registry/wrangles/select/highest-confidence.md @@ -33,13 +33,13 @@ parameters: - name: input description: List of the input columns to select from. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: If two columns; the result and confidence. If one column; [result, confidence]. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/select/left.md b/wrangles-docs/static/registry/wrangles/select/left.md index 6b97981..4fb625f 100644 --- a/wrangles-docs/static/registry/wrangles/select/left.md +++ b/wrangles-docs/static/registry/wrangles/select/left.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -44,13 +44,13 @@ parameters: Number of characters to include from the left. If negative, this will remove the specified number of characters from the left. May not equal 0. required: true - role: option + param_group: Options schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/length.md b/wrangles-docs/static/registry/wrangles/select/length.md index 3bbbe19..00ee362 100644 --- a/wrangles-docs/static/registry/wrangles/select/length.md +++ b/wrangles-docs/static/registry/wrangles/select/length.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the input column(s). required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/list-element.md b/wrangles-docs/static/registry/wrangles/select/list-element.md index be97b68..c83902b 100644 --- a/wrangles-docs/static/registry/wrangles/select/list-element.md +++ b/wrangles-docs/static/registry/wrangles/select/list-element.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the input column. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -40,7 +40,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,14 +52,14 @@ parameters: The numbered element of the list to select. Starts from zero. This may use python slicing syntax to select a subset of the list. required: false - role: option + param_group: Options runtime_default: 0 schema: type: integer - name: default description: Set the default value to return if the specified element doesn't exist. required: false - role: fallback-value + param_group: Errors runtime_default: '' schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/right.md b/wrangles-docs/static/registry/wrangles/select/right.md index c080b2c..c46a334 100644 --- a/wrangles-docs/static/registry/wrangles/select/right.md +++ b/wrangles-docs/static/registry/wrangles/select/right.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -44,13 +44,13 @@ parameters: Number of characters to include from the right. If negative, this will remove the specified number of characters from the right. May not equal 0. required: true - role: option + param_group: Options schema: type: integer - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/sample.md b/wrangles-docs/static/registry/wrangles/select/sample.md index b200b89..d663762 100644 --- a/wrangles-docs/static/registry/wrangles/select/sample.md +++ b/wrangles-docs/static/registry/wrangles/select/sample.md @@ -33,7 +33,7 @@ parameters: If a whole number, will select that number of rows. If a decimal between 0 and 1 will select that fraction of the rows e.g. 0.1 => 10% of rows will be returned. required: true - role: option + param_group: Options schema: type: - integer diff --git a/wrangles-docs/static/registry/wrangles/select/substring.md b/wrangles-docs/static/registry/wrangles/select/substring.md index 7e10467..a2461c0 100644 --- a/wrangles-docs/static/registry/wrangles/select/substring.md +++ b/wrangles-docs/static/registry/wrangles/select/substring.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column(s) to edit. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: The position of the first character to select. If ommited will start from the beginning and length must be provided. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -54,7 +54,7 @@ parameters: The length of the string to select. If ommited will select to the end of the string and start must be provided. required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -64,7 +64,7 @@ parameters: - name: output description: Name of the output column(s). required: false - role: column-output + param_group: I/O runtime_default: null schema: type: diff --git a/wrangles-docs/static/registry/wrangles/select/tail.md b/wrangles-docs/static/registry/wrangles/select/tail.md index 9464a8e..be85f43 100644 --- a/wrangles-docs/static/registry/wrangles/select/tail.md +++ b/wrangles-docs/static/registry/wrangles/select/tail.md @@ -31,7 +31,7 @@ parameters: - name: n description: Number of rows to return. required: true - role: option + param_group: Options schema: type: integer examples: [] diff --git a/wrangles-docs/static/registry/wrangles/select/threshold.md b/wrangles-docs/static/registry/wrangles/select/threshold.md index 342440e..58dcb94 100644 --- a/wrangles-docs/static/registry/wrangles/select/threshold.md +++ b/wrangles-docs/static/registry/wrangles/select/threshold.md @@ -31,19 +31,19 @@ parameters: - name: input description: List of the input columns to select from. required: true - role: column-selector + param_group: I/O schema: type: array - name: output description: Name of the output column. required: true - role: column-output + param_group: I/O schema: type: string - name: threshold description: Threshold above which to choose the first option, otherwise the second. required: true - role: option + param_group: Options schema: type: number minimum: 0 diff --git a/wrangles-docs/static/registry/wrangles/split/dictionary.md b/wrangles-docs/static/registry/wrangles/split/dictionary.md index 131ac37..163b991 100644 --- a/wrangles-docs/static/registry/wrangles/split/dictionary.md +++ b/wrangles-docs/static/registry/wrangles/split/dictionary.md @@ -35,7 +35,7 @@ parameters: Name or lists of the column(s) containing dictionaries to be split. If providing multiple dictionaries and the dictionaries contain overlapping values, the last value will be returned. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -49,7 +49,7 @@ parameters: be two output columns for the keys and values lists. If not provided, Keys and Values will be used. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -59,7 +59,7 @@ parameters: - name: default description: Provide a set of default headings and values if they are not found within the input. required: false - role: fallback-value + param_group: Errors runtime_default: null schema: type: @@ -70,7 +70,7 @@ parameters: How to split the dictionary. columns creates one output column for each dictionary key. to_lists creates two output columns containing lists of keys and values. required: false - role: column-output + param_group: Formatting runtime_default: columns schema: type: string diff --git a/wrangles-docs/static/registry/wrangles/split/list.md b/wrangles-docs/static/registry/wrangles/split/list.md index c96388e..889d858 100644 --- a/wrangles-docs/static/registry/wrangles/split/list.md +++ b/wrangles-docs/static/registry/wrangles/split/list.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column to be split. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -41,7 +41,7 @@ parameters: Name of column(s) for the results. If providing a single column, use a wildcard (*) to indicate a incrementing integer. required: true - role: column-output + param_group: I/O schema: type: - string diff --git a/wrangles-docs/static/registry/wrangles/split/text.md b/wrangles-docs/static/registry/wrangles/split/text.md index 1d69806..968c26d 100644 --- a/wrangles-docs/static/registry/wrangles/split/text.md +++ b/wrangles-docs/static/registry/wrangles/split/text.md @@ -31,7 +31,7 @@ parameters: - name: input description: Name of the column to be split. required: true - role: column-selector + param_group: I/O schema: type: string - name: output @@ -40,7 +40,7 @@ parameters: list If multiple columns are listed, the results will be separated into the columns. If omitted, will overwrite the input. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -52,7 +52,7 @@ parameters: Set the character(s) to split on. Default comma (,) Can also prefix with "regex:" to split on a pattern. required: false - role: option + param_group: Options runtime_default: ',' schema: type: string @@ -61,7 +61,7 @@ parameters: Choose whether to pad to ensure a consistent length. Default true if outputting to columns, false for lists. required: false - role: option + param_group: Formatting runtime_default: null schema: type: @@ -72,7 +72,7 @@ parameters: Select a specific element or range after splitting using slicing syntax. e.g. 0, ":5", "5:", "2:8:2". required: false - role: option + param_group: Options runtime_default: null schema: type: @@ -82,14 +82,14 @@ parameters: - name: inclusive description: If true, include the split character in the output. Default False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: skip_empty description: Whether to skip empty values. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean diff --git a/wrangles-docs/static/registry/wrangles/split/tokenize.md b/wrangles-docs/static/registry/wrangles/split/tokenize.md index bfd3a10..00b6211 100644 --- a/wrangles-docs/static/registry/wrangles/split/tokenize.md +++ b/wrangles-docs/static/registry/wrangles/split/tokenize.md @@ -33,7 +33,7 @@ parameters: - name: input description: Column(s) to be split into tokens. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name of the output column. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -54,7 +54,7 @@ parameters: Method to split the list. Options include `space`, `boundary`, `boundary_ignore_space`, custom functions as `custom.`, or regex patterns as `regex:`. required: false - role: option + param_group: Options runtime_default: space schema: anyOf: diff --git a/wrangles-docs/static/registry/wrangles/standardize/clean.md b/wrangles-docs/static/registry/wrangles/standardize/clean.md index f8a7a9b..4bcbff0 100644 --- a/wrangles-docs/static/registry/wrangles/standardize/clean.md +++ b/wrangles-docs/static/registry/wrangles/standardize/clean.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: output description: Name or list of output columns. Defaults to overwriting input. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -53,7 +53,7 @@ parameters: - name: fix_encoding description: Repair mojibake and other reversible encoding errors. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean @@ -62,7 +62,7 @@ parameters: Decode HTML character references. Auto avoids decoding text that appears to contain HTML markup. required: false - role: option + param_group: Options runtime_default: auto schema: anyOf: @@ -73,7 +73,7 @@ parameters: - name: normalization description: Unicode normalization form. required: false - role: option + param_group: Options runtime_default: NFC schema: type: string @@ -86,49 +86,49 @@ parameters: - name: fix_character_width description: Normalize fullwidth and halfwidth characters. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: uncurl_quotes description: Replace typographic quotes with straight quotes. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: remove_control_chars description: Remove C0 and C1 control characters. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: collapse_whitespace description: Collapse runs of Unicode whitespace. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: preserve_line_breaks description: Preserve line breaks while collapsing other whitespace. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean - name: trim description: Remove leading and trailing whitespace. required: false - role: option + param_group: Options runtime_default: true schema: type: boolean - name: separator description: Text used to join multiple input columns into one output. required: false - role: option + param_group: Formatting runtime_default: ' ' schema: type: string @@ -146,4 +146,4 @@ sources: Repair common encoding, Unicode, HTML character reference, control character, and whitespace problems locally. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/wrangles-docs/static/registry/wrangles/standardize/custom.md b/wrangles-docs/static/registry/wrangles/standardize/custom.md index 5f9e07c..a8e98e2 100644 --- a/wrangles-docs/static/registry/wrangles/standardize/custom.md +++ b/wrangles-docs/static/registry/wrangles/standardize/custom.md @@ -33,7 +33,7 @@ parameters: - name: input description: Name or list of input columns. required: true - role: column-selector + param_group: I/O schema: type: - string @@ -42,7 +42,7 @@ parameters: - name: model_id description: The ID of the wrangle to use (do not include 'find' and 'replace'). required: true - role: model-reference + param_group: Details schema: type: - string @@ -50,7 +50,7 @@ parameters: - name: output description: Name or list of output columns. required: false - role: column-output + param_group: I/O runtime_default: null schema: type: @@ -60,7 +60,7 @@ parameters: - name: case_sensitive description: Allows the wrangle to be case sensitive if set to True, default is False. required: false - role: option + param_group: Options runtime_default: false schema: type: boolean @@ -78,4 +78,4 @@ sources: Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription. -This first-pass guidance is derived from the callable signature and its embedded Python schema docstring. +This guidance was derived from the callable signature and its embedded Python schema docstring. diff --git a/wrangles-docs/static/schemas/recipes/pilot/schema.json b/wrangles-docs/static/schemas/recipes/registry/schema.json similarity index 99% rename from wrangles-docs/static/schemas/recipes/pilot/schema.json rename to wrangles-docs/static/schemas/recipes/registry/schema.json index 7491583..741e2c6 100644 --- a/wrangles-docs/static/schemas/recipes/pilot/schema.json +++ b/wrangles-docs/static/schemas/recipes/registry/schema.json @@ -1,9 +1,9 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - "$id": "https://docs.wrangles.com/schemas/recipes/pilot/schema.json", - "title": "Wrangles Recipe Schema - Registry Pilot", - "description": "Pilot schema containing only the initial Registry entries.", - "$comment": "Not the production recipe schema. Generated from Registry version 0.1.0-pilot.", + "$id": "https://docs.wrangles.com/schemas/recipes/registry/schema.json", + "title": "Wrangles Recipe Schema - Registry", + "description": "Pre-production schema generated from the Docs Registry.", + "$comment": "Pre-production artifact. See the Registry contract for production-readiness criteria.", "type": "object", "additionalProperties": false, "required": [ @@ -5707,7 +5707,7 @@ }, "standardize": { "type": "object", - "description": "Standardize data using a DIY or bespoke standardization wrangle. Requires WrangleWorks Account and Subscription.", + "description": "Deprecated compatibility key for `standardize.custom`, which standardizes data using a trained DIY or bespoke model.", "additionalProperties": false, "properties": { "input": {