Skip to content

fix and reordering of process to improve matching - #37

Merged
shauntruelove merged 5 commits into
mainfrom
fix_matching_Aug2026
Sep 4, 2026
Merged

shauntruelove merged 5 commits into
mainfrom
fix_matching_Aug2026

Conversation

@shauntruelove

Copy link
Copy Markdown
Contributor

This pull request introduces several improvements and clarifications to data quality assurance (DQA) and Google Places API integration in the codebase. The most significant changes include deprecating legacy DQA functions in favor of newer, dplyr-based implementations, and enhancing the Google Places API utilities to better handle school renames and cache key collisions. Below is a summary of the most important changes:

Deprecation and Documentation of Legacy DQA Functions:

  • Marked all legacy DQA functions in R/dqa_checks.R and R/generate_dqa_summary.R as deprecated, with clear documentation pointing to their modern replacements in final_format_utils.R. This helps guide developers to use the current, supported pipeline and clarifies the status of legacy code. [1] [2] [3] [4] [5] [6] [7] [8] [9]
  • Updated the documentation for clean_state_data() to indicate its deprecated status and explain differences from the preferred utility functions.

Google Places API and Caching Improvements:

  • Improved cache key handling in load_and_merge_cache() (in R/googleapi_utils.R) to use both school_name and county_std when available, preventing collisions for schools with the same name in different counties. [1] [2] [3]
  • Enhanced the Places API utility by renaming and expanding get_business_status() to get_place_details(), which now returns both the business status and the current Google name for a place. This supports more robust detection of school renames and maintains backward compatibility with a deprecated wrapper. [1] [2]
  • Updated the school status caching logic to consistently include the new google_place_name field, handling older cache files gracefully and ensuring all relevant columns are present in the merged result. [1] [2] [3]

NAMESPACE and Export Updates:

  • Exported new and updated functions related to Google name variants and matching, as well as new cascade and place ID resolution utilities, ensuring they are available for use elsewhere in the package. [1] [2] [3]

These changes collectively improve code clarity, maintainability, and robustness, especially around DQA and geocoding workflows.

Copilot AI lite review requested due to automatic review settings August 11, 2026 20:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the school standardization/matching pipeline to use Google “identity” (Place ID + current Google place name) earlier in the workflow to better bridge school renames, while also deprecating legacy DQA helpers in favor of the newer final_format_utils.R implementations.

Changes:

  • Adds early Google identity resolution (resolve_school_place_ids(), add_google_name_variants()) and uses place_id + Google-current-name as additional matching signals.
  • Refactors repeated multi-pass matching into a shared run_matching_cascade() helper (including an exact-join “Pass 0” for place_id).
  • Marks legacy DQA functions / wrappers as deprecated with updated roxygen + generated man pages; updates Places status lookup to also cache/return google_place_name.

Reviewed changes

Copilot reviewed 21 out of 40 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/testthat/test-school_identity_utils.R Adds unit tests for the new identity/rename matching behavior (no live API calls).
R/standardize_schools.R Reorders pipeline to resolve Google identity earlier and writes rename QC output sooner.
R/school_identity_utils.R New utilities to resolve place_id and attach standardized Google current names; adds rename report writer.
R/preprocessing_utils.R Extracts shared name/county/level/district standardization into .standardize_school_source().
R/matching_schools_utils.R Introduces .run_exact_join_pass() and exported run_matching_cascade() for multi-pass matching.
R/googleapi_utils.R Improves cache keying, expands Places lookup to include name, and propagates google_place_name through caching.
R/run_dqa_checks.R Updates documentation to reflect deprecation of legacy DQA orchestration.
R/generate_dqa_summary.R Updates documentation to reflect deprecation of legacy DQA summary generation.
R/dqa_checks.R Adds file-level and per-function deprecation documentation mapping to new DQA checks.
R/clean_state_data.R Documents deprecated status and why it remains separate from newer formatting utilities.
NAMESPACE Exports new identity/matching helpers.
man/*.Rd (multiple) Adds/updates generated documentation for new APIs and deprecations.
data/state_zip_prefix.csv Adds CSV companion to existing dataset for ZIP-prefix filtering used in get_geo_info().
Files not reviewed (15)
  • man/add_google_name_variants.Rd: Generated file
  • man/augment_with_google_name_variant.Rd: Generated file
  • man/build_reference_key.Rd: Generated file
  • man/check_coverage_outliers.Rd: Generated file
  • man/check_duplicates.Rd: Generated file
  • man/check_enrollment_deviation.Rd: Generated file
  • man/check_exceeding_enrollment_values.Rd: Generated file
  • man/check_extreme_outliers.Rd: Generated file
  • man/check_negative_values.Rd: Generated file
  • man/check_vaccination_deviation.Rd: Generated file
  • man/clean_state_data.Rd: Generated file
  • man/dot-run_exact_join_pass.Rd: Generated file
  • man/dot-standardize_school_source.Rd: Generated file
  • man/generate_detailed_dqa_report.Rd: Generated file
  • man/generate_dqa_summary.Rd: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread R/matching_schools_utils.R Outdated
shauntruelove and others added 4 commits August 13, 2026 10:08
…aude-written, so error checking might be needed
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@shauntruelove
shauntruelove merged commit 68498aa into main Sep 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants