fix(seo): exclude shadow-banned profiles from index and sitemap#3864
Conversation
Adds a derived `noindex` boolean on the User GraphQL type combining shadow-ban (`flags.vordr`) and the existing low-reputation signal so shadow-banned profile pages stop passing link equity, and excludes vordr users from `users.xml` so Google never gets pointed at them. The profile page itself keeps returning 200 with normal content — the shadow-ban illusion is preserved; only Googlebot is affected via the `noindex` meta tag and sitemap omission. Co-authored-by: Cursor <cursoragent@cursor.com>
|
🍹 The Update (preview) for dailydotdev/api/prod (at dce98fb) was successful. ✨ Neo ExplanationRoutine deployment of commit `93d84afa` that adds `noindex` support for shadow-banned and low-reputation users, excluding them from sitemaps and exposing a `noindex` GraphQL field. ✅ Low Risk — no stateful resource replacements, no security group changes; DB and Clickhouse migration jobs run as part of the standard release process.This is a standard application deployment rolling out a new image ( The Resource Changes Name Type Operation
~ vpc-native-private-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-temporal-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-clean-channel-highlights-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-analytics-clickhouse-cron kubernetes:batch/v1:CronJob update
- vpc-native-api-clickhouse-migration-2fc38143 kubernetes:batch/v1:Job delete
~ vpc-native-sync-subscription-with-cio-cron kubernetes:batch/v1:CronJob update
~ vpc-native-ws-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-hourly-notification-cron kubernetes:batch/v1:CronJob update
~ vpc-native-validate-active-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-expire-super-agent-trial-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-check-analytics-report-cron kubernetes:batch/v1:CronJob update
~ vpc-native-daily-digest-cron kubernetes:batch/v1:CronJob update
~ vpc-native-worker-job-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-clean-gifted-plus-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-user-companies-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-tags-str-cron kubernetes:batch/v1:CronJob update
~ vpc-native-personalized-digest-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-analytics-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-rotate-daily-quests-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-views-cron kubernetes:batch/v1:CronJob update
+ vpc-native-api-db-migration-93d84afa kubernetes:batch/v1:Job create
~ vpc-native-post-analytics-history-day-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-images-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-trending-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-expired-better-auth-sessions-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-highlighted-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-rotate-weekly-quests-cron kubernetes:batch/v1:CronJob update
~ vpc-native-squad-posts-analytics-refresh-cron kubernetes:batch/v1:CronJob update
~ vpc-native-channel-highlights-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-opportunities-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-tag-materialized-views-cron kubernetes:batch/v1:CronJob update
- vpc-native-api-db-migration-2fc38143 kubernetes:batch/v1:Job delete
~ vpc-native-update-source-public-threshold-cron kubernetes:batch/v1:CronJob update
~ vpc-native-personalized-digest-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-update-achievement-rarity-cron kubernetes:batch/v1:CronJob update
~ vpc-native-calculate-top-readers-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-current-streak-cron kubernetes:batch/v1:CronJob update
~ vpc-native-generate-search-invites-cron kubernetes:batch/v1:CronJob update
~ vpc-native-generic-referral-reminder-cron kubernetes:batch/v1:CronJob update
... and 11 other changes |
Summary
Shadow-banned (vordr) user profiles currently return 200 with full content, no
noindex, and are listed inusers.xml— so Google indexes them and inbound links pass equity through. This PR closes both vectors while preserving the shadow-ban illusion (page content unchanged).User.noindex: Boolean!GraphQL field, computed fromflags.vordr || reputation <= MIN_INDEXABLE_REPUTATION. The webapp will consume it (companion PR: dailydotdev/apps#TBD).buildUsersSitemapQueryvia the existingwhereVordrFilterhelper.MIN_INDEXABLE_REPUTATION = 10constant insrc/common/users.tsand uses it in both places to prevent threshold drift.Trade-off (intentional)
noindexis publicly queryable. For users withreputation > 10,noindex: truedeterministically reveals shadow-ban via GraphQL. We accepted this over a real 404 because 404 would be a clean tell on the rendered page itself; a GraphQL-only signal is a smaller surface than a status-code change.Test plan
__tests__/users.ts— three newquery user noindexcases (rep > 10 + no vordr, rep <= 10, vordr + high rep).__tests__/sitemaps.ts— extendedusers.xmltest asserts vordr user is excluded and a non-vordr user withflags: {vordr: false}IS included.Made with Cursor