-
Notifications
You must be signed in to change notification settings - Fork 414
CS-656 Create script to Migrate all users from old domain to new domain - remove duplicate people records #3304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
github-actions
wants to merge
31
commits into
main
Choose a base branch
from
chas/email-migration-script
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 11 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
81582b1
fix(api): add merge-duplicate-user task to consolidate users after em…
chasprowebdev df9a837
fix(api): create trigger job to find duplicate members and migrate
chasprowebdev 4bcfe57
fix(api): normalize emails and domains to lowercase before matching i…
chasprowebdev 1f91792
fix(api): update the logger details in merge-duplicate-user trigger job
chasprowebdev 4ae255c
fix(api): add guard against normalized old/new domains in merge-org-e…
chasprowebdev 6a647f3
fix(api): re-point user-level relations before deleting old user in m…
chasprowebdev e06d1a1
fix(api): migrate SOA and ISMS document approver before deleting old …
chasprowebdev bba164e
fix(api): above user deletion inn merge-duplicate-user
chasprowebdev 7f30f83
fix(api): dedupe offboarding and background check records before memb…
chasprowebdev fc47152
fix(api): remove user after merging in merge-duplicate-user
chasprowebdev 39d0d89
fix(api): avoid deleting user record in merge-duplicate-user
chasprowebdev 1298f76
Merge branch 'main' into chas/email-migration-script
tofikwest eec2b5f
Merge branch 'main' into chas/email-migration-script
tofikwest b1faa18
fix(api): skip user-level merge for old users in multiple orgs
chasprowebdev f8e0cc8
fix(api): re-point remaining user-level relations in merge-duplicate-…
chasprowebdev 85c134d
fix(api): correct surviving user/member ids in merge-duplicate-user log
chasprowebdev 5f2825e
fix(api): clean up stale comments in merge-duplicate-user
chasprowebdev 9985da7
Merge branch 'chas/email-migration-script' of https://github.com/tryc…
chasprowebdev ade97fa
Merge branch 'main' of https://github.com/trycompai/comp into chas/em…
chasprowebdev c56dd32
fix(api): remove scripts from .gitignore and make seeded DB test for …
chasprowebdev 95c5779
fix(api): put local scripts to gitignore
chasprowebdev 2a45f2f
fix(api): write jest tests for email domain migration scripts
chasprowebdev 21cad2d
fix(api): keep old user record instead of deleting in merge-duplicate…
chasprowebdev f1dad21
fix(api): scope OffboardingAccessRevocation.revokedById to the multi-…
chasprowebdev 3976d84
fix(api): re-point IsmsObjective.ownerMemberId in merge-duplicate-user
chasprowebdev e071d0d
Merge branch 'main' of https://github.com/trycompai/comp into chas/em…
chasprowebdev 0297b43
fix(api): add inequality guard to merge-duplicate-user
chasprowebdev f134a35
fix(api): recompute org-membership check inside merge-duplicate-user …
chasprowebdev 842eb0a
Merge branch 'main' into chas/email-migration-script
tofikwest 128d9d1
Merge branch 'main' of https://github.com/trycompai/comp into chas/em…
chasprowebdev 4465ddb
fix(api): discover member foreign keys from Postgres catalog instead …
chasprowebdev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
381 changes: 381 additions & 0 deletions
381
apps/api/src/trigger/tasks/people/merge-duplicate-user.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,381 @@ | ||
| import { db } from '@db'; | ||
| import { logger, schemaTask, tags } from '@trigger.dev/sdk'; | ||
| import { z } from 'zod'; | ||
|
|
||
| export const mergeDuplicateUser = schemaTask({ | ||
| id: 'merge-duplicate-user', | ||
| schema: z.object({ | ||
| organizationId: z.string(), | ||
| oldEmail: z.string().email(), | ||
| newEmail: z.string().email(), | ||
| }), | ||
| run: async ({ organizationId, oldEmail, newEmail }) => { | ||
|
chasprowebdev marked this conversation as resolved.
|
||
| await tags.add([`org:${organizationId}`]); | ||
|
|
||
| // ── 1. Resolve both users ──────────────────────────────────────────────── | ||
|
|
||
| const [oldUser, newUser] = await Promise.all([ | ||
| db.user.findUnique({ where: { email: oldEmail } }), | ||
| db.user.findUnique({ where: { email: newEmail } }), | ||
| ]); | ||
|
|
||
| if (!oldUser) { | ||
| throw new Error(`Old user not found: ${oldEmail}`); | ||
| } | ||
| if (!newUser) { | ||
| throw new Error(`New user not found: ${newEmail}`); | ||
| } | ||
|
|
||
| logger.info('Resolved users', { | ||
| oldUserId: oldUser.id, | ||
| newUserId: newUser.id, | ||
| }); | ||
|
|
||
| // ── 2. Resolve both members in this org ────────────────────────────────── | ||
|
|
||
| const [oldMember, newMember] = await Promise.all([ | ||
|
chasprowebdev marked this conversation as resolved.
|
||
| db.member.findFirst({ where: { userId: oldUser.id, organizationId } }), | ||
| db.member.findFirst({ where: { userId: newUser.id, organizationId } }), | ||
| ]); | ||
|
|
||
| if (!oldMember) { | ||
| throw new Error( | ||
| `Old member not found for user ${oldUser.id} in org ${organizationId}`, | ||
| ); | ||
| } | ||
| if (!newMember) { | ||
| throw new Error( | ||
| `New member not found for user ${newUser.id} in org ${organizationId}`, | ||
| ); | ||
| } | ||
|
|
||
| logger.info('Resolved members', { | ||
| oldMemberId: oldMember.id, | ||
| newMemberId: newMember.id, | ||
| }); | ||
|
|
||
| // ── 3. Merge inside a transaction ──────────────────────────────────────── | ||
|
|
||
| await db.$transaction( | ||
| async (tx) => { | ||
| const o = oldMember.id; | ||
| const n = newMember.id; | ||
|
chasprowebdev marked this conversation as resolved.
|
||
|
|
||
| // Policies: assigneeId, approverId, signedBy (String[] — replace in array) | ||
| await tx.policy.updateMany({ | ||
| where: { assigneeId: o }, | ||
| data: { assigneeId: n }, | ||
| }); | ||
| await tx.policy.updateMany({ | ||
| where: { approverId: o }, | ||
| data: { approverId: n }, | ||
| }); | ||
|
|
||
| // signedBy is a String[], use raw update to replace the member id in the array | ||
| const policiesWithSignature = await tx.policy.findMany({ | ||
| where: { signedBy: { has: o } }, | ||
| select: { id: true, signedBy: true }, | ||
| }); | ||
| for (const policy of policiesWithSignature) { | ||
| const updated = policy.signedBy.map((id) => (id === o ? n : id)); | ||
|
chasprowebdev marked this conversation as resolved.
Outdated
|
||
| await tx.policy.update({ | ||
| where: { id: policy.id }, | ||
| data: { signedBy: updated }, | ||
| }); | ||
| } | ||
|
|
||
| // PolicyVersion: publishedById | ||
| await tx.policyVersion.updateMany({ | ||
| where: { publishedById: o }, | ||
| data: { publishedById: n }, | ||
| }); | ||
|
|
||
| // Risk: assigneeId | ||
| await tx.risk.updateMany({ | ||
| where: { assigneeId: o }, | ||
| data: { assigneeId: n }, | ||
| }); | ||
|
|
||
| // Task: assigneeId, approverId | ||
| await tx.task.updateMany({ | ||
| where: { assigneeId: o }, | ||
| data: { assigneeId: n }, | ||
| }); | ||
| await tx.task.updateMany({ | ||
| where: { approverId: o }, | ||
| data: { approverId: n }, | ||
| }); | ||
|
|
||
| // TaskItem: assigneeId, createdById, updatedById | ||
| await tx.taskItem.updateMany({ | ||
| where: { assigneeId: o }, | ||
| data: { assigneeId: n }, | ||
| }); | ||
| await tx.taskItem.updateMany({ | ||
| where: { createdById: o }, | ||
| data: { createdById: n }, | ||
| }); | ||
| await tx.taskItem.updateMany({ | ||
| where: { updatedById: o }, | ||
| data: { updatedById: n }, | ||
| }); | ||
|
|
||
| // Vendor: assigneeId | ||
| await tx.vendor.updateMany({ | ||
| where: { assigneeId: o }, | ||
| data: { assigneeId: n }, | ||
| }); | ||
|
|
||
| // Finding: memberId (subject), createdById | ||
| await tx.finding.updateMany({ | ||
| where: { memberId: o }, | ||
| data: { memberId: n }, | ||
| }); | ||
| await tx.finding.updateMany({ | ||
| where: { createdById: o }, | ||
| data: { createdById: n }, | ||
| }); | ||
|
|
||
| // FrameworkSyncOperation: performedById | ||
| await tx.frameworkSyncOperation.updateMany({ | ||
| where: { performedById: o }, | ||
| data: { performedById: n }, | ||
| }); | ||
|
|
||
| // Comment: memberId | ||
| await tx.comment.updateMany({ | ||
| where: { authorId: o }, | ||
| data: { authorId: n }, | ||
| }); | ||
|
|
||
| // AuditLog: memberId | ||
| await tx.auditLog.updateMany({ | ||
| where: { memberId: o }, | ||
| data: { memberId: n }, | ||
| }); | ||
|
|
||
| // Device: memberId | ||
| await tx.device.updateMany({ | ||
| where: { memberId: o }, | ||
| data: { memberId: n }, | ||
| }); | ||
|
|
||
| // BackgroundCheckRequest: unique (organizationId, memberId) — delete old if new member already has one | ||
| const newBgCheck = await tx.backgroundCheckRequest.findUnique({ | ||
| where: { organizationId_memberId: { organizationId, memberId: n } }, | ||
| select: { id: true }, | ||
| }); | ||
| if (newBgCheck) { | ||
|
chasprowebdev marked this conversation as resolved.
Outdated
|
||
| await tx.backgroundCheckRequest.deleteMany({ where: { memberId: o } }); | ||
|
chasprowebdev marked this conversation as resolved.
Outdated
|
||
| } else { | ||
| await tx.backgroundCheckRequest.updateMany({ | ||
| where: { memberId: o }, | ||
| data: { memberId: n }, | ||
| }); | ||
| } | ||
|
|
||
| // TrustAccessRequest: reviewerMemberId | ||
| await tx.trustAccessRequest.updateMany({ | ||
| where: { reviewerMemberId: o }, | ||
| data: { reviewerMemberId: n }, | ||
| }); | ||
|
|
||
| // TrustAccessGrant: issuedByMemberId, revokedByMemberId | ||
| await tx.trustAccessGrant.updateMany({ | ||
| where: { issuedByMemberId: o }, | ||
| data: { issuedByMemberId: n }, | ||
| }); | ||
| await tx.trustAccessGrant.updateMany({ | ||
| where: { revokedByMemberId: o }, | ||
| data: { revokedByMemberId: n }, | ||
| }); | ||
|
|
||
| // OffboardingChecklistCompletion: unique (memberId, templateItemId) — skip items new member already has | ||
| const existingChecklistCompletions = | ||
| await tx.offboardingChecklistCompletion.findMany({ | ||
| where: { memberId: o }, | ||
| select: { id: true, templateItemId: true }, | ||
| }); | ||
| const newChecklistTemplateItemIds = new Set( | ||
| ( | ||
| await tx.offboardingChecklistCompletion.findMany({ | ||
| where: { memberId: n }, | ||
| select: { templateItemId: true }, | ||
| }) | ||
| ).map((c) => c.templateItemId), | ||
| ); | ||
| const checklistToMigrate = existingChecklistCompletions.filter( | ||
| (c) => !newChecklistTemplateItemIds.has(c.templateItemId), | ||
| ); | ||
| const checklistToDrop = existingChecklistCompletions.filter((c) => | ||
|
chasprowebdev marked this conversation as resolved.
Outdated
|
||
| newChecklistTemplateItemIds.has(c.templateItemId), | ||
| ); | ||
| if (checklistToMigrate.length > 0) { | ||
| await tx.offboardingChecklistCompletion.updateMany({ | ||
| where: { id: { in: checklistToMigrate.map((c) => c.id) } }, | ||
| data: { memberId: n }, | ||
| }); | ||
| } | ||
| if (checklistToDrop.length > 0) { | ||
| await tx.offboardingChecklistCompletion.deleteMany({ | ||
| where: { id: { in: checklistToDrop.map((c) => c.id) } }, | ||
| }); | ||
| } | ||
|
|
||
| // OffboardingAccessRevocation: unique (memberId, vendorId) — skip vendors new member already has | ||
| const existingRevocations = | ||
| await tx.offboardingAccessRevocation.findMany({ | ||
| where: { memberId: o }, | ||
| select: { id: true, vendorId: true }, | ||
| }); | ||
| const newRevocationVendorIds = new Set( | ||
| ( | ||
| await tx.offboardingAccessRevocation.findMany({ | ||
| where: { memberId: n }, | ||
| select: { vendorId: true }, | ||
| }) | ||
| ).map((r) => r.vendorId), | ||
| ); | ||
| const revocationsToMigrate = existingRevocations.filter( | ||
| (r) => !newRevocationVendorIds.has(r.vendorId), | ||
| ); | ||
| const revocationsToDrop = existingRevocations.filter((r) => | ||
| newRevocationVendorIds.has(r.vendorId), | ||
| ); | ||
| if (revocationsToMigrate.length > 0) { | ||
| await tx.offboardingAccessRevocation.updateMany({ | ||
| where: { id: { in: revocationsToMigrate.map((r) => r.id) } }, | ||
| data: { memberId: n }, | ||
| }); | ||
| } | ||
| if (revocationsToDrop.length > 0) { | ||
| await tx.offboardingAccessRevocation.deleteMany({ | ||
| where: { id: { in: revocationsToDrop.map((r) => r.id) } }, | ||
| }); | ||
| } | ||
| await tx.offboardingAccessRevocation.updateMany({ | ||
| where: { revokedById: oldMember.userId }, | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| data: { revokedById: newMember.userId }, | ||
| }); | ||
|
|
||
| // EmployeeTrainingVideoCompletion: skip videos the old member already has | ||
| const existingCompletions = | ||
| await tx.employeeTrainingVideoCompletion.findMany({ | ||
| where: { memberId: o }, | ||
| select: { id: true, videoId: true }, | ||
| }); | ||
|
|
||
| const newCompletions = | ||
| await tx.employeeTrainingVideoCompletion.findMany({ | ||
| where: { memberId: n }, | ||
| select: { id: true, videoId: true }, | ||
| }); | ||
| const newCompletedVideoIds = new Set( | ||
| newCompletions.map((c) => c.videoId), | ||
| ); | ||
|
|
||
| const toMigrate = existingCompletions.filter( | ||
| (c) => !newCompletedVideoIds.has(c.videoId), | ||
| ); | ||
|
|
||
| if (toMigrate.length > 0) { | ||
| await tx.employeeTrainingVideoCompletion.updateMany({ | ||
| where: { id: { in: toMigrate.map((c) => c.id) } }, | ||
| data: { memberId: n }, | ||
| }); | ||
| } | ||
|
|
||
| logger.info('Re-pointed member relations', { | ||
| policiesWithSignature: policiesWithSignature.length, | ||
| trainingMigrated: toMigrate.length, | ||
| trainingDropped: existingCompletions.length - toMigrate.length, | ||
| }); | ||
|
|
||
| // SOADocument / IsmsDocument: approverId (SetNull on delete — re-point to preserve assignments) | ||
| await tx.sOADocument.updateMany({ | ||
| where: { approverId: o }, | ||
| data: { approverId: n }, | ||
| }); | ||
| await tx.ismsDocument.updateMany({ | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| where: { approverId: o }, | ||
| data: { approverId: n }, | ||
| }); | ||
|
|
||
| // ── Delete old member ──────────────────────────────────────────────── | ||
| await tx.member.delete({ where: { id: o } }); | ||
|
chasprowebdev marked this conversation as resolved.
|
||
|
|
||
| // ── Re-point user-level relations before deleting oldUser ─────────── | ||
| // Must happen before the delete to prevent cascade wiping these records. | ||
|
|
||
| // OAuth accounts: move to surviving user | ||
| await tx.account.updateMany({ | ||
|
chasprowebdev marked this conversation as resolved.
Outdated
|
||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // AuditLog: onDelete Cascade — re-point to preserve history | ||
| await tx.auditLog.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // FleetPolicyResult: onDelete Cascade — re-point to preserve results | ||
| await tx.fleetPolicyResult.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // OauthAccessToken: onDelete Cascade | ||
| await tx.oauthAccessToken.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // OauthConsent: onDelete Cascade | ||
| await tx.oauthConsent.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // McpOrgBinding: onDelete Cascade, unique on userId — delete old, keep new | ||
| await tx.mcpOrgBinding.deleteMany({ where: { userId: oldUser.id } }); | ||
|
|
||
| // IntegrationSyncLog / IntegrationOAuthError: nullable userId — re-point to preserve actor | ||
| await tx.integrationSyncLog.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
| await tx.integrationOAuthError.updateMany({ | ||
| where: { userId: oldUser.id }, | ||
| data: { userId: newUser.id }, | ||
| }); | ||
|
|
||
| // ── Delete old user sessions ───────────────────── | ||
| await tx.session.deleteMany({ where: { userId: oldUser.id } }); | ||
|
|
||
| // ── Update pending invitations ─────────────────────────────────────── | ||
| await tx.invitation.updateMany({ | ||
| where: { email: oldEmail, organizationId }, | ||
| data: { email: newEmail }, | ||
| }); | ||
| }, | ||
| { timeout: 30000 }, | ||
| ); | ||
|
|
||
| logger.info('Merge complete', { | ||
| organizationId, | ||
| oldEmail, | ||
| newEmail, | ||
| survivingUserId: oldUser.id, | ||
| survivingMemberId: oldMember.id, | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
Outdated
|
||
| }); | ||
|
|
||
| return { | ||
| success: true, | ||
| survivingUserId: newUser.id, | ||
| survivingMemberId: newMember.id, | ||
| mergedUserId: oldUser.id, | ||
| mergedMemberId: oldMember.id, | ||
| }; | ||
| }, | ||
| }); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.