Repository navigation
Conversation
The web-server backup copied every service's */code/ directory, the git working tree cloned at deploy time, which can add gigabytes per service and is re-created on the next deploy. Exclude applications/*/code/ and compose/*/code/, but keep the code of drop (zip upload) applications, since their code/ directory is the only copy. Closes Dokploy#5731 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| (appName) => `--include=${quote([`applications/${appName}/code/***`])}`, | ||
| ), | ||
| "--exclude='applications/*/code/'", | ||
| "--exclude='compose/*/code/'", |
There was a problem hiding this comment.
The blanket Compose exclusion removes live files, not just cloned source. A raw Compose service with ./data:/var/lib/postgresql/data stores its database under compose/<appName>/code/data. Those files cannot be rebuilt from the saved Compose text. Restore deletes the existing data path before copying the archive, so this change loses that database on restore.
Preserve bind-mounted paths under code/ instead of excluding the whole directory.
Knowledge Base Used: Backups and restore
There was a problem hiding this comment.
You're right. Restore runs rm -rf "$BASE_PATH/"* before copying, so excluding compose/*/code/ would delete a running service's ./data on a same-server restore. Fixed in fd7533f: the backup now keeps every working tree and only excludes applications/*/code/.git/ and compose/*/code/.git/, which every deploy re-clones. The rsync test now asserts that a compose/stack/code/data file is kept. I updated the PR description to explain the narrower scope.
| const dropApps = await db.query.applications.findMany({ | ||
| columns: { appName: true }, | ||
| where: eq(applications.sourceType, "drop"), | ||
| }); |
There was a problem hiding this comment.
The drop-app list is read after pg_dump, so it can disagree with the saved database. If a user switches a drop app to Git while the dump runs, the dump can still contain sourceType: "drop" while this query omits the app. The new exclusion then skips its uploaded code even if those files remain untouched. Restoring that backup brings back a drop app with no source to redeploy.
Use the dump's snapshot for the include list, or conservatively keep code for apps whose source changes during the backup.
There was a problem hiding this comment.
Resolved by fd7533f: with only .git excluded there's no drop-app lookup anymore, so nothing depends on database state and there's no race with pg_dump. Drop apps' code is kept like everything else.
Excluding whole */code/ directories dropped live data: compose bind mounts like ./data resolve inside code/, and restore replaces BASE_PATH, so a same-server restore would delete that data. Keep the working trees and exclude only */code/.git/, which every deploy re-clones. This also removes the drop-app lookup, so there is no race with the database dump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gourav224
changed the title
# Conflicts: # packages/server/src/utils/backups/web-server.ts
|
Could we allow users to provide their own backup exclusion patterns ? This way, if one actually want to optimise backup size, it is its responsability to verify what is excluded is not required for a restore (like in my case when you never use Drop nor compose local volumes). Thank you anyway for the quick answer on this subject and for the hard and awesome work on Dokploy! |
|
This was already fixed. Thank you for your time. @EmrysMyrddin that can be done in another time. For now just doing two specific paths. If more people are in need of this feature we can come back to this. Thank you |
What is this PR about?
The web-server backup rsyncs all of
BASE_PATH, including the git checkouts Dokploy clones at deploy time underapplications/*/code/andcompose/*/code/(#5731). Their.githistory can be huge: in the report it was 1.5 GB of the 3 GB a single service added.This PR excludes
applications/*/code/.git/andcompose/*/code/.git/from the backup. That history is never needed after a restore. Every deploy runsrm -rf <code>and clones again (the git providers and raw compose all do this).Why not exclude all of
*/code/, as the issue suggests? An earlier version of this PR did, and review showed it isn't safe:./data:/var/lib/postgresql/dataresolves insidecompose/<app>/code/, socode/can hold a running service's live data.restoreWebServerBackuprunsrm -rf "$BASE_PATH/"*before copying the archive back. A same-server restore would delete that data and never bring it back.applications/<app>/code/.The working tree therefore stays in the backup, and only
.gitis skipped. That still removes the largest reproducible part. The tracked files themselves (1.7 GB in the report) can't be told apart from bind-mounted data without parsing every compose file's volumes, so they're out of scope here.Checklist
Before submitting this PR, please make sure that:
canarybranch.Testing: the new
__test__/backups/web-server-filters.test.tsruns the realrsyncwith the excludes on a sampleBASE_PATH. Both.gitdirectories are skipped. The application working tree, a drop app's code, a composecode/databind mount,files/andtraefik/are all kept. It is skipped ifrsyncisn't installed. All__test__/backupstests andtsc --noEmitforpackages/serverpass.Issues related (if applicable)
Partially addresses #5731 (the
.gitpart; see above for why the working tree is kept).🤖 Generated with Claude Code