Skip to content

feat: Scheduled Cleanup of Build Logs for all Deployments - #66

Open
alronova wants to merge 2 commits into
mdgspace:developfrom
alronova:feat/optimize-logs
Open

feat: Scheduled Cleanup of Build Logs for all Deployments#66
alronova wants to merge 2 commits into
mdgspace:developfrom
alronova:feat/optimize-logs

Conversation

@alronova

@alronova alronova commented Jun 8, 2026

Copy link
Copy Markdown
Member

Feat: Scheduled Log Cleanup for Build Logs

Problem

Build logs introduced in #63 are written via tee -a in container.sh and automate.sh, causing {subdomain}.log files to grow indefinitely across deployments and CI/CD redeployments.

This leads to:

  • Unbounded log growth — Frequently redeployed subdomains can accumulate logs far larger than the 100KB ever served by the frontend.
  • Orphaned files — Failed subdomain deletions can leave .log and .status files behind indefinitely.

Solution

Added a scheduled cleanup module (log-cleanup.ts) that runs every 24 hours (following the existing health-monitor.ts scheduler pattern) and performs two cleanup strategies.

1. Log Truncation

For each active subdomain log in /hostpipe/logs/:

if file_size > 500KB:
    keep only the last 500KB

Implementation:

  1. Open the file for reading.
  2. Seek to file_size - 500KB.
  3. Read the last 500KB into memory.
  4. Overwrite the file with that content.

This preserves recent deployment history while preventing unlimited growth. The existing getLogs API continues to serve the last 100KB via readLastNBytes(), while 500KB provides additional debugging context between cleanup cycles.

2. Orphan Cleanup

For every .log and .status file in /hostpipe/logs/ and /hostpipe/status/:

extract subdomain from filename
if subdomain not found in MongoDB:
    delete file

This removes artifacts left behind by incomplete deletions. Orphan removals are reported to Sentry for visibility and auditing.

Scheduler Behavior

  • First run: 10 seconds after server startup (setTimeout)
  • Subsequent runs: every 24 hours (setInterval)
  • Per-file error isolation via individual try/catch
  • Missing directories are handled gracefully and skipped

Checklist

  • Backward Compatibility is maintained
  • The code implementation has been tested on the staging server and all features are working as expected
  • Commits are Signed-off

alronova added 2 commits June 8, 2026 19:28
- truncate log files for those exceeding 500kb file size
- cleanup of `.log` & `.status` files for subdomains that don't exist on the DB (failed deployments)

Signed-off-by: alronova <aditya.divyam001@gmail.com>
Signed-off-by: alronova <aditya.divyam001@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant