Skip to main content

Deploy & Rollback Runbook

CalGest ships with release-please driving versioned, per-component deploys.
  1. Conventional-commit PRs merge to main.
  2. The Release Please workflow (.github/workflows/release-please.yml) keeps one rolling release PR open, updating each component’s version + CHANGELOG.md.
  3. Merging that PR tags the bumped components (<component>@X.Y.Z) and creates a GitHub Release for each.
  4. The same workflow then dispatches the Deploy workflow (.github/workflows/deploy.yml) once per released component — pinned to the new tag — sequentially, in order backend → cdn → dashboard → web → mobile, waiting for each to finish before starting the next. If a deploy fails, the remaining components are skipped (backend is always live before its clients).
Deploy is workflow_dispatch only (component + version inputs): a merge to main never deploys a surface by itself — only the release workflow (or an operator) dispatches it. Each dispatch checks out the <component>@<version> tag, re-runs the full preflight (types, lint, test, build, doctor, audit), then runs the component’s deploy + a post-deploy smoke test — if the app isn’t serving, the deploy job fails and turns red. Manual deploy / rollback: dispatch Deploy with the target component and an already-released version (e.g. an older tag) to roll a surface back to that version. Convex backends do not truly reverse (migrations don’t unwind) — a backend “rollback” is a roll-forward redeploy of an earlier good version; see the Convex section below. This runbook covers how to tell something is broken and how to roll back each surface fast.

Surfaces

SurfacePlatformDeployed byProduction URL
BackendConvexconvex deploy*.convex.site (HTTP host)
DashboardCloudflare (Vite)pnpm --filter dashboard run deployhttps://dashboard.calgest.com
WebCloudflare (Vite)pnpm --filter web run deployhttps://calgest.com
CDNCloudflare Workerwrangler deployhttps://cdn.calgest.com
MobileExpo EAS Updateeas update --branch production --environment productionOTA channel production

How to tell it’s broken

  1. The deploy pipeline is red. A failed smoke-test step is the first signal — check the failing job’s log for the ::error:: line.
  2. Manual smoke checks (same as CI — run any of these locally):
    curl -fsS https://<your-deployment>.convex.site/healthz          # expect {"status":"ok"}
    curl -s -o /dev/null -w '%{http_code}\n' https://dashboard.calgest.com   # expect 200
    curl -s -o /dev/null -w '%{http_code}\n' https://calgest.com            # expect 200
    curl -s -o /dev/null -w '%{http_code}\n' https://cdn.calgest.com/       # expect non-5xx (404)
    
  3. Uptime / error alerting (Sentry, uptime monitor) firing on the surface.
If a deploy job stayed green but the app is misbehaving, roll back the affected surface below, then investigate.

Rollback — happy path (~2 min per surface)

Cloudflare — Dashboard & Web (Vite deploy)

Both are Cloudflare Workers/Pages-style deployments driven by Wrangler.
# From the app dir (apps/dashboard or apps/web):
cd apps/dashboard      # or apps/web
npx wrangler deployments list            # find the last-good deployment ID
npx wrangler rollback [<deployment-id>]  # omit ID to roll back to the previous version
  • wrangler rollback with no ID rolls back to the immediately previous deployment.
  • Dashboard UI path: Cloudflare dashboard → Workers & Pages → select calgest-dashboard / calgestDeployments → find the last-good version → ⋯ → Rollback.
  • Re-run the smoke curl above to confirm 200.

Cloudflare — CDN Worker

cd apps/cdn
npx wrangler deployments list
npx wrangler rollback [<deployment-id>]        # production
npx wrangler rollback [<deployment-id>] --env dev   # dev worker, if needed
Confirm with curl -s -o /dev/null -w '%{http_code}\n' https://cdn.calgest.com/ (expect a non-5xx, normally 404).

Convex — Backend

Convex has no in-place “rollback” button; you roll forward by re-deploying a known-good commit.
# Fastest: redeploy the previous good commit.
git checkout <last-good-sha>
pnpm --filter @packages/backend exec convex deploy   # needs CONVEX_DEPLOY_KEY
git checkout main
  • Inspect deploy history & logs: Convex dashboard → your deployment → Functions / Logs / History to identify the last-good version and any failing function.
  • If the break is a bad schema/migration rather than code, coordinate with the backend owner — a forward-fix migration is usually safer than reverting schema.
  • Confirm with curl -fsS https://<deployment>.convex.site/healthz{"status":"ok"}.

Mobile — Expo EAS Update (OTA)

EAS Updates are immutable; you “roll back” by republishing a previous update to the production channel so clients pull the good bundle on next launch.
cd apps/mobile
eas update:list --branch production          # find the last-good update group ID
eas update:republish --group <good-group-id> --branch production
Alternative: eas channel:edit production --branch <good-branch> to point the production channel at a different branch entirely. Native binary changes cannot be rolled back via OTA — those require a new store build.
  • CDN isolation. Production uses the EU calgest-prod R2 bucket and dev uses calgest-dev. Create calgest-prod before the first deployment. The production workflow only writes the production worker’s ASSET_TOKEN_SECRET; configure a separate secret directly on calgest-cdn-dev when deploying the dev worker.
  • Environment protection. Each deploy job still targets the production GitHub Environment. Add a required reviewer if the repository plan supports it; manual workflow dispatch is the enforced fallback gate.
  • Staging (optional). There is currently no staging environment. If desired, add a parallel set of deploy targets (a staging Convex deployment, *-staging Cloudflare workers, a staging EAS channel) and a staging GitHub Environment, then promote to production after staging smoke tests pass.