DIKA
GET IN TOUCH

Vessel Deploy

Last updated Apr 15, 20262025

Vessel is the deployment path for a set of services that previously shipped by following a wiki page. Same services, same infrastructure — the change was making the release itself an artifact instead of a ritual.

Problem

Releasing took about two hours of one engineer's attention and could only be done by three people who knew the undocumented steps.

  • Environments drifted. Staging and production were configured by hand months apart, so "works on staging" carried little information.
  • Rollback was a redeploy of the previous tag, which meant rollback took as long as a deploy — during an incident, that's the worst possible time.
  • No one could answer "what's in production?" without reading commit logs and guessing.

Architecture

Two ideas: infrastructure defined in version control, and immutable images promoted between environments rather than rebuilt per environment.

Build once, tag by digest

CI builds a container image per commit and records its digest. Nothing downstream ever rebuilds — the artifact that passed tests is the artifact that runs.

Promote by moving a pointer

Deploying to an environment updates that environment's image digest in a config repo. The change is a diff, reviewable and revertable.

Reconcile from the repo

A controller applies whatever the config repo says. Deploy and rollback are the same operation in opposite directions.

Verify automatically

Health checks and smoke tests gate the promotion; a failing gate reverts the pointer without a human deciding.

# Environments differ only in variables, never in resource shape.
module "service" {
  source = "../../modules/service"
 
  name         = "atlas-api"
  image_digest = var.image_digest # set by the promotion pipeline
  replicas     = var.replicas     # 2 in staging, 6 in production
  cpu_limit    = var.cpu_limit
}

Note

Terraform owns infrastructure; the config repo owns which version runs. Keeping those separate is what makes a rollback a one-line diff instead of a plan and apply.

Stack

GitHub Actions

Build, test, image publish, and promotion workflows.

Terraform

Environment definitions as modules with per-environment variables only.

Docker + registry digests

Immutable artifacts promoted across environments, never rebuilt.

OpenTelemetry

Deploy markers on dashboards, so a regression is attributable to a release.

Results

MetricBeforeAfter
Time to release~2 hours, attended10 min, unattended
Time to roll back~2 hours90 s
Engineers able to release3everyone on the team
Environment config driftunknownzero by construction

What I'd do differently

I automated the deploy before adding deploy markers to the dashboards. For two weeks we could ship quickly but couldn't correlate a latency change with the release that caused it — faster deploys made that correlation more important, not less. Observability should have shipped first.

I'd also start with one service instead of four. The abstraction I extracted from four simultaneous migrations had parameters that existed only because one service was unusual, and unpicking that later cost more than the parallelism saved.