All writing

ArchitectureBy 3 min read

Moving 16 microservices from Kubernetes to Cloud Run, and cutting the bill by 40%

We ran 16 web microservices on Kubernetes around the clock, including the hours nobody used them. Moving them to Cloud Run, where an idle service scales to zero, cut infrastructure cost by 40%. What made a service a good fit, and what to check before you move yours.

Updated

On this page · 6 sections
  1. What we were paying for
  2. Why Cloud Run fit
  3. How to check if a service is a good fit
  4. Cold starts are the trade-off
  5. What I would tell a team planning the same move
  6. Questions

As Principal Software Engineer I at Zemoso, I led the move of 16 web microservices from Kubernetes to Cloud Run on Google Cloud. Infrastructure cost came down by 40%. I can’t share the client’s bill, so that percentage is the only figure in this post. What I can share is why the saving happened and how to tell whether your own services would see the same.

What we were paying for

All 16 services ran on a Kubernetes cluster. Kubernetes is good at keeping things running, which in our case was exactly the problem: the nodes stayed up all day and all night, sized for the busiest hour, while most of the services were quiet for long stretches. Nights, weekends and the gaps between bursts of traffic still cost the same as peak time.

None of these services needed a cluster to themselves. They were web services: a request comes in, the service does its work, a response goes out.

Why Cloud Run fit

Cloud Run runs a container image and starts instances only when requests arrive. When the requests stop, it scales the service down, all the way to zero if you let it. A service with no traffic costs almost nothing.

For our 16 services, that changed the shape of the bill. We stopped paying for capacity that might be needed and paid for the capacity we actually used, which is where most of the 40% came from.

The services were already containerised, which made the move far easier. Cloud Run takes the same images. The main change on the code side is that each service has to listen on the port Cloud Run passes in the PORT environment variable.

How to check if a service is a good fit

Before moving a service, I would ask four questions about it:

  • Is it stateless? Cloud Run can stop an instance at any time, so anything kept in local memory or on local disk between requests will be lost. Sessions, caches and files belong in a database, Memorystore or Cloud Storage.
  • Is it request-driven? A service that only does work while answering HTTP requests fits well. A worker that pulls from a queue all day, or holds a connection open for hours, does not.
  • Does it start quickly? With scale to zero, the first request after a quiet period waits for a container to start. A service that takes 30 seconds to boot will make that first user wait 30 seconds.
  • Does it have quiet periods? Scale to zero only saves money when there is idle time. A service that is busy around the clock will save much less.

A service that fails one of these is not ruled out, but it needs a plan before it moves.

Cold starts are the trade-off

Scaling to zero costs you some latency. If a user-facing service must answer fast at all times, set a minimum of one instance for it. You pay a small fixed amount for that service and keep scale to zero everywhere else. Internal services and anything called by a scheduled job can usually live with the occasional slow first request.

What I would tell a team planning the same move

Look at when your services are idle before you look at what they cost.

If your Kubernetes nodes are busy all day, Cloud Run may not save you much. If your services spend long hours doing nothing, as ours did, you are paying for that idle time, and scale to zero is the most direct way to stop.

Questions

Why does moving from Kubernetes to Cloud Run save money?

A Kubernetes cluster runs its nodes whether or not traffic arrives, so quiet services still cost money. Cloud Run starts container instances when requests come in and can scale a service down to zero when they stop, so you pay closer to actual use.

Which microservices are a good fit for Cloud Run?

Stateless HTTP services that keep no local data between requests, start quickly and have quiet periods. Long-running background workers, services that hold connections open for hours, or anything that needs a persistent local disk are a poorer fit.

What is the catch with scale to zero?

Cold starts. The first request after an idle period waits for a container to start. For a user-facing service where that delay matters, set a minimum of one instance and accept a small fixed cost for that service.