Repository navigation
Description
Steps to reproduce
- On dstack 0.22.5 with a gateway, apply a new service that scales to zero:
type: service name: srv image: nginx port: 80 replicas: 0..1 scaling: metric: rps target: 1
- Send no requests to it.
Actual behaviour
A replica is provisioned straight away. In our case, a 0..1 RTX 5090 service submitted at 10:12:11 UTC had a job submitted and an instance created 14 seconds later, without any request. The replica then ran until scale_down_delay passed and was scaled down (SCALED_DOWN). After that the service behaved as expected: it stayed at zero until a request arrived.
An in-place update of an existing service does not do this. Only a new run does.
Expected behaviour
A new service with replicas: 0..N stays at zero replicas until the first request, as described in #3003.
dstack version
0.22.5 (server and gateway)
Server logs
No response
Additional information
The cause appears to be the placeholder replica count combined with the pre-0.20.7 compatibility branch:
submit_run(server/services/runs/__init__.py) creates every run withdesired_replica_count=1 # a relevant value will be set in RunPipeline, and leavesdesired_replica_countsempty.- When the pending run is first processed,
_process_pending_service(background/pipeline_tasks/runs/pending.py) callscompute_desired_replica_counts(background/pipeline_tasks/runs/common.py). - There,
desired_replica_countsis empty and the service has the single default replica group. So the special case for services created by a pre-0.20.7 server applies:prev_counts = {DEFAULT_REPLICA_GROUP_NAME: run_model.desired_replica_count}, which is the placeholder1. - The gateway has no stats for a service that was just registered. So
RPSAutoscaler.get_desired_countreturnscurrent_desired_countas is (if not stats: return current_desired_count), which is 1. total == 1, so the run is submitted with one replica.
Two possible fixes, either of which should be enough:
- Apply the compatibility branch only to runs that already have jobs, for example
prev_counts == {} and run_model.jobs and .... A service created by a pre-0.20.7 server had jobs by the time a 0.20.7+ server processed it, while a new run has none. - Or, for services, create the run with
desired_replica_countset to the replica minimum (0 for0..N) instead of the placeholder 1.
As hardening, the no-stats return in RPSAutoscaler.get_desired_count could also be clipped to [min_replicas, max_replicas], like the other branches.
Impact: every new scale-to-zero service costs one unrequested cold start. For a GPU service that means the full image pull, weight download and warm-up (about 20 minutes for us), plus the scale_down_delay.
I'm happy to send a PR if one of these fixes looks right to you.
Activity
peterschmidt85 commentedon Oct 27, 2026
cc @jvstme
jvstme commentedon Oct 29, 2026
Hi @vitostrokov, thanks for reporting. I believe #4433 fixes the bug. The new
dstackrelease with the fix will most likely be available later this week, please give it a try when you have a chance