Skip to content

Rollouts

Use the Rollouts page to follow changes that Watchtower delivers to many Edges in steps, and to pause, resume, stop or revert them.

A rollout limits the damage of a bad change: it updates a few Edges first and pauses when too many fail.

When a change becomes a rollout

You do not create rollouts on this page. Watchtower creates one for you:

Change Delivery
Saving one Edge's Config tab At once
A group, site or secret change that affects 5 Edges or fewer At once
A group, site, organization or secret change that affects more than 5 Edges Rollout
Publish on a template used by at least one Edge Rollout
Roll out version on the Updates page Rollout

How a rollout proceeds

Watchtower releases the change in three steps: 10 %, then 50 %, then 100 % of the Edges. The same Edges go first each time.

  • Within a step, Watchtower releases up to 20 Edges per minute.
  • When every online Edge in a step has finished, Watchtower waits 15 minutes before the next step.
  • Offline Edges do not hold a step back. They receive the change when they reconnect.
  • If more than 10 % of the Edges released in a step end Failed or Rolled back, the rollout pauses.

Read the page

Each rollout shows a card with:

  • The description, for example template <id> version 7.
  • A status line: state, current step and scope (template, org, site, group or device).
  • One bar per step, with counts such as 3 in sync · 1 failed · 2 waiting · 6. Green is in sync, red is failed or rolled back, amber is waiting.
State Meaning
running Steps are progressing.
paused Stopped by a user, or automatically after too many failures.
completed Every step finished.
aborted Stopped for good.

No rollouts. means none exist yet. No steps recorded yet. means the rollout has just started.

Control a rollout

Owners and Admins can use the buttons on each card. If an action does not apply to the rollout's current state, an error appears at the top of the page.

Button Effect
pause Stops releasing Edges. Edges already updated keep the change.
resume Continues a paused rollout. Only one rollout per scope runs at a time, so any other running rollout for the same scope pauses.
abort Ends the rollout. Edges already updated keep the change.
skip-wait Starts the next step as soon as the online Edges of the current step finish.
revert For a paused, aborted or completed template rollout only: sets the groups and Edges that used this version back to the previous published version, and starts a new rollout of it. Fails if there is no earlier published version.

To undo a group, site or organization change, change that layer back. Watchtower then delivers the correction.

Respond to a paused rollout

  1. On Rollouts, find the step that paused and its failed count.
  2. On Edges, set the sync filter to Not in sync. Hover the Sync cell of each failed Edge to read the error.
  3. Fix the cause. Then do one of:
    • Publish a corrected template version. This starts a new rollout.
    • Click revert on a template rollout.
    • Click resume if the failures were unrelated to the change, for example Edges that lost power.