GitHub Workflow error: Canceling since a higher priority waiting request for [a concurrency group] exists

by

Hi, in this post, I’ll share learning from my recent attempt to set up GitHub workflow in a way that:

  • At most, one workflow runs at a time,
  • The workflows starting later are queued in a FIFO manner and patiently wait for their time.

We wanted to achieve such a setup to allow running end-to-end tests on a single Azure environment without causing interference between workflows running simultaneously.

An attempt that failed

I expected it to be as easy as specifying a concurrency group in a GitHub Workflow, with something like:

on: workflow_dispatch

concurrency:
  group: workflows-sharing-e2e-environment
  cancel-in-progress: false

Code language: YAML (yaml)

At first glance, this appeared to have worked! When we started two workflow runs, the first started, and the other was patiently waiting in a queue. We happily merged the solution.

Workflows getting canceled

We soon realized, that contrary to the intention, some workflows weren’t queued but canceled with an error like:

❌ Deploy trunk to dev: .github#L1
Canceling since a higher priority waiting request for ‘workflows-sharing-e2e-environment’ exists

So, what is happening?

It turned out, that contrary to our intuition, and also the intuition of many other developers reporting this issue, setting concurency group on a workflow doesn’t cause the workflows to be queued, but canceled by default.

Now, you might notice there is this parameter, cancel-in-progress: false, that should solve the issue, right? Unfortunately not, it only makes exceptions for the workflow in progress, but not for pending workflows. So instead of the FIFO queue we expected, we’ve built a kind of queue that can queue at most 1 workflow 🤔. Let me try to explain it with a picture:

Workarounds?

Unfortunately, I didn’t find any practical workaround at the moment. Of course, there is always some way of coding a custom solution, e.g., to monitor the status of workflows and re-schedule the ones that got canceled, but it’s a high-effort one.

A request from the community that I find reasonable as well as to extend the concurrency group configuration with a new flag like:

on: workflow_dispatch

concurrency:
  group: workflows-sharing-e2e-environment
  cancel-in-progress: false
  cancel-pending: false # such option doesn't exist at the moment
Code language: PHP (php)

This blog post doesn’t provide any solution or workaround, but I hope it helped explain the issue, and maybe it will help you progress anyway. Thanks for stopping by!

Leave a Comment