# Flask CI on GitHub Actions: build times and cost per run on GitHub vs Sengi runners (Nov 2026)

Source: https://sengi.run/benchmarks/flask

By Sengi · Last checked November 17, 2026

## How long does Flask CI take on GitHub Actions, and what does a build cost?

Measured November 17, 2026 on Flask (pallets/flask, commit 22d9247): Python 3.13 test suite (tox). Results at 4 vCPU, GitHub-hosted 4 vCPU took 22 s median over 4 runs, $0.012 a build; Sengi Basic 4 vCPU took 14 s median over 3 runs, $0.002 a build, 35% less time and 83% less cost against GitHub-hosted 4 vCPU; Sengi Premium 4 vCPU took 14 s median over 3 runs, $0.006 a build, 35% less time and 50% less cost against GitHub-hosted 4 vCPU. No Sengi runner of GitHub’s size (4 vCPU) took more median time than GitHub-hosted 4 vCPU on Flask where there were enough runs to compare. Times are median job seconds, printed only with at least 3 runs; these results cover this build only. No Flask job failed in this round.

Measured November 17, 2026, last checked November 17, 2026. This page shows one project from the [12-project benchmark](https://sengi.run/benchmarks). Every number is in [one JSON file](https://sengi.run/data/github-actions-benchmarks.json) and [a CSV file](https://sengi.run/data/github-actions-benchmarks.csv), and every median links to its run log in the public repository [sengirunners/benchmarks](https://github.com/sengirunners/benchmarks/tree/e73c896795525a61b47ce9f44657a1a045b0a98b).

## Flask results at 4 vCPU: GitHub’s runner, Sengi Basic 4 vCPU and Sengi Premium 4 vCPU

One row per runner. Time is the median job run time in seconds, from the job’s start to its end, over the runs counted; min and max are the fastest and slowest of those runs. Cost per build is the median of the runs’ billed minutes, each job rounded up to a whole minute, at the runner’s list price. A cell with fewer than 3 runs prints “not enough runs”.

**Flask: median job time, run count and cost per build at 4 vCPU, measured November 17, 2026**

| Project | Commit | Runner | Runs | Median time (s) | Min–max (s) | Cost per build (USD) | Time against GitHub | Cost against GitHub |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Flask | `22d9247` | GitHub-hosted 4 vCPU | 4 | [22](https://github.com/sengirunners/benchmarks/actions/runs/44059823560/job/133964347514) | 18–29 | $0.012 | baseline | baseline |
| Flask | `22d9247` | Sengi Basic 4 vCPU | 3 | [14](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133989236037) | 13–16 | $0.002 | 35% less time | 83% less cost |
| Flask | `22d9247` | Sengi Premium 4 vCPU | 3 | [14](https://github.com/sengirunners/benchmarks/actions/runs/44057933305/job/133957021121) | 13–16 | $0.006 | 35% less time | 50% less cost |

## Flask median job time at 4 vCPU

Flask median job time in seconds, measured November 17, 2026. Sengi runners in blue, GitHub-hosted 4 vCPU in grey. The same rows as the table above.

## Not like-for-like: Sengi Basic 2 vCPU and Sengi Premium 2 vCPU

Sengi Basic 2 vCPU has half the cores of GitHub’s 4 vCPU runner. Sengi Premium 2 vCPU has half the cores of GitHub’s 4 vCPU runner. These Sengi runners were run on the same project; they are not a like-for-like comparison, and no GitHub runner of their size was measured. The last two columns still set each against GitHub-hosted 4 vCPU.

**Flask, not like-for-like: median job time, run count and cost per build on Sengi Basic 2 vCPU (half the cores of GitHub’s 4 vCPU runner) and Sengi Premium 2 vCPU (half the cores of GitHub’s 4 vCPU runner), measured November 17, 2026**

| Project | Commit | Runner | Runs | Median time (s) | Min–max (s) | Cost per build (USD) | Time against GitHub | Cost against GitHub |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Flask | `22d9247` | Sengi Basic 2 vCPU | 4 | [19](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133930890110) | 16–21 | $0.001 | 14% less time | 92% less cost |
| Flask | `22d9247` | Sengi Premium 2 vCPU | 4 | [13](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133981028771) | 9–15 | $0.003 | 42% less time | 75% less cost |

## Where did Sengi take more time on Flask?

Every Flask result where a Sengi runner’s median was above GitHub’s is printed in the tables above and listed here.

### At 4 vCPU, the same size as GitHub’s runner

Nowhere on Flask at 4 vCPU in this round: wherever the runs were enough to compare, the median of each Sengi runner of GitHub’s size was at or below GitHub’s.

## What was left out of the Flask medians?

No Flask job failed in this round, so every measured Flask run is in a median above. The first run of each runner class is a warm-up that fills its dependency caches and is not measured.

## How was Flask measured?

- **Time:** Job run time, start to end, from the GitHub API. Queue time is kept apart.
- **Runs:** Median of the measured runs, with n, min and max; a median needs n >= 3.
- **Cost:** Billed minutes (each job rounded up to a whole minute) times the runner's list price per minute.
- **GitHub cost:** GitHub does not bill public repositories for this runner; its cost is what a private repository pays at GitHub's listed rate for the measured size.
- **Order:** One runner class at a time; at most four jobs at once within a run. One warm-up run per class first fills dependency caches.
- **Selection:** The project list is the whole workload list of an existing open-source CI benchmark suite, adopted before running it here. Each project is pinned to a release commit.
- **Exclusions:** Every run is listed. A job is left out of the medians only for the reason given next to it.
- **Dates:** 2026-11-17 to 2026-11-17.
- **Runner size:** the vCPU count printed for each runner was read with `nproc` inside the job, not taken from a label.
- **Prices:** GitHub-hosted 4 vCPU $0.012, Sengi Basic 2 vCPU $0.001, Sengi Premium 2 vCPU $0.003, Sengi Basic 4 vCPU $0.002 and Sengi Premium 4 vCPU $0.006 a minute. GitHub does not bill public repositories for its standard runner, which is 4 vCPU there. The GitHub column is priced at GitHub’s listed rate for its 4 vCPU Linux larger runner (sold to Team and Enterprise Cloud organizations), $0.012 a minute: what a private repository pays for these minutes beyond its plan’s included usage.
- **Queue time** is recorded for every run in the data files and is not part of the times above.

| Project | Repository | Language | What the job runs | Commit |
| --- | --- | --- | --- | --- |
| Flask | pallets/flask | Python | Python 3.13 test suite (tox) | `22d924701a6ae2e4cd01e9a15bbaf3946094af65` |

The benchmark workflows are at commit `e73c896795525a61b47ce9f44657a1a045b0a98b` of [sengirunners/benchmarks](https://github.com/sengirunners/benchmarks/tree/e73c896795525a61b47ce9f44657a1a045b0a98b). Flask is not affiliated with Sengi; its name identifies the code that was built.

## Other projects in the benchmark

The same measurement for the other 11 projects, and [all 12 on one page](https://sengi.run/benchmarks).

- [Black CI build time and cost](https://sengi.run/benchmarks/black)
- [GitHub CLI CI build time and cost](https://sengi.run/benchmarks/cli)
- [Codex CLI CI build time and cost](https://sengi.run/benchmarks/codex)
- [Mastodon CI build time and cost](https://sengi.run/benchmarks/mastodon)
- [Guava CI build time and cost](https://sengi.run/benchmarks/guava)
- [AI SDK CI build time and cost](https://sengi.run/benchmarks/vercel-ai)
- [Pydantic CI build time and cost](https://sengi.run/benchmarks/pydantic)
- [TypeScript CI build time and cost](https://sengi.run/benchmarks/typescript)
- [Vite CI build time and cost](https://sengi.run/benchmarks/vite)
- [Vue CI build time and cost](https://sengi.run/benchmarks/vue)
- [fd CI build time and cost](https://sengi.run/benchmarks/fd)

## Where is the data?

- [`/data/github-actions-benchmarks.json`](https://sengi.run/data/github-actions-benchmarks.json): runners, projects and commits, the summary rows, exclusions and every run with its log URL.
- [`/data/github-actions-benchmarks.csv`](https://sengi.run/data/github-actions-benchmarks.csv): one line per project and runner, with the project commit, the cost per build and the median run’s log URL.

Prices per minute for every runner are on [Sengi pricing](https://sengi.run/pricing) and in the [runner comparison](https://sengi.run/compare).

## Frequently asked questions

### How long does Flask CI take on GitHub Actions and on Sengi?

Measured November 17, 2026, at commit 22d9247 of pallets/flask: GitHub-hosted 4 vCPU took 22 s median over 4 runs, $0.012 a build; Sengi Basic 4 vCPU took 14 s median over 3 runs, $0.002 a build, 35% less time and 83% less cost against GitHub-hosted 4 vCPU; Sengi Premium 4 vCPU took 14 s median over 3 runs, $0.006 a build, 35% less time and 50% less cost against GitHub-hosted 4 vCPU. Times are median job seconds at 4 vCPU, printed only with at least 3 runs.

### What does one Flask build cost?

Cost is billed minutes, each job rounded up to a whole minute, at the list price: GitHub-hosted 4 vCPU $0.012 a minute, Sengi Basic 4 vCPU $0.002 a minute and Sengi Premium 4 vCPU $0.006 a minute. Per build: GitHub-hosted 4 vCPU $0.012, Sengi Basic 4 vCPU $0.002, Sengi Premium 4 vCPU $0.006.

### What does the Flask job run?

Python 3.13 test suite (tox). Language: Python. Repository: pallets/flask, pinned to commit 22d924701a6ae2e4cd01e9a15bbaf3946094af65.

### Did Sengi take more time than GitHub’s runner on Flask?

No Sengi runner of GitHub’s size (4 vCPU) took more median time than GitHub-hosted 4 vCPU on Flask where there were enough runs to compare. Every result is in the tables on this page.

### Why does the comparison use 4 vCPU?

The benchmark runs in a public repository so that every run log is public, and GitHub’s ubuntu-24.04 runner measured 4 vCPU there (nproc inside each job). Sengi runners with the same vCPU count are compared with it directly; Sengi Basic 2 vCPU has half the cores of GitHub’s 4 vCPU runner, and Sengi Premium 2 vCPU has half the cores of GitHub’s 4 vCPU runner, so they are shown separately and not as a like-for-like comparison.

### Why is GitHub’s runner priced if public repositories are not billed for it?

GitHub does not bill public repositories for its standard runner, which is 4 vCPU there. The GitHub column is priced at GitHub’s listed rate for its 4 vCPU Linux larger runner (sold to Team and Enterprise Cloud organizations), $0.012 a minute: what a private repository pays for these minutes beyond its plan’s included usage.

### Can I check the Flask numbers?

Yes. Every printed median links to the log of the run nearest that median in the public repository sengirunners/benchmarks, at commit e73c896. Every run, failed ones included, is in the JSON file on this page, and the CSV file has one line per project and runner.
