# GitHub Actions runner benchmarks: 12 open-source builds on Sengi and GitHub’s runner

Source: https://sengi.run/benchmarks

By Sengi · Last checked November 17, 2026

## How long do open-source builds take on Sengi and on GitHub’s runner, and what do they cost?

Measured November 17, 2026 on 12 open-source projects, each built at a pinned commit with its own CI commands, on Sengi runners with the same 4 vCPU as GitHub’s runner: building all 12 once cost $0.094 on Sengi Basic 4 vCPU against $0.624 on GitHub-hosted 4 vCPU, and Sengi Basic 4 vCPU cost less per build on 12 of 12. Sengi Basic 4 vCPU took less time than GitHub’s runner on 9 of 12 projects and more time on 3 (Codex CLI, Mastodon and Guava). Sengi Premium 4 vCPU took less time than GitHub’s runner on 11 of 11 projects; Mastodon had too few runs to compare because 1 of 3 runs on Sengi Premium 4 vCPU failed. Times are median job seconds over at least 3 runs; these results cover these builds only. 3 jobs failed, all on Sengi runners, and none is counted in any median. Mastodon on Sengi Basic 2 vCPU and Mastodon on Sengi Premium 4 vCPU: Sengi's build cache served a truncated layer, a short read ending in unexpected EOF, so the Docker image build failed. TypeScript on Sengi Basic 2 vCPU: the compiler test suite failed on mocha test timeouts.

Measured November 17, 2026, last checked November 17, 2026. Every number on this page is in [one JSON file](https://sengi.run/data/github-actions-benchmarks.json) and [a CSV file](https://sengi.run/data/github-actions-benchmarks.csv), and every median links to its run log in the public repository [sengirunners/benchmarks](https://github.com/sengirunners/benchmarks/tree/e73c896795525a61b47ce9f44657a1a045b0a98b).

## Results at 4 vCPU: GitHub’s runner, Sengi Basic 4 vCPU and Sengi Premium 4 vCPU

One row per project and runner. Time is the median job run time in seconds, from the job’s start to its end, over the runs counted; min and max are the fastest and slowest of those runs. Cost per build is the median of the runs’ billed minutes, each job rounded up to a whole minute, at the runner’s list price. A cell with fewer than 3 runs prints “not enough runs”.

**Median job time, run count and cost per build at 4 vCPU, measured November 17, 2026**

| Project | Commit | Runner | Runs | Median time (s) | Min–max (s) | Cost per build (USD) | Time against GitHub | Cost against GitHub |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Black | `87928e6` | GitHub-hosted 4 vCPU | 4 | [99](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133907685793) | 96–133 | $0.024 | baseline | baseline |
| Black | `87928e6` | Sengi Basic 4 vCPU | 3 | [69](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133987524966) | 67–90 | $0.004 | 30% less time | 83% less cost |
| Black | `87928e6` | Sengi Premium 4 vCPU | 3 | [69](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133993387518) | 53–82 | $0.012 | 30% less time | 50% less cost |
| GitHub CLI | `fc4b137` | GitHub-hosted 4 vCPU | 4 | [184](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133907139240) | 170–198 | $0.042 | baseline | baseline |
| GitHub CLI | `fc4b137` | Sengi Basic 4 vCPU | 3 | [163](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133986501687) | 161–185 | $0.006 | 11% less time | 86% less cost |
| GitHub CLI | `fc4b137` | Sengi Premium 4 vCPU | 3 | [161](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133992956299) | 153–167 | $0.018 | 12% less time | 57% less cost |
| Codex CLI | `01fc69f` | GitHub-hosted 4 vCPU | 4 | [368](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133904356558) | 347–415 | $0.084 | baseline | baseline |
| Codex CLI | `01fc69f` | Sengi Basic 4 vCPU | 3 | [381](https://github.com/sengirunners/benchmarks/actions/runs/44055822257/job/133944876593) | 369–391 | $0.014 | 4% more time | 83% less cost |
| Codex CLI | `01fc69f` | Sengi Premium 4 vCPU | 3 | [286](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133990416563) | 279–313 | $0.030 | 22% less time | 64% less cost |
| Mastodon | `3987b9f` | GitHub-hosted 4 vCPU | 4 | [153](https://github.com/sengirunners/benchmarks/actions/runs/44059823560/job/133960202675) | 124–200 | $0.036 | baseline | baseline |
| Mastodon | `3987b9f` | Sengi Basic 4 vCPU | 3 | [168](https://github.com/sengirunners/benchmarks/actions/runs/44055822257/job/133946312686) | 115–190 | $0.006 | 10% more time | 83% less cost |
| Mastodon | `3987b9f` | Sengi Premium 4 vCPU | 2 (1 failed) | not enough runs | – | – | – | – |
| Flask | `22d9247` | GitHub-hosted 4 vCPU | 4 | [22](https://github.com/sengirunners/benchmarks/actions/runs/44059823560/job/133964347514) | 18–29 | $0.012 | baseline | baseline |
| Flask | `22d9247` | Sengi Basic 4 vCPU | 3 | [14](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133989236037) | 13–16 | $0.002 | 35% less time | 83% less cost |
| Flask | `22d9247` | Sengi Premium 4 vCPU | 3 | [14](https://github.com/sengirunners/benchmarks/actions/runs/44057933305/job/133957021121) | 13–16 | $0.006 | 35% less time | 50% less cost |
| Guava | `666338c` | GitHub-hosted 4 vCPU | 4 | [735](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133904356424) | 730–740 | $0.156 | baseline | baseline |
| Guava | `666338c` | Sengi Basic 4 vCPU | 3 | [759](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133983154807) | 754–772 | $0.026 | 3% more time | 83% less cost |
| Guava | `666338c` | Sengi Premium 4 vCPU | 3 | [650](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133990416620) | 632–680 | $0.066 | 12% less time | 58% less cost |
| AI SDK | `45e1fbc` | GitHub-hosted 4 vCPU | 4 | [184](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133904356389) | 181–190 | $0.048 | baseline | baseline |
| AI SDK | `45e1fbc` | Sengi Basic 4 vCPU | 3 | [164](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133983155216) | 160–173 | $0.006 | 11% less time | 88% less cost |
| AI SDK | `45e1fbc` | Sengi Premium 4 vCPU | 3 | [123](https://github.com/sengirunners/benchmarks/actions/runs/44080996859/job/134031284111) | 115–125 | $0.018 | 33% less time | 63% less cost |
| Pydantic | `001dea0` | GitHub-hosted 4 vCPU | 4 | [121](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133906729733) | 113–133 | $0.030 | baseline | baseline |
| Pydantic | `001dea0` | Sengi Basic 4 vCPU | 3 | [106](https://github.com/sengirunners/benchmarks/actions/runs/44055822257/job/133947875032) | 104–112 | $0.004 | 12% less time | 87% less cost |
| Pydantic | `001dea0` | Sengi Premium 4 vCPU | 3 | [89](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133992554731) | 83–99 | $0.012 | 26% less time | 60% less cost |
| TypeScript | `050880c` | GitHub-hosted 4 vCPU | 4 | [409](https://github.com/sengirunners/benchmarks/actions/runs/44070981002/job/133996893698) | 372–543 | $0.090 | baseline | baseline |
| TypeScript | `050880c` | Sengi Basic 4 vCPU | 3 | [394](https://github.com/sengirunners/benchmarks/actions/runs/44066983499/job/133983154781) | 367–425 | $0.014 | 4% less time | 84% less cost |
| TypeScript | `050880c` | Sengi Premium 4 vCPU | 3 | [310](https://github.com/sengirunners/benchmarks/actions/runs/44069088421/job/133990416528) | 308–334 | $0.036 | 24% less time | 60% less cost |
| Vite | `39ddf7c` | GitHub-hosted 4 vCPU | 4 | [259](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133908560908) | 254–266 | $0.060 | baseline | baseline |
| Vite | `39ddf7c` | Sengi Basic 4 vCPU | 3 | [183](https://github.com/sengirunners/benchmarks/actions/runs/44078847496/job/134027757592) | 173–209 | $0.008 | 29% less time | 87% less cost |
| Vite | `39ddf7c` | Sengi Premium 4 vCPU | 3 | [190](https://github.com/sengirunners/benchmarks/actions/runs/44057933305/job/133955034861) | 162–240 | $0.024 | 26% less time | 60% less cost |
| Vue | `5be58b4` | GitHub-hosted 4 vCPU | 4 | [62](https://github.com/sengirunners/benchmarks/actions/runs/44070981002/job/134002427841) | 48–68 | $0.018 | baseline | baseline |
| Vue | `5be58b4` | Sengi Basic 4 vCPU | 3 | [48](https://github.com/sengirunners/benchmarks/actions/runs/44055822257/job/133949771546) | 47–50 | $0.002 | 23% less time | 89% less cost |
| Vue | `5be58b4` | Sengi Premium 4 vCPU | 3 | [41](https://github.com/sengirunners/benchmarks/actions/runs/44057933305/job/133956099639) | 37–42 | $0.006 | 34% less time | 67% less cost |
| fd | `4f81778` | GitHub-hosted 4 vCPU | 4 | [70](https://github.com/sengirunners/benchmarks/actions/runs/44043756102/job/133909021002) | 63–72 | $0.024 | baseline | baseline |
| fd | `4f81778` | Sengi Basic 4 vCPU | 3 | [48](https://github.com/sengirunners/benchmarks/actions/runs/44078847496/job/134028853794) | 46–53 | $0.002 | 31% less time | 92% less cost |
| fd | `4f81778` | Sengi Premium 4 vCPU | 3 | [44](https://github.com/sengirunners/benchmarks/actions/runs/44057933305/job/133956560653) | 41–46 | $0.006 | 37% less time | 75% less cost |

Sengi Basic 4 vCPU took less time than GitHub’s runner on 9 of 12 projects and more time on 3 (Codex CLI, Mastodon and Guava); building all 12 once cost $0.094 on Sengi Basic 4 vCPU against $0.624 on GitHub-hosted 4 vCPU.

Sengi Premium 4 vCPU took less time than GitHub’s runner on 11 of 11 projects; Mastodon had too few runs to compare because 1 of 3 runs on Sengi Premium 4 vCPU failed; building all 11 once cost $0.234 on Sengi Premium 4 vCPU against $0.588 on GitHub-hosted 4 vCPU.

## Median job time at 4 vCPU, by project

Median job time in seconds, measured November 17, 2026. Sengi runners in blue, GitHub-hosted 4 vCPU in grey. The same rows as the table above.

## Not like-for-like: Sengi Basic 2 vCPU and Sengi Premium 2 vCPU

Sengi Basic 2 vCPU has half the cores of GitHub’s 4 vCPU runner. Sengi Premium 2 vCPU has half the cores of GitHub’s 4 vCPU runner. These Sengi runners were run on the same projects; they are not a like-for-like comparison, and no GitHub runner of their size was measured. The last two columns still set each against GitHub-hosted 4 vCPU.

**Not like-for-like: median job time, run count and cost per build on Sengi Basic 2 vCPU (half the cores of GitHub’s 4 vCPU runner) and Sengi Premium 2 vCPU (half the cores of GitHub’s 4 vCPU runner), measured November 17, 2026**

| Project | Commit | Runner | Runs | Median time (s) | Min–max (s) | Cost per build (USD) | Time against GitHub | Cost against GitHub |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Black | `87928e6` | Sengi Basic 2 vCPU | 4 | [117](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133927774213) | 108–123 | $0.002 | 18% more time | 92% less cost |
| Black | `87928e6` | Sengi Premium 2 vCPU | 4 | [77](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133978986234) | 73–91 | $0.006 | 22% less time | 75% less cost |
| GitHub CLI | `fc4b137` | Sengi Basic 2 vCPU | 4 | [349](https://github.com/sengirunners/benchmarks/actions/runs/44061963873/job/133971409406) | 340–455 | $0.006 | 90% more time | 86% less cost |
| GitHub CLI | `fc4b137` | Sengi Premium 2 vCPU | 4 | [311](https://github.com/sengirunners/benchmarks/actions/runs/44075890696/job/134020472191) | 305–329 | $0.018 | 69% more time | 57% less cost |
| Codex CLI | `01fc69f` | Sengi Basic 2 vCPU | 4 | [764](https://github.com/sengirunners/benchmarks/actions/runs/44073122528/job/134004175059) | 604–2167 | $0.014 | 108% more time | 84% less cost |
| Codex CLI | `01fc69f` | Sengi Premium 2 vCPU | 4 | [661](https://github.com/sengirunners/benchmarks/actions/runs/44052276791/job/133932847277) | 540–777 | $0.035 | 80% more time | 59% less cost |
| Mastodon | `3987b9f` | Sengi Basic 2 vCPU | 3 (1 failed) | [208](https://github.com/sengirunners/benchmarks/actions/runs/44073122528/job/134006627145) | 138–1643 | $0.004 | 36% more time | 89% less cost |
| Mastodon | `3987b9f` | Sengi Premium 2 vCPU | 4 | [129](https://github.com/sengirunners/benchmarks/actions/runs/44052276791/job/133938252764) | 127–159 | $0.009 | 16% less time | 75% less cost |
| Flask | `22d9247` | Sengi Basic 2 vCPU | 4 | [19](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133930890110) | 16–21 | $0.001 | 14% less time | 92% less cost |
| Flask | `22d9247` | Sengi Premium 2 vCPU | 4 | [13](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133981028771) | 9–15 | $0.003 | 42% less time | 75% less cost |
| Guava | `666338c` | Sengi Basic 2 vCPU | 4 | [971](https://github.com/sengirunners/benchmarks/actions/runs/44073122528/job/134004175386) | 900–2185 | $0.017 | 32% more time | 89% less cost |
| Guava | `666338c` | Sengi Premium 2 vCPU | 4 | [813](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133975023165) | 695–941 | $0.042 | 11% more time | 73% less cost |
| AI SDK | `45e1fbc` | Sengi Basic 2 vCPU | 4 | [289](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133910810709) | 227–333 | $0.006 | 57% more time | 89% less cost |
| AI SDK | `45e1fbc` | Sengi Premium 2 vCPU | 4 | [210](https://github.com/sengirunners/benchmarks/actions/runs/44052276791/job/133932850370) | 184–304 | $0.012 | 14% more time | 75% less cost |
| Pydantic | `001dea0` | Sengi Basic 2 vCPU | 4 | [133](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133926557523) | 118–150 | $0.003 | 10% more time | 90% less cost |
| Pydantic | `001dea0` | Sengi Premium 2 vCPU | 4 | [107](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133977939883) | 94–117 | $0.006 | 12% less time | 80% less cost |
| TypeScript | `050880c` | Sengi Basic 2 vCPU | 3 (1 failed) | [583](https://github.com/sengirunners/benchmarks/actions/runs/44073122528/job/134004174897) | 494–701 | $0.010 | 43% more time | 89% less cost |
| TypeScript | `050880c` | Sengi Premium 2 vCPU | 4 | [565](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133975023195) | 412–679 | $0.030 | 38% more time | 67% less cost |
| Vite | `39ddf7c` | Sengi Basic 2 vCPU | 4 | [459](https://github.com/sengirunners/benchmarks/actions/runs/44061963873/job/133970444536) | 444–500 | $0.008 | 78% more time | 87% less cost |
| Vite | `39ddf7c` | Sengi Premium 2 vCPU | 4 | [401](https://github.com/sengirunners/benchmarks/actions/runs/44052276791/job/133939253189) | 368–586 | $0.023 | 55% more time | 63% less cost |
| Vue | `5be58b4` | Sengi Basic 2 vCPU | 4 | [89](https://github.com/sengirunners/benchmarks/actions/runs/44061963873/job/133971655398) | 77–128 | $0.002 | 43% more time | 89% less cost |
| Vue | `5be58b4` | Sengi Premium 2 vCPU | 4 | [68](https://github.com/sengirunners/benchmarks/actions/runs/44064619098/job/133979765824) | 63–74 | $0.006 | 9% more time | 67% less cost |
| fd | `4f81778` | Sengi Basic 2 vCPU | 4 | [70](https://github.com/sengirunners/benchmarks/actions/runs/44073122528/job/134011625801) | 59–100 | $0.002 | the same time | 92% less cost |
| fd | `4f81778` | Sengi Premium 2 vCPU | 4 | [56](https://github.com/sengirunners/benchmarks/actions/runs/44052276791/job/133942228682) | 49–62 | $0.003 | 21% less time | 88% less cost |

## Where did Sengi take more time?

Every result where a Sengi runner’s median was above GitHub’s is printed in the tables above and listed here.

### At 4 vCPU, the same size as GitHub’s runner

- **Codex CLI, Sengi Basic 4 vCPU:** 381 s against 368 s on GitHub’s runner, 4% more time, at $0.014 a build against $0.084.
- **Mastodon, Sengi Basic 4 vCPU:** 168 s against 153 s on GitHub’s runner, 10% more time, at $0.006 a build against $0.036.
- **Guava, Sengi Basic 4 vCPU:** 759 s against 735 s on GitHub’s runner, 3% more time, at $0.026 a build against $0.156.

### Not like-for-like: smaller or larger Sengi runners against GitHub’s 4 vCPU runner

- **Black, Sengi Basic 2 vCPU:** 117 s against 99 s on GitHub’s runner, 18% more time, at $0.002 a build against $0.024.
- **GitHub CLI, Sengi Basic 2 vCPU:** 349 s against 184 s on GitHub’s runner, 90% more time, at $0.006 a build against $0.042.
- **Codex CLI, Sengi Basic 2 vCPU:** 764 s against 368 s on GitHub’s runner, 108% more time, at $0.014 a build against $0.084.
- **Mastodon, Sengi Basic 2 vCPU:** 208 s against 153 s on GitHub’s runner, 36% more time, at $0.004 a build against $0.036.
- **Guava, Sengi Basic 2 vCPU:** 971 s against 735 s on GitHub’s runner, 32% more time, at $0.017 a build against $0.156.
- **AI SDK, Sengi Basic 2 vCPU:** 289 s against 184 s on GitHub’s runner, 57% more time, at $0.006 a build against $0.048.
- **Pydantic, Sengi Basic 2 vCPU:** 133 s against 121 s on GitHub’s runner, 10% more time, at $0.003 a build against $0.030.
- **TypeScript, Sengi Basic 2 vCPU:** 583 s against 409 s on GitHub’s runner, 43% more time, at $0.010 a build against $0.090.
- **Vite, Sengi Basic 2 vCPU:** 459 s against 259 s on GitHub’s runner, 78% more time, at $0.008 a build against $0.060.
- **Vue, Sengi Basic 2 vCPU:** 89 s against 62 s on GitHub’s runner, 43% more time, at $0.002 a build against $0.018.
- **GitHub CLI, Sengi Premium 2 vCPU:** 311 s against 184 s on GitHub’s runner, 69% more time, at $0.018 a build against $0.042.
- **Codex CLI, Sengi Premium 2 vCPU:** 661 s against 368 s on GitHub’s runner, 80% more time, at $0.035 a build against $0.084.
- **Guava, Sengi Premium 2 vCPU:** 813 s against 735 s on GitHub’s runner, 11% more time, at $0.042 a build against $0.156.
- **AI SDK, Sengi Premium 2 vCPU:** 210 s against 184 s on GitHub’s runner, 14% more time, at $0.012 a build against $0.048.
- **TypeScript, Sengi Premium 2 vCPU:** 565 s against 409 s on GitHub’s runner, 38% more time, at $0.030 a build against $0.090.
- **Vite, Sengi Premium 2 vCPU:** 401 s against 259 s on GitHub’s runner, 55% more time, at $0.023 a build against $0.060.
- **Vue, Sengi Premium 2 vCPU:** 68 s against 62 s on GitHub’s runner, 9% more time, at $0.006 a build against $0.018.

## How was it measured?

- **Time:** Job run time, start to end, from the GitHub API. Queue time is kept apart.
- **Runs:** Median of the measured runs, with n, min and max; a median needs n >= 3.
- **Cost:** Billed minutes (each job rounded up to a whole minute) times the runner's list price per minute.
- **GitHub cost:** GitHub does not bill public repositories for this runner; its cost is what a private repository pays at GitHub's listed rate for the measured size.
- **Order:** One runner class at a time; at most four jobs at once within a run. One warm-up run per class first fills dependency caches.
- **Selection:** The project list is the whole workload list of an existing open-source CI benchmark suite, adopted before running it here. Each project is pinned to a release commit.
- **Exclusions:** Every run is listed. A job is left out of the medians only for the reason given next to it.
- **Dates:** 2026-11-17 to 2026-11-17.
- **Runner size:** the vCPU count printed for each runner was read with `nproc` inside the job, not taken from a label.
- **Prices:** GitHub-hosted 4 vCPU $0.012, Sengi Basic 2 vCPU $0.001, Sengi Premium 2 vCPU $0.003, Sengi Basic 4 vCPU $0.002 and Sengi Premium 4 vCPU $0.006 a minute. GitHub does not bill public repositories for its standard runner, which is 4 vCPU there. The GitHub column is priced at GitHub’s listed rate for its 4 vCPU Linux larger runner (sold to Team and Enterprise Cloud organizations), $0.012 a minute: what a private repository pays for these minutes beyond its plan’s included usage.
- **Queue time** is recorded for every run in the data files and is not part of the times above.

## Which projects, at which commits?

The project list is the workload list of an existing open-source CI benchmark suite. We adopted the whole list before running it here and did not choose the projects. Each is pinned to a release commit. Every project we ran is shown, including the ones where Sengi took more time. A project is left out only when its build fails on GitHub’s runner as well, and this page lists it with the reason.

| Project | Repository | Language | What the job runs | Commit |
| --- | --- | --- | --- | --- |
| Black | psf/black | Python | Python 3.13 test suite (tox) | `87928e6d6761a4a6d22250e1fee5601b3998086e` |
| GitHub CLI | cli/cli | Go | Unit and integration tests with the race detector, then a build | `fc4b137cdef0a6bd28fd461b7cf9c84a5812a8cd` |
| Codex CLI | openai/codex | Rust | cargo clippy over the Rust workspace, tests included | `01fc69f4026735edfdf6789820549727a4867b11` |
| Mastodon | mastodon/mastodon | Docker | Docker image build (linux/amd64) after a one-file change, layer cache on | `3987b9fd624010eeeaeb7cad96606b05aa813f6a` |
| Flask | pallets/flask | Python | Python 3.13 test suite (tox) | `22d924701a6ae2e4cd01e9a15bbaf3946094af65` |
| Guava | google/guava | Java | Maven install and verify on JDK 25 | `666338cc49d5686c07e3e4d17f0aa1cec25a2be6` |
| AI SDK | vercel/ai | TypeScript | Turborepo build of three Next.js examples | `45e1fbc3ea10d22f852e12b1c1851e289c85e835` |
| Pydantic | pydantic/pydantic | Python | pytest suite on Python 3.13 | `001dea020e0809844e5b17666432c9135a976f46` |
| TypeScript | microsoft/TypeScript | TypeScript | Compiler test suite on Node 24 | `050880ce59e30b356b686bd3144efe24f875ebc8` |
| Vite | vitejs/vite | TypeScript | Build, unit tests and Playwright end-to-end tests on Node 24 | `39ddf7ccf7e7469ff6a3ba37bca38c32ea804d6e` |
| Vue | vuejs/core | TypeScript | Unit tests (Vitest) | `5be58b4c475c1d14b4abacbfeda610394a0ee4e5` |
| fd | sharkdp/fd | Rust | cargo build --release, then cargo test | `4f81778774463bf414a184cbe6d5219ad2229646` |

The benchmark workflows are at commit `e73c896795525a61b47ce9f44657a1a045b0a98b` of [sengirunners/benchmarks](https://github.com/sengirunners/benchmarks/tree/e73c896795525a61b47ce9f44657a1a045b0a98b). Project names identify the code that was built; none of the projects is affiliated with Sengi.

## Is there a page for each project?

Yes. Each project has its own page with the same numbers for that project alone: its results, where Sengi took more time, any failed job and its run logs.

- [Black CI build time and cost](https://sengi.run/benchmarks/black)
- [GitHub CLI CI build time and cost](https://sengi.run/benchmarks/cli)
- [Codex CLI CI build time and cost](https://sengi.run/benchmarks/codex)
- [Mastodon CI build time and cost](https://sengi.run/benchmarks/mastodon)
- [Flask CI build time and cost](https://sengi.run/benchmarks/flask)
- [Guava CI build time and cost](https://sengi.run/benchmarks/guava)
- [AI SDK CI build time and cost](https://sengi.run/benchmarks/vercel-ai)
- [Pydantic CI build time and cost](https://sengi.run/benchmarks/pydantic)
- [TypeScript CI build time and cost](https://sengi.run/benchmarks/typescript)
- [Vite CI build time and cost](https://sengi.run/benchmarks/vite)
- [Vue CI build time and cost](https://sengi.run/benchmarks/vue)
- [fd CI build time and cost](https://sengi.run/benchmarks/fd)

## What was left out?

3 jobs failed, all on Sengi runners, and none is counted in any median. Mastodon on Sengi Basic 2 vCPU and Mastodon on Sengi Premium 4 vCPU: Sengi's build cache served a truncated layer, a short read ending in unexpected EOF, so the Docker image build failed. TypeScript on Sengi Basic 2 vCPU: the compiler test suite failed on mocha test timeouts. Each failed job is also counted in its row’s failures.

- [Run 44030937414](https://github.com/sengirunners/benchmarks/actions/runs/44030937414): warm-up run: fills each runner's dependency caches first; not measured
- [Run 44033028003](https://github.com/sengirunners/benchmarks/actions/runs/44033028003): warm-up run: fills each runner's dependency caches first; not measured
- [Run 44037191239](https://github.com/sengirunners/benchmarks/actions/runs/44037191239): warm-up run: fills each runner's dependency caches first; not measured
- [Run 44039576472](https://github.com/sengirunners/benchmarks/actions/runs/44039576472): warm-up run: fills each runner's dependency caches first; not measured
- [Run 44041540026](https://github.com/sengirunners/benchmarks/actions/runs/44041540026): warm-up run: fills each runner's dependency caches first; not measured
- [Job 133910810755](https://github.com/sengirunners/benchmarks/actions/runs/44045710342/job/133910810755): the compiler test suite failed on mocha test timeouts
- [Job 134032483827](https://github.com/sengirunners/benchmarks/actions/runs/44080996859/job/134032483827): Sengi's build cache served a truncated layer, a short read ending in unexpected EOF, so the Docker image build failed
- [Job 134048517537](https://github.com/sengirunners/benchmarks/actions/runs/44085096635/job/134048517537): Sengi's build cache served a truncated layer, a short read ending in unexpected EOF, so the Docker image build failed

## Where is the data?

- [`/data/github-actions-benchmarks.json`](https://sengi.run/data/github-actions-benchmarks.json): runners, projects and commits, the summary rows, exclusions and every run with its log URL.
- [`/data/github-actions-benchmarks.csv`](https://sengi.run/data/github-actions-benchmarks.csv): one line per project and runner, with the project commit, the cost per build and the median run’s log URL.

Prices per minute for every runner are on [Sengi pricing](https://sengi.run/pricing) and in the [runner comparison](https://sengi.run/compare).

## Frequently asked questions

### Do Sengi runners finish open-source builds in less time than GitHub’s runner?

On these builds, at these commits: Sengi Basic 4 vCPU took less time than GitHub’s runner on 9 of 12 projects and more time on 3 (Codex CLI, Mastodon and Guava); Sengi Premium 4 vCPU took less time than GitHub’s runner on 11 of 11 projects; Mastodon had too few runs to compare because 1 of 3 runs on Sengi Premium 4 vCPU failed. Times are median job seconds over at least 3 runs at 4 vCPU, measured November 17, 2026.

### What does a build cost on Sengi Basic compared with GitHub’s runner?

Measured November 17, 2026, building all 12 once cost $0.094 on Sengi Basic 4 vCPU against $0.624 on GitHub-hosted 4 vCPU. Cost is billed minutes, each job rounded up to a whole minute, at the list price: $0.002 a minute for Sengi Basic 4 vCPU and $0.012 for a 4 vCPU GitHub runner.

### Why does the comparison use 4 vCPU?

The benchmark runs in a public repository so that every run log is public, and GitHub’s ubuntu-24.04 runner measured 4 vCPU there (nproc inside each job). The Sengi runners with the same vCPU count are compared with it directly; Sengi Basic 2 vCPU has half the cores of GitHub’s 4 vCPU runner, and Sengi Premium 2 vCPU has half the cores of GitHub’s 4 vCPU runner, so they are shown separately and not as a like-for-like comparison.

### Why is GitHub’s runner priced if public repositories are not billed for it?

GitHub does not bill public repositories for its standard runner, which is 4 vCPU there. The GitHub column is priced at GitHub’s listed rate for its 4 vCPU Linux larger runner (sold to Team and Enterprise Cloud organizations), $0.012 a minute: what a private repository pays for these minutes beyond its plan’s included usage.

### How were the projects chosen?

The project list is the workload list of an existing open-source CI benchmark suite, adopted whole before any of it was run here. Each project is pinned to a release commit. Every project run is shown, including the ones where Sengi took more time.

### Where Sengi took more time, is that shown?

Yes. At 4 vCPU: Codex CLI on Sengi Basic 4 vCPU, 381 s against 368 s; Mastodon on Sengi Basic 4 vCPU, 168 s against 153 s; Guava on Sengi Basic 4 vCPU, 759 s against 735 s. 17 more results on smaller or larger Sengi runners, not like-for-like with GitHub’s 4 vCPU runner, took more time than it; all are listed on this page.

### Can I check the numbers?

Yes. Every printed median links to the log of the run nearest that median in the public repository sengirunners/benchmarks, at commit e73c896. Every run, failed ones included, is in the JSON file on this page, and the CSV file has one line per project and runner.
