← all organisations

gchq / Code

41 languages, 157.1 MB of source, and what the live estate is written in today.

Language figures come from GitHub's own per-repository byte counts, which measure source rather than a single headline label. File counts and repository sizes come from the mirrors themselves. The two tell different stories, and the difference is the point.

The codebase

41
languages in use
157.1 MB of source
40.5k
files at HEAD
across every analysed repository
0 B
on disk as mirrors
full history, every branch and tag
648.1 MB
largest repository
stroom-docs

What stands out

One archived repository holds 29% of all the bytes.

stroom-docs is 648.1 MB of git - a 2013 static snapshot of www.gov.uk, not source code. It single-handedly makes HTML the largest language by volume (3% of all source bytes). Any size or language statistic that does not exclude it is measuring an archive, not a codebase.

Sizes here are GitHub's reported figures.

This organisation was harvested with blobless partial clones, which fetch every commit and file path but not file contents, so there is no local size to measure against. Checked elsewhere in this dataset against 1,526 fully mirrored repositories, GitHub's reported size is accurate to within about six percent.

Ruby is the house language, but its share is falling.

Ruby is the primary language of 0% of archived repositories but only 0% of live ones. Python, Go, TypeScript and HCL have taken the difference - the signature of a shift from monolithic Rails publishing apps toward data pipelines, infrastructure-as-code and platform tooling.

Most repositories are small; a handful are enormous.

42% of repositories are under a megabyte of git data. The estate is not a monolith - it is a very long tail of small services, gems and prototypes around a few large publishing applications.

Languages, two ways

By volume of source

Bytes of code. Dominated by archived static HTML.

JavaJava: 83.6 MB83.6 MBJavaScriptJavaScript: 30.5 MB30.5 MBC++C++: 21.3 MB21.3 MBTypeScriptTypeScript: 5.6 MB5.6 MBPythonPython: 4.8 MB4.8 MBHTMLHTML: 4.3 MB4.3 MBShellShell: 1.8 MB1.8 MBCSSCSS: 897.8 kB897.8 kBRustRust: 895.0 kB895.0 kBJupyter NotebookJupyter Notebook: 458.1 kB458.1 kBHaskellHaskell: 431.6 kB431.6 kBXSLTXSLT: 409.2 kB409.2 kBStarlarkStarlark: 322.6 kB322.6 kBKotlinKotlin: 321.4 kB321.4 kB

By number of repositories

How many repositories name each language as their primary one. A better guide to what engineers actually work in.

ShellShell: 4545JavaJava: 3333HTMLHTML: 2323PythonPython: 2222JavaScriptJavaScript: 2222DockerfileDockerfile: 2121CSSCSS: 1414Jupyter NotebookJupyter Notebook: 88MakefileMakefile: 66TypeScriptTypeScript: 55XSLTXSLT: 55SmartySmarty: 55FreeMarkerFreeMarker: 55BatchfileBatchfile: 44

Live estate versus archive

The same language ranking, split by whether the repository is still active. The shift is the clearest signal of technical direction in the whole dataset.

Active repositories

JavaJava: 1717PythonPython: 88ShellShell: 99JavaScriptJavaScript: 88TypeScriptTypeScript: 33GroovyGroovy: 22GoGo: 22C++C++: 11HaskellHaskell: 11Jupyter NotebookJupyter Notebook: 11

Archived repositories

JavaJava: 99PythonPython: 22ShellShell: 11JavaScriptJavaScript: 00TypeScriptTypeScript: 11GroovyGroovy: 00GoGo: 00C++C++: 00HaskellHaskell: 00Jupyter NotebookJupyter Notebook: 00

Scale

Repository size distribution

Git data per repository, log-ish buckets.

2518.812.56.20< 10 kB: 2< 10 kB< 100 kB: 4< 100 kB< 1 MB: 25< 1 MB< 10 MB: 20< 10 MB< 100 MB: 16< 100 MB100 MB +: 6100 MB +

Commits against files

Each circle is a repository, sized by number of distinct authors. Both axes are logarithmic. The diagonal band is the normal relationship between codebase size and churn; outliers above it are long-lived apps with heavy iteration, below it are dumps and imports.

35k4.6k591.676.910324.4198.71.6k13.2kBailo: 16,239 commits, 1,356 files, 70 authorsBoilingFrogs: 23 commits, 3 files, 6 authorsConcourseTools: 183 commits, 99 files, 6 authorsCyberChef: 7,976 commits, 984 files, 670 authorsCyberChef-server: 292 commits, 29 files, 27 authorsGaffer: 13,578 commits, 2,621 files, 141 authorsHQDM: 62 commits, 506 files, 5 authorsKai: 448 commits, 64 files, 15 authorsLD-Explorer: 537 commits, 360 files, 5 authorsMaestro: 200 commits, 425 files, 7 authorsMagmaCore: 627 commits, 819 files, 11 authorsPalisade: 2,604 commits, 116 files, 39 authorsPalisade-clients: 513 commits, 145 files, 19 authorsPalisade-common: 639 commits, 39 files, 21 authorsPalisade-examples: 1,387 commits, 210 files, 19 authorsPalisade-readers: 727 commits, 83 files, 19 authorsPalisade-services: 4,956 commits, 978 files, 27 authorsVanguard: 1,499 commits, 234 files, 15 authorsannchor: 161 commits, 57 files, 5 authorscoreax: 5,810 commits, 196 files, 26 authorscoreax-metadata: 456 commits, 524 files, 2 authorsdetections-schema: 22 commits, 25 files, 1 authorsevent-logging: 375 commits, 123 files, 7 authorsevent-logging-schema: 678 commits, 218 files, 15 authorsgaffer-doc: 1,576 commits, 138 files, 30 authorsgaffer-docker: 1,312 commits, 342 files, 37 authorsgaffer-experimental: 6,151 commits, 449 files, 41 authorsgaffer-tools: 4,328 commits, 155 files, 63 authorsgafferpy: 2,076 commits, 83 files, 42 authorsgchq-data-quality: 139 commits, 138 files, 6 authorsgradle-dependency-analyze: 176 commits, 17 files, 15 authorshadoop-common-shaded: 29 commits, 13 files, 4 authorshadoop-hdfs-shaded: 25 commits, 13 files, 4 authorshbase-common-shaded: 11 commits, 11 files, 2 authorsiris-schedule-viewer: 11 commits, 32 files, 4 authorsiris-timeline-viewer: 10 commits, 35 files, 4 authorsiris-worm: 12 commits, 20 files, 4 authorskoryphe: 887 commits, 455 files, 48 authorsnix-bootstrap: 109 commits, 158 files, 2 authorssleeper: 35,001 commits, 3,600 files, 32 authorsstroom: 23,005 commits, 13,167 files, 38 authorsstroom-analytic-demo: 156 commits, 172 files, 1 authorsstroom-annotations: 123 commits, 109 files, 2 authorsstroom-ansible: 294 commits, 268 files, 13 authorsstroom-auth: 1,220 commits, 206 files, 6 authorsstroom-clients: 67 commits, 34 files, 6 authorsstroom-content: 398 commits, 1,265 files, 13 authorsstroom-data-generator: 109 commits, 608 files, 2 authorsstroom-docs: 2,982 commits, 1,392 files, 26 authorsstroom-expression: 255 commits, 189 files, 8 authorsstroom-js: 17 commits, 10 files, 1 authorsstroom-k8s-operator: 246 commits, 132 files, 6 authorsstroom-proxy: 105 commits, 396 files, 4 authorsstroom-query: 729 commits, 276 files, 10 authorsstroom-query-elastic: 117 commits, 145 files, 2 authorsstroom-resources: 2,790 commits, 309 files, 17 authorsstroom-spark-datasource: 64 commits, 109 files, 1 authorsstroom-stats: 574 commits, 416 files, 7 authorsstroom-test-data: 50 commits, 31 files, 4 authorsstroom-timeline: 39 commits, 60 files, 2 authorsstroom-timeline-loader: 30 commits, 24 files, 3 authorsstroom-ui: 803 commits, 1,043 files, 7 authorsstroom-visualisations-dev: 708 commits, 553 files, 16 authorssynthetic-data-generator: 15 commits, 31 files, 2 authorsurlDependencies-plugin: 16 commits, 14 files, 2 authorsvanguard-metadata: 63 commits, 72 files, 2 authorsfiles at HEAD (log)commits (log)

The shape of a repository

Directory layout is the cheapest reliable signal of what a repository actually is. app/ with config/ and spec/ is a Rails application; src/ with test/ is Java or Node. Counted by how many repositories contain each top-level directory at HEAD.

Most common top-level directories

srcsrc: 2323gradlegradle: 2020docsdocs: 1313code-stylecode-style: 1010logoslogos: 77licenceslicences: 77.devcontainer.devcontainer: 66teststests: 66cdcd: 55scriptsscripts: 55.vscode.vscode: 44examplesexamples: 44licenseslicenses: 44.cspell.cspell: 44pre_commit_hookspre_commit_hooks: 44configconfig: 44.idea.idea: 44liblib: 33

Most common file extensions

By total file count across the estate.

.java.java: 17.7k17.7k.xml.xml: 1.9k1.9k.json.json: 1.5k1.5k.ts.ts: 1.3k1.3k.md.md: 1.2k1.2k.node.node: 964964.meta.meta: 894894.mjs.mjs: 884884.png.png: 841841.yaml.yaml: 803803.js.js: 802802.svg.svg: 785785.tsx.tsx: 703703.cc.cc: 634634.py.py: 585585.sh.sh: 569569

The twenty largest repositories

RepositoryLanguageGit size CommitsFilesStatus
stroom-docsShell648.1 MB2,9821,392active
GafferJava233.3 MB13,5782,621archived
stroomJava227.9 MB23,00513,167active
BailoTypeScript199.4 MB16,2391,356active
stroom-k8s-operatorGo171.5 MB246132active
CyberChefJavaScript145.7 MB7,976984active
sleeperJava81.5 MB35,0013,600active
gaffer-toolsPython78.4 MB4,328155archived
coreaxPython59.6 MB5,810196active
gaffer-doc-54.0 MB1,576138archived
stroom-kubernetes-operatorGo51.4 MB1130active
hbase-common-shaded-29.3 MB1111active
stroom-proxyJava27.4 MB105396active
BoilingFrogs-23.1 MB233active
annchorPython21.7 MB16157active
stroom-analytic-demoJupyter Notebook20.8 MB156172active
hadoop-common-shadedShell20.1 MB2913active
stroom-visualisations-devJavaScript17.6 MB708553active
hadoop-hdfs-shadedShell16.0 MB2513active
gaffer-experimentalJava14.5 MB6,151449archived