Skip to content
DataFlowBench

Languages — v0.7.0

Rows are (language, scorecard, tier) partitions — they share a table for navigation, not a denominator. Definitive counts reached and not-reached outcomes; incomplete counts inconclusive, unsupported, and runner-error, which are reported as capability coverage and never scored as negatives.

Three tiers appear here for java, javascript, and python: the core kernel, the modeling matrix under the benchmark-controlled profile, and the modeling probe set under the tool-native profile. They are separate populations sharing a language name. Read the scorecard column, not just the language.

Analyzer depth per language is unchanged from v0.6.1 — java carries seven analyzers, kotlin five, python five, c and cpp four, scala one. A language with more rows is a language this freeze can say more about, not a harder or an easier one, and a one-row language is a language whose numbers rest on a single engine’s word.

Seven rows moved in this release, and they moved in two opposite directions. Bifrost gains in two languages: go 34 → 36 and php 32 → 34 correct, as the array-element and map-iteration pairs each turn from a declined pair into a decided one on the v0.10.9 pin. CodeQL loses one definitive answer in five: c 41 → 40, cpp 42 → 41, csharp 47 → 46, python 48 → 47, and ruby 49 → 48 correct, with each of those five gaining one inconclusive — the infeasible-branch negative the endpoint probe withdrew, in the five languages whose extractor prunes the dead branch before the analyzer ever sees it. No decisive mistake was added or removed in any language.

Every other cell in this table is the number v0.6.1 published, including all thirty-three Semgrep CE, Joern and OpenTaint slices, which were re-run on new pins and came back identical.

Language Scorecard Model profile Tier Cases Definitive Incomplete TPR (macro) FPR (macro)
c bifrost-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 40 8 100.0% 0.0%
c bifrost-c-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 1 1 100.0% n/a
c bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
c codeql-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 47 1 75.6% 16.1%
c codeql-c-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 2 0 100.0% n/a
c infer-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 48 0 72.4% 7.6%
c semgrep-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 14 34 100.0% 42.9%
cpp bifrost-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 30 26 100.0% 0.0%
cpp bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
cpp codeql-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 55 1 53.2% 10.8%
cpp infer-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 56 0 52.9% 0.0%
cpp semgrep-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 14 42 100.0% 42.9%
csharp bifrost-csharp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 34 24 100.0% 0.0%
csharp bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
csharp codeql-csharp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 57 1 69.4% 13.7%
go bifrost-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 36 22 100.0% 0.0%
go bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
go codeql-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 66.6% 16.7%
go semgrep-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
java bifrost-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 40 18 100.0% 0.0%
java bifrost-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 7 17 100.0% 0.0%
java bifrost-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled calibration 2 1 1 not scored not scored
java bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
java codeql-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 79.6% 19.8%
java codeql-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
java codeql-java-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 29.2%
java flowdroid-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 85.6% 14.3%
java flowdroid-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 14 10 100.0% 0.0%
java flowdroid-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java infer-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 66.9% 0.0%
java infer-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
java infer-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java joern-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 71.8% 18.4%
java joern-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
java joern-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java opentaint-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 80.8% 17.8%
java opentaint-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 12 12 100.0% 0.0%
java opentaint-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java semgrep-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
java semgrep-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
java semgrep-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript bifrost-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 38 20 100.0% 0.0%
javascript bifrost-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 4 20 100.0% 0.0%
javascript bifrost-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
javascript codeql-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 77.4% 9.0%
javascript codeql-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
javascript codeql-javascript-native-taint-taint-tool-native tool-native modeling 12 12 0 70.8% 58.3%
javascript joern-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 68.1% 18.6%
javascript joern-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
javascript joern-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript semgrep-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
javascript semgrep-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
javascript semgrep-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
kotlin bifrost-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 28 30 100.0% 0.0%
kotlin bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
kotlin codeql-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 68.3% 17.1%
kotlin flowdroid-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 89.0% 21.3%
kotlin opentaint-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 85.5% 23.7%
kotlin semgrep-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
php bifrost-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 34 24 100.0% 0.0%
php bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
php joern-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 70.2% 16.7%
php semgrep-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
python bifrost-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 36 22 100.0% 0.0%
python bifrost-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 7 17 100.0% 0.0%
python bifrost-python-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
python bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
python codeql-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 57 1 70.5% 12.8%
python codeql-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
python codeql-python-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 58.3%
python joern-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 70.2% 14.9%
python joern-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
python joern-python-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
python pysa-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 59.1% 7.9%
python pysa-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 20 4 100.0% 0.0%
python pysa-python-native-taint-taint-tool-native tool-native modeling 12 12 0 0.0% 0.0%
python semgrep-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
python semgrep-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
python semgrep-python-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 91.7%
ruby bifrost-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 22 36 100.0% 0.0%
ruby bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
ruby codeql-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 57 1 64.3% 9.8%
ruby joern-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 54.8% 31.2%
ruby semgrep-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
rust bifrost-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 38 16 100.0% 0.0%
rust bifrost-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 0 2 n/a n/a
rust bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
rust codeql-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 54 0 71.4% 10.4%
rust codeql-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 2 0 0.0% 0.0%
rust joern-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 54 0 70.3% 14.3%
rust semgrep-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 14 40 100.0% 42.9%
scala bifrost-scala-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 38 20 100.0% 0.0%
scala bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
typescript bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
typescript bifrost-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 36 22 100.0% 0.0%
typescript codeql-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 77.4% 9.0%
typescript semgrep-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%