Skip to content
DataFlowBench

Languages — v0.6.1

Rows are (language, scorecard, tier) partitions — they share a table for navigation, not a denominator. Definitive counts reached and not-reached outcomes; incomplete counts inconclusive, unsupported, and runner-error, which are reported as capability coverage and never scored as negatives.

Three tiers appear here for java, javascript, and python: the core kernel, the modeling matrix under the benchmark-controlled profile, and the modeling probe set under the tool-native profile. They are separate populations sharing a language name. Read the scorecard column, not just the language.

Analyzer depth per language is unchanged from v0.6.0 — java carries seven analyzers, kotlin five, python five, c and cpp four, scala one. A language with more rows is a language this freeze can say more about, not a harder or an easier one, and a one-row language is a language whose numbers rest on a single engine’s word.

The only rows that moved in this patch release are Bifrost’s, and only in six languages: csharp 32 → 34, java 37 → 40, javascript 36 → 38, ruby 21 → 22, rust 36 → 38, and typescript 34 → 36 correct, against go 35 → 34 where the array-element pair regressed to inconclusive. Every other cell in this table is carried over.

Language Scorecard Model profile Tier Cases Definitive Incomplete TPR (macro) FPR (macro)
c bifrost-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 40 8 100.0% 0.0%
c bifrost-c-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 1 1 100.0% n/a
c bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
c codeql-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 48 0 75.6% 14.0%
c codeql-c-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 2 0 100.0% n/a
c infer-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 48 0 72.4% 7.6%
c semgrep-c-kernel-taint-taint-benchmark-controlled benchmark-controlled core 48 14 34 100.0% 42.9%
cpp bifrost-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 30 26 100.0% 0.0%
cpp bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
cpp codeql-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 56 0 53.2% 8.9%
cpp infer-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 56 0 52.9% 0.0%
cpp semgrep-cpp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 56 14 42 100.0% 42.9%
csharp bifrost-csharp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 34 24 100.0% 0.0%
csharp bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
csharp codeql-csharp-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 69.4% 11.8%
go bifrost-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 34 24 100.0% 0.0%
go bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
go codeql-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 66.6% 16.7%
go semgrep-go-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
java bifrost-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 40 18 100.0% 0.0%
java bifrost-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 8 16 100.0% 0.0%
java bifrost-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled calibration 2 1 1 not scored not scored
java bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
java codeql-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 79.6% 19.8%
java codeql-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
java codeql-java-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 29.2%
java flowdroid-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 85.6% 14.3%
java flowdroid-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 14 10 100.0% 0.0%
java flowdroid-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java infer-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 66.9% 0.0%
java infer-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
java infer-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java joern-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 71.8% 18.4%
java joern-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
java joern-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java opentaint-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 80.8% 17.8%
java opentaint-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 12 12 100.0% 0.0%
java opentaint-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
java semgrep-java-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
java semgrep-java-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
java semgrep-java-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript bifrost-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 38 20 100.0% 0.0%
javascript bifrost-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 5 19 100.0% 0.0%
javascript bifrost-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
javascript codeql-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 77.4% 9.0%
javascript codeql-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
javascript codeql-javascript-native-taint-taint-tool-native tool-native modeling 12 12 0 70.8% 58.3%
javascript joern-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 68.1% 18.6%
javascript joern-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
javascript joern-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
javascript semgrep-javascript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
javascript semgrep-javascript-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
javascript semgrep-javascript-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
kotlin bifrost-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 28 30 100.0% 0.0%
kotlin bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
kotlin codeql-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 68.3% 17.1%
kotlin flowdroid-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 89.0% 21.3%
kotlin opentaint-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 85.5% 23.7%
kotlin semgrep-kotlin-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
php bifrost-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 32 26 100.0% 0.0%
php bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
php joern-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 70.2% 16.7%
php semgrep-php-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
python bifrost-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 36 22 100.0% 0.0%
python bifrost-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 8 16 100.0% 0.0%
python bifrost-python-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
python bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 32 32 0 100.0% 0.0%
python codeql-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 70.5% 10.9%
python codeql-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 24 0 100.0% 0.0%
python codeql-python-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 58.3%
python joern-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 70.2% 14.9%
python joern-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 16 8 50.0% 0.0%
python joern-python-native-taint-taint-tool-native tool-native modeling 12 0 12 n/a n/a
python pysa-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 59.1% 7.9%
python pysa-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 20 4 100.0% 0.0%
python pysa-python-native-taint-taint-tool-native tool-native modeling 12 12 0 0.0% 0.0%
python semgrep-python-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
python semgrep-python-modeling-taint-taint-benchmark-controlled benchmark-controlled modeling 24 10 14 100.0% 0.0%
python semgrep-python-native-taint-taint-tool-native tool-native modeling 12 12 0 100.0% 91.7%
ruby bifrost-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 22 36 100.0% 0.0%
ruby bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
ruby codeql-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 64.3% 7.9%
ruby joern-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 54.8% 31.2%
ruby semgrep-ruby-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%
rust bifrost-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 38 16 100.0% 0.0%
rust bifrost-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 0 2 n/a n/a
rust bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
rust codeql-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 54 0 71.4% 10.4%
rust codeql-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled language-extension 2 2 0 0.0% 0.0%
rust joern-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 54 0 70.3% 14.3%
rust semgrep-rust-kernel-taint-taint-benchmark-controlled benchmark-controlled core 54 14 40 100.0% 42.9%
scala bifrost-scala-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 38 20 100.0% 0.0%
scala bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
typescript bifrost-smoke-taint-taint-benchmark-controlled benchmark-controlled core 2 2 0 100.0% 0.0%
typescript bifrost-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 36 22 100.0% 0.0%
typescript codeql-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 58 0 77.4% 9.0%
typescript semgrep-typescript-kernel-taint-taint-benchmark-controlled benchmark-controlled core 58 14 44 100.0% 42.9%