This tests various parsers agains the toml-test suite.

"Decoder tests" check if valid TOML documents are processed correctly, "encoder tests" checks writing TOML files, and "invalid input" checks that the parser rejects invalid TOML documents. Implementations are marked as "compliant" (✅) if both the "valid" and "encoder" tests pass, otherwise they're marked as non-compliant (❌). Usually it's not a big deal to accept some technically invalid TOML; almost all failing "invalid" tests are uncommon edge cases.

The benchmarks are a simple decode of a 15k and 5M file. Results are displayed as "good", "slow", and "very slow" only. Doing a fully fair comparison is not straight-forward, and it's not a contest on who can write the fastest parser – it's just to establish that performance is within reasonable bounds.

Show only:
Name Decoder tests Encoder tests Invalid input 15k file 5M file
C++ nullptr-toml
pass: 188 fail: 3
(doesn't include encoder)
pass: 371 fail: 0
(TODO) (TODO) output
C++ toml++
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good good output
C++ toml11
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good slow output
C toml-c
pass: 191 fail: 0
(doesn't include encoder)
pass: 357 fail: 14
good slow output
C tomlc99
pass: 183 fail: 8
(doesn't include encoder)
pass: 293 fail: 78
good slow output
C# tomlyn
pass: 188 fail: 3
(TODO)
pass: 366 fail: 5
slow good output
Dart toml.dart
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good good output
Fortran toml-f
pass: 180 fail: 11
pass: 179 fail: 12
pass: 339 fail: 32
good very slow output
Go go-toml
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good slow output
Go toml
pass: 191 fail: 0
pass: 191 fail: 0
pass: 358 fail: 13
good good output
Guile toml
pass: 186 fail: 5
pass: 187 fail: 4
pass: 357 fail: 14
(TODO) (TODO) output
Haskell toml-parser
pass: 190 fail: 1
pass: 191 fail: 0
pass: 369 fail: 2
good good output
Haskell toml-reader
pass: 189 fail: 2
(doesn't include encoder)
pass: 369 fail: 2
good slow output
JS deno-toml
pass: 162 fail: 28
pass: 102 fail: 88
pass: 246 fail: 125
(TODO) (TODO) output
JS j-toml
pass: 191 fail: 0
pass: 191 fail: 0
pass: 357 fail: 14
good good output
JS smol-toml
pass: 190 fail: 0
pass: 190 fail: 0
pass: 357 fail: 14
good good output
JS toml-eslint-parser
pass: 191 fail: 0
(doesn't include encoder)
pass: 362 fail: 9
good good output
Lisp clop
pass: 187 fail: 4
(doesn't include encoder)
pass: 371 fail: 0
(TODO) (TODO) output
Lua tinytoml
pass: 186 fail: 4
(doesn't include encoder)
pass: 370 fail: 1
(TODO) (TODO) output
OCAML otoml
pass: 188 fail: 3
pass: 183 fail: 8
pass: 319 fail: 52
(TODO) (TODO) output
Odin toml_parser
pass: 181 fail: 10
(doesn't include encoder)
pass: 166 fail: 205
(TODO) (TODO) output
Python toml
pass: 173 fail: 18
pass: 177 fail: 14
pass: 298 fail: 73
good slow output
Python tomli
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good good output
Python tomlkit
pass: 191 fail: 0
pass: 191 fail: 0
pass: 365 fail: 6
slow very slow output
Python tomllib
pass: 191 fail: 0
(doesn't include encoder)
pass: 371 fail: 0
good good output
Racket toml-racket
pass: 189 fail: 2
pass: 184 fail: 7
pass: 370 fail: 1
(TODO) (TODO) output
Ruby perfect_toml
pass: 191 fail: 0
pass: 190 fail: 1
pass: 362 fail: 9
good good output
Ruby toml-rb
pass: 171 fail: 20
pass: 163 fail: 28
pass: 301 fail: 70
slow very slow output
Ruby tomlrb
pass: 190 fail: 1
(doesn't include encoder)
pass: 351 fail: 20
good slow output
Rust basic-toml
pass: 170 fail: 21
(TODO)
pass: 367 fail: 4
good good output
Rust boml
pass: 191 fail: 0
(doesn't include encoder)
pass: 237 fail: 134
good good output
Rust taplo
pass: 185 fail: 6
(TODO)
pass: 368 fail: 3
good very slow output
Rust toml
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good good output
Rust toml_edit
pass: 191 fail: 0
pass: 191 fail: 0
pass: 371 fail: 0
good good output
Swift TOMLDecoder
pass: 190 fail: 1
(doesn't include encoder)
pass: 366 fail: 5
(TODO) (TODO) output