This tests various parsers agains the toml-test suite.

"Decoder tests" check if valid TOML documents are processed correctly, "encoder tests" checks writing TOML files, and "invalid input" checks that the parser rejects invalid TOML documents. Implementations are marked as "compliant" (✅) if both the "valid" and "encoder" tests pass, otherwise they're marked as non-compliant (❌). Usually it's not a big deal to accept some technically invalid TOML; almost all failing "invalid" tests are uncommon edge cases.

The benchmarks are a simple decode of a 15k and 5M file. Results are displayed as "good", "slow", and "very slow" only. Doing a fully fair comparison is not straight-forward, and it's not a contest on who can write the fastest parser – it's just to establish that performance is within reasonable bounds.

Show only:
Name Decoder tests Encoder tests Invalid input 15k file 5M file
C++ nullptr-toml
pass: 201 fail: 4
(doesn't include encoder)
pass: 524 fail: 5
(TODO) (TODO) output
C++ toml++
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good good output
C++ toml11
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good slow output
C toml-c
pass: 205 fail: 0
(doesn't include encoder)
pass: 514 fail: 15
good slow output
C tomlc99
pass: 194 fail: 11
(doesn't include encoder)
pass: 435 fail: 94
good slow output
C# tomlet
pass: 188 fail: 17
(TODO)
pass: 409 fail: 120
(TODO) (TODO) output
C# tomlyn
pass: 201 fail: 4
(TODO)
pass: 520 fail: 9
slow good output
Dart toml.dart
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good good output
Fortran toml-f
pass: 191 fail: 14
pass: 192 fail: 13
pass: 492 fail: 37
good very slow output
Go go-toml
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good slow output
Go toml
pass: 205 fail: 0
pass: 205 fail: 0
pass: 515 fail: 14
good good output
Guile toml
pass: 199 fail: 6
pass: 200 fail: 5
pass: 515 fail: 14
(TODO) (TODO) output
JS deno-toml
pass: 174 fail: 30
pass: 115 fail: 89
pass: 420 fail: 109
good good output
JS j-toml
pass: 205 fail: 0
pass: 205 fail: 0
pass: 515 fail: 14
good good output
JS smol-toml
pass: 203 fail: 1
pass: 204 fail: 0
pass: 512 fail: 17
good good output
JS toml-eslint-parser
pass: 195 fail: 10
(doesn't include encoder)
pass: 519 fail: 10
good good output
Lisp clop
pass: 200 fail: 5
(doesn't include encoder)
pass: 529 fail: 0
(TODO) (TODO) output
Lua tinytoml
pass: 201 fail: 2
(doesn't include encoder)
pass: 526 fail: 3
(TODO) (TODO) output
OCAML otoml
pass: 201 fail: 4
pass: 197 fail: 8
pass: 463 fail: 66
(TODO) (TODO) output
Odin toml_parser
pass: 185 fail: 20
(doesn't include encoder)
pass: 428 fail: 101
(TODO) (TODO) output
Python toml
pass: 183 fail: 22
pass: 191 fail: 14
pass: 418 fail: 111
good slow output
Python tomli
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good good output
Python tomlkit
pass: 205 fail: 0
pass: 205 fail: 0
pass: 522 fail: 7
slow very slow output
Python tomllib
pass: 205 fail: 0
(doesn't include encoder)
pass: 529 fail: 0
good good output
Racket toml-racket
pass: 202 fail: 3
pass: 198 fail: 7
pass: 525 fail: 4
(TODO) (TODO) output
Ruby perfect_toml
pass: 205 fail: 0
pass: 204 fail: 1
pass: 520 fail: 9
good good output
Ruby toml-rb
pass: 180 fail: 25
pass: 177 fail: 28
pass: 446 fail: 83
slow very slow output
Ruby tomlrb
pass: 202 fail: 3
(doesn't include encoder)
pass: 502 fail: 27
good slow output
Rust basic-toml
pass: 184 fail: 21
(TODO)
pass: 525 fail: 4
good good output
Rust boml
pass: 185 fail: 20
(doesn't include encoder)
pass: 364 fail: 165
good good output
Rust taplo
pass: 198 fail: 7
(TODO)
pass: 525 fail: 4
good very slow output
Rust toml
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good good output
Rust toml_edit
pass: 205 fail: 0
pass: 205 fail: 0
pass: 529 fail: 0
good good output
Swift TOMLDecoder
pass: 198 fail: 7
(doesn't include encoder)
pass: 520 fail: 9
(TODO) (TODO) output