Benchmarks

schiva compared with ajv and nine other JSON Schema validators: how many tests of the JSON-Schema-Test-Suite each one passes, how fast it validates the suite and realistic payloads, how fast it compiles a schema, and how fast the features beyond validating are.

Summary

4,950tests of the JSON-Schema-Test-Suite, draft-04 to 2020-12: all passed (ajv: 4,836)
2.1×ajv's speed on the draft-07 test suite, first error
2.2×ajv's speed on the payloads below, on average (geometric mean), first error
44×ajv's speed compiling a schema

schiva passes every test of the five drafts it supports. In the same mode, it validates faster than every other library in every benchmark on this page, compiles faster than every other library that compiles schemas, and is faster than ajv on every feature they share (2.5× on average).

Every chart compares libraries doing the same work: pick the mode above it. true/false only tells whether the value is valid; first error stops at the first failing check and builds its error; all errors builds every error. Longer is faster, and next to each library is how many times slower than schiva it is.

Tests passed

Tests of the JSON-Schema-Test-Suite each library gets right, draft by draft (a schema it cannot compile counts as wrong for every test of its group). Only schiva and ajv run draft-04 and draft-06 here; drafts 2019-09 and 2020-12 run with the libraries that implement them.

draft-04draft-06draft-072019-092020-12
schiva
618 / 618
841 / 841
929 / 929
1261 / 1261
1301 / 1301
ajv
610 / 618
833 / 841
921 / 929
1233 / 1261
1239 / 1301
@exodus/schemasafenot runnot run
905 / 929
1228 / 1261
1263 / 1301
@cfworker/json-schemanot runnot run
919 / 929
1248 / 1261
1249 / 1301
json-schema-librarynot runnot run
907 / 929
1260 / 1261
1291 / 1301
is-my-json-validnot runnot run
801 / 929
not runnot run
jsennot runnot run
762 / 929
not runnot run
djvnot runnot run
791 / 929
not runnot run
jsonschemanot runnot run
905 / 929
not runnot run
tv4not runnot run
805 / 929
not runnot run
z-schemanot runnot run
859 / 929
not runnot run

Speed on the test suite

How many times per second each library validates every test of the groups all the libraries of the draft pass, so that each one does the same work.

Draft-07

runs per second over 430 tests

schiva
124k
@exodus/schemasafe
98.2k1.3× slower
ajv
60.2k2.1× slower

Draft 2019-09

runs per second over 1178 tests

schiva
22.1k
@exodus/schemasafe
14.9k1.5× slower
ajv
4.02k5.5× slower

Draft 2020-12

runs per second over 1051 tests

schiva
25.7k
@exodus/schemasafe
17.1k1.5× slower
ajv
13.8k1.9× slower

ajv's speed on drafts 2019-09 and 2020-12 changes a lot from one process to the next (between about 4k and 15k runs per second in our runs).

All the numbers
Runs per seconddraft-072019-092020-12
schiva (boolean)177k30.9k34.7k
@exodus/schemasafe (boolean)147k24.0k27.5k
schiva (first error)124k22.1k25.7k
schiva (all errors)101k17.1k19.4k
@exodus/schemasafe (first error)98.2k14.9k17.1k
@exodus/schemasafe (all errors)63.9k6.85k9.72k
ajv (first error)60.2k4.02k13.8k
ajv (all errors)51.5k3.61k11.7k
jsen42.6k––
is-my-json-valid33.9k––
djv5.54k––
@cfworker/json-schema1.22k251276
json-schema-library1.00k5574
jsonschema269––
tv495––
z-schema51––

Speed on payloads

Validations per second of one value, valid and invalid, by each library that compiles the schema and gives the right answer for it. The schemas are in bench/lib/cases.js.

moltar strict

The flat object of typescript-runtime-type-benchmarks, with no extra keys allowed.

moltar strict, valid

validations per second

schiva
33.1M
ajv
29.1M1.1× slower
@exodus/schemasafe
9.06M3.7× slower

moltar strict, invalid

validations per second

schiva
78.9M
ajv
36.0M2.2× slower
@exodus/schemasafe
11.7M6.7× slower

order (20 lines)

A business object with 20 order lines: pattern, enum, const, number ranges, anyOf, nullable types, uniqueItems.

order (20 lines), valid

validations per second

schiva
2.16M
ajv
737k2.9× slower
@exodus/schemasafe
546k4.0× slower

order (20 lines), invalid

validations per second

schiva
17.1M
ajv
12.2M1.4× slower
@exodus/schemasafe
7.20M2.4× slower

order 2020-12, unevaluatedProperties (20 lines)

The same order in draft 2020-12, with $ref, allOf and unevaluatedProperties.

order 2020-12, unevaluatedProperties (20 lines), valid

validations per second

schiva
2.32M
ajv
657k3.5× slower
@exodus/schemasafe
549k4.2× slower

order 2020-12, unevaluatedProperties (20 lines), invalid

validations per second

schiva
31.7M
ajv
15.0M2.1× slower
@exodus/schemasafe
10.1M3.2× slower

payment 2020-12, oneOf + unevaluatedProperties

A payment whose variant (oneOf) decides the keys it may have (unevaluatedProperties).

payment 2020-12, oneOf + unevaluatedProperties, valid

validations per second

schiva
15.2M
ajv
8.81M1.7× slower
@exodus/schemasafe
7.21M2.1× slower

payment 2020-12, oneOf + unevaluatedProperties, invalid

validations per second

schiva
14.5M
ajv
7.54M1.9× slower
@exodus/schemasafe
4.89M3.0× slower

shapes, discriminator (8 kinds)

A shape among 8, picked by an OpenAPI discriminator (ajv with its option discriminator: true).

shapes, discriminator (8 kinds), valid

validations per second

schiva
117M
ajv
44.8M2.6× slower
@exodus/schemasafe
29.8M3.9× slower

shapes, discriminator (8 kinds), invalid

validations per second

schiva
122M
ajv
37.1M3.3× slower
@exodus/schemasafe
32.7M3.7× slower
All the numbers

moltar strict

The flat object of typescript-runtime-type-benchmarks, with no extra keys allowed.

Validations per secondvalidinvalid
schiva (boolean)38.5M110M
schiva (first error)33.1M78.9M
schiva (all errors)30.6M20.8M
ajv (first error)29.1M36.0M
ajv (all errors)27.9M17.5M
is-my-json-valid23.4M16.2M
@exodus/schemasafe (boolean)9.18M13.1M
@exodus/schemasafe (first error)9.06M11.7M
jsen6.70M7.95M
@exodus/schemasafe (all errors)5.65M4.49M
tv4333k15.8k
json-schema-library250k135k
@cfworker/json-schema187k164k
jsonschema24.3k23.3k
z-schema5.84k5.50k
djv–89.3k

Left out: djv: wrong result (false).

order (20 lines)

A business object with 20 order lines: pattern, enum, const, number ranges, anyOf, nullable types, uniqueItems.

Validations per secondvalidinvalid
schiva (boolean)2.37M17.0M
schiva (all errors)2.23M1.86M
schiva (first error)2.16M17.1M
is-my-json-valid1.69M1.51M
ajv (first error)737k12.2M
ajv (all errors)712k439k
@exodus/schemasafe (first error)546k7.20M
@exodus/schemasafe (boolean)504k7.49M
@exodus/schemasafe (all errors)361k323k
jsen161k4.89M
json-schema-library14.0k12.8k
@cfworker/json-schema8.58k8.50k
jsonschema3.00k2.97k
z-schema2.04k1.94k
tv41.36k16.7k
djv–99.6k

Left out: djv: wrong result (false).

order 2020-12, unevaluatedProperties (20 lines)

The same order in draft 2020-12, with $ref, allOf and unevaluatedProperties.

Validations per secondvalidinvalid
schiva (boolean)2.46M36.7M
schiva (first error)2.32M31.7M
schiva (all errors)2.29M1.53M
ajv (first error)657k15.0M
ajv (all errors)655k381k
@exodus/schemasafe (boolean)572k10.9M
@exodus/schemasafe (first error)549k10.1M
@exodus/schemasafe (all errors)320k281k
@cfworker/json-schema7.97k7.84k
json-schema-library730464

payment 2020-12, oneOf + unevaluatedProperties

A payment whose variant (oneOf) decides the keys it may have (unevaluatedProperties).

Validations per secondvalidinvalid
schiva (boolean)16.2M15.5M
schiva (all errors)15.3M13.8M
schiva (first error)15.2M14.5M
ajv (first error)8.81M7.54M
ajv (all errors)8.00M6.21M
@exodus/schemasafe (first error)7.21M4.89M
@exodus/schemasafe (boolean)7.12M6.01M
@exodus/schemasafe (all errors)4.23M2.99M
json-schema-library140k8.42k
@cfworker/json-schema103k90.9k

shapes, discriminator (8 kinds)

A shape among 8, picked by an OpenAPI discriminator (ajv with its option discriminator: true).

Validations per secondvalidinvalid
schiva (boolean)121M162M
schiva (first error)117M122M
schiva (all errors)105M44.3M
is-my-json-valid85.1M38.7M
ajv (first error)44.8M37.1M
ajv (all errors)44.7M19.7M
@exodus/schemasafe (boolean)30.9M41.3M
@exodus/schemasafe (first error)29.8M32.7M
@exodus/schemasafe (all errors)16.8M13.3M
jsen1.68M1.76M
djv85.5k109k
json-schema-library37.9k26.0k
@cfworker/json-schema34.2k35.8k
jsonschema6.47k6.25k
tv42.98k2.47k
z-schema1.67k1.64k

Compiling

Schemas per second, from the order schema to a function that validates it. jsonschema, tv4 and @cfworker/json-schema interpret the schema instead of compiling it: their "compiling" only builds an object, and they pay for it at every validation, which is why they are the slowest on the payloads above.

schivacompiles the schemainterprets it at every validation

Compiling the order schema

schemas per second, first error

jsonschema
80.4k
tv4
79.6k
@cfworker/json-schema
31.6k
schiva
8.41k
json-schema-library
8.02k
jsen
5.37k1.6× slower
is-my-json-valid
3.53k2.4× slower
djv
3.47k2.4× slower
z-schema
1.20k7.0× slower
@exodus/schemasafe
81410× slower
ajv
19144× slower
All the numbers
Schemas per secondorder schema
jsonschema80.4k
tv479.6k
@cfworker/json-schema31.6k
schiva (boolean)10.6k
schiva (first error)8.41k
schiva (all errors)8.29k
json-schema-library8.02k
jsen5.37k
is-my-json-valid3.53k
djv3.47k
z-schema1.20k
@exodus/schemasafe (boolean)1.04k
@exodus/schemasafe (first error)814
@exodus/schemasafe (all errors)703
ajv (all errors)228
ajv (first error)191

Speed of the features

schiva and ajv on the features beyond validating JSON Schema, each with the same options and plugins (ajv-keywords, ajv-formats, ajv-draft-04, ajv's standalone code), with schiva's speed in times ajv's. With the options that change the data, each call validates a new object, made the same way for both. The cases are in bench/features.js.

custom keyword (validate)7.6×
schiva
899M
ajv
118M
ajv-keywords (range, regexp, prohibited, uniqueItemProperties)1.3×
schiva
28.3M
ajv
22.1M
useDefaults and removeAdditional (defaults filled, 2 keys removed)1.1×
schiva
2.62M
ajv
2.41M
coerceTypes (query string values)1.4×
schiva
19.4M
ajv
14.4M
formats with formatMinimum and formatMaximum1.1×
schiva
869k
ajv
769k
tagged oneOf without discriminator (8 kinds)20×
schiva
115M
ajv
5.72M
draft-04 (boolean exclusiveMaximum, id)2.2×
schiva
185M
ajv
83.3M
error objects (order with 20 lines, invalid)1.4×
schiva
16.9M
ajv
12.2M
standalone code (order with 20 lines, valid)3.3×
schiva
2.33M
ajv
696k
unevaluatedProperties (payment 2020-12)1.7×
schiva
14.9M
ajv
8.76M
All the numbers
Validations per secondschiva, all errorsajv, all errorsschiva, first errorajv, first errorschiva / ajv
custom keyword (validate)879M114M899M118M7.7×
ajv-keywords (range, regexp, prohibited, uniqueItemProperties)28.5M22.2M28.3M22.1M1.3×
useDefaults and removeAdditional (defaults filled, 2 keys removed)2.64M2.38M2.62M2.41M1.1×
coerceTypes (query string values)19.2M14.1M19.4M14.4M1.4×
formats with formatMinimum and formatMaximum876k759k869k769k1.2×
tagged oneOf without discriminator (8 kinds)103M3.93M115M5.72M26.2×
draft-04 (boolean exclusiveMaximum, id)184M82.5M185M83.3M2.2×
error objects (order with 20 lines, invalid)722k418k16.9M12.2M1.7×
standalone code (order with 20 lines, valid)2.37M683k2.33M696k3.5×
unevaluatedProperties (payment 2020-12)14.2M7.85M14.9M8.76M1.8×

schiva and ajv, feature by feature

schiva 0.6.0ajv 8.20.0
JSON Schema draftsdraft-04, draft-06, draft-07, 2019-09, 2020-12draft-06 to 2020-12; draft-04 with ajv-draft-04
JSON-Schema-Test-Suite, all five drafts4950 tests passed, all of them4836 tests passed
JSON Type Definition (JTD)–✓
Schema DSL for JavaScript✓ (String(), Integer()...), with inferred TypeScript types–
Messages with the path of the field✓ (lines[3].price must be at least 0)messages without the path; instancePath apart
Error objects✓✓
Every error, first error, or true/false✓✓
Standalone code✓ (no dependency at all)✓ (requires parts of ajv for some keywords)
Formats✓ built in, idn-hostname and idn-email includedwith ajv-formats
Optional format tests passed (2020-12)874 / 874733 / 874 (with ajv-formats)
formatMinimum, formatMaximum✓with ajv-formats
Keywords of your own: validate, compile, macro✓✓
Keywords of your own that write generated code–✓
ajv-keywords✓ built in (13 of them)with ajv-keywords
useDefaults, removeAdditional, coerceTypes✓✓
$data references–✓
Asynchronous validation ($async)–✓
Loading referenced documents (compileAsync)✓ compileJsonSchemaAsync✓
OpenAPI discriminator✓ with mapping and implicit namesconst and enum only
Tagged oneOf detected without discriminator✓–
strict mode✓✓
Translated messages, errorMessage–with ajv-i18n, ajv-errors
Dependencies04

See Migrating from ajv for the options, errors and APIs of each one.

Libraries

Every library of json-schema-benchmark that runs on current Node.js, with the drafts and modes each one is benchmarked in. (ajv always returns true or false, and builds the first error, or all of them, on the way.)

LibraryVersionDrafts benchmarkedModes benchmarked
schiva0.6.0draft-07, 2019-09, 2020-12all errors, first error, true/false
ajv8.20.0draft-07, 2019-09, 2020-12first error, all errors
@exodus/schemasafe1.3.0draft-07, 2019-09, 2020-12true/false, first error, all errors
@cfworker/json-schema4.1.1draft-07, 2019-09, 2020-12true/false
json-schema-library11.6.2draft-07, 2019-09, 2020-12true/false
is-my-json-valid2.20.6draft-07true/false
jsen0.6.6draft-07true/false
djv2.1.4draft-07true/false
jsonschema1.5.0draft-07true/false
tv41.3.0draft-07true/false
z-schema12.4.6draft-07true/false

Method

cd bench
pnpm install
pnpm start                          # test suite and payloads, isolated (about 30 minutes)
pnpm run features                   # the features of schiva and ajv
pnpm run conformance draft2020-12   # tests passed by schiva and ajv, file by file
pnpm run report                     # writes this page