I'm looking for an offline command line program that can validate JSON files on Ubuntu: I have a folder containing a list of JSON files, some possibly corrupted, so I would have to have the list of corrupted JSON files.
2 Answers
Without the need for a specific application, if you just want to separate the wheat from the chaff, and have PHP installed on the Linux machine, you can use a simple one-liner for this:
for json in folder/*; do php -r "if ( ! \$foo=json_decode(file_get_contents('$json')) ) echo \"$json\n\";"; done
This would list all broken .json
files in the directory named folder
. Take out the exlamation mark (!
) to only list the .json
files which are fine.
It might well be PHP spits out some additional error message here. If that happens, just re-direct error output (STDERR) to the machine's black hole (/dev/null
):
for json in folder/*; do php -r "if ( ! \$foo=json_decode(file_get_contents('$json')) ) echo \"$json\n\";" 2>/dev/null; done
TL;DR – a short explanation of what that one-liner does
for json in folder/*; do […] done
: loop over all.json
files in the given location. On each iteration, store the file name into a variable named$json
.php -r
: invoke the PHP executable. The-r
parameter tells it to (r)un a command instead of expecting a file. The command must directly follow this, separated by white-space."[…]"
: using double-quotes, we can easier integrate shell variables. The price is we have to escape all$
signs belonging to PHP variables.\$foo
: see previous point, we need to escape the$
here.$foo=json_decode(file_get_contents('$json'))
: have PHP reading the file (file_get_contents
; the$json
here is our Bash variable from the first bullet-point), then convert the JSON to a PHP object (json_decode
). We don't want to see the result, so we assign it to a variable ($foo
in my example).
If decoding was successful, the assignment will returnTRUE
, otherwiseFALSE
– which is why we can use this construction as our "if condition".if ( […] ) echo \"$json\n\";"
: only write the file name (this again is our Bash variable here) to the console if our condition is met – to "separate the wheat from the chaff", as I've put it initially. As usual in PHP, the command needs to be terminated by a semi-colon (;
). Also note that I needed to escape the double-quotes here, so our "outer Shell wrapper" doesn't end prematurely. I needed the double-quotes, so\n
is interpreted as "new line" – with single-quotes, it would print a literal\n
instead.2>/dev/null
: Redirect error output (STDERR) to the "black hole" (i.e. don't show any errors if they occur). Ourecho
commands go to STDOUT, and thus are not affected by this.
You can use jsonlint:
- CLI
- open source (written in JavaScript)
- check whether file(s) are valid:
Example:
# Install
sudo apt-get install -y nodejs npm
sudo npm install jsonlint -g
sudo ln -s /usr/bin/nodejs /usr/bin/node # to avoid "/usr/bin/env: node: No such file or directory" error
jsonlint -qc *.json
# Use
ubuntu@pm:~/pubmed/pubmed$ jsonlint --compact --validate *.json
my_file_bad.json: line 279916, col 18, found: 'EOF' - expected: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['.
ubuntu@pm:~/pubmed/pubmed$
Useful options:
-c, --compact compact error display
-V, --validate a JSON schema to use for validation
for json in folder/*; do echo $json; php -r "\$foo=json_decode(file_get_contents('$json'));"; done
This will echo all file names followed by errors for that file if there are any – otherwise the filename only. Adjust for perfection:for json in folder/*; do php -r "if ( ! \$foo=json_decode(file_get_contents('$json')) ) echo \"$json has errors\n\";"; done
, which would only list the corrupted files.