The built-in JSON module can be used as a validator:
import json
def parse(text):
try:
return json.loads(text)
except ValueError as e:
print('invalid json: %s' % e)
return None # or: raise
You can make it work with files by using:
with open(filename) as f:
return json.load(f)
instead of json.loads and you can include the filename as well in the error message.
On Python 3.3.5, for {test: "foo"}, I get:
invalid json: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
and on 2.7.6:
invalid json: Expecting property name: line 1 column 2 (char 1)
This is because the correct json is {"test": "foo"}.
When handling the invalid files, it is best to not process them any further. You can build a skipped.txt file listing the files with the error, so they can be checked and fixed by hand.
If possible, you should check the site/program that generated the invalid json files, fix that and then re-generate the json file. Otherwise, you are going to keep having new files that are invalid JSON.
Failing that, you will need to write a custom json parser that fixes common errors. With that, you should be putting the original under source control (or archived), so you can see and check the differences that the automated tool fixes (as a sanity check). Ambiguous cases should be fixed by hand.
Answer from reece on Stack OverflowThe built-in JSON module can be used as a validator:
import json
def parse(text):
try:
return json.loads(text)
except ValueError as e:
print('invalid json: %s' % e)
return None # or: raise
You can make it work with files by using:
with open(filename) as f:
return json.load(f)
instead of json.loads and you can include the filename as well in the error message.
On Python 3.3.5, for {test: "foo"}, I get:
invalid json: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
and on 2.7.6:
invalid json: Expecting property name: line 1 column 2 (char 1)
This is because the correct json is {"test": "foo"}.
When handling the invalid files, it is best to not process them any further. You can build a skipped.txt file listing the files with the error, so they can be checked and fixed by hand.
If possible, you should check the site/program that generated the invalid json files, fix that and then re-generate the json file. Otherwise, you are going to keep having new files that are invalid JSON.
Failing that, you will need to write a custom json parser that fixes common errors. With that, you should be putting the original under source control (or archived), so you can see and check the differences that the automated tool fixes (as a sanity check). Ambiguous cases should be fixed by hand.
Yes, there are ways to validate that a JSON file is valid. One way is to use a JSON parsing library that will throw exceptions if the input you provide is not well-formatted.
try:
load_json_file(filename)
except InvalidDataException: # or something
# oops guess it's not valid
Of course, if you want to fix it, you naturally cannot use a JSON loader since, well, it's not valid JSON in the first place. Unless the library you're using will automatically fix things for you, in which case you probably wouldn't even have this question.
One way is to load the file manually and tokenize it and attempt to detect errors and try to fix them as you go, but I'm sure there are cases where the error is just not possible to fix automatically and would be better off throwing an error and asking the user to fix their files.
I have not written a JSON fixer myself so I can't provide any details on how you might go about actually fixing errors.
However I am not sure whether it would be a good idea to fix all errors, since then you'd have assume your fixes are what the user actually wants. If it's a missing comma or they have an extra trailing comma, then that might be OK, but there may be cases where it is ambiguous what the user wants.
Videos
Try jsonlint:
sudo apt install jsonlint
The basic usage syntax is
jsonlint YOUR-FILE.JSON
You find its manual by typing man jsonlint or visiting its online manpage:
An excerpt:
NAME
jsonlint - A JSON syntax validator and formatter tool
SYNOPSIS
jsonlint [-v][-s|-S][-f|-F][-ecodec]inputfile.json...
[...]
OPTIONS
The return status will be 0 if the file is legal JSON, or non-zero
otherwise. Use -v to see the warning details.
[...]
-v, --verbose
Show details of lint checking
-s, --strict
Be strict in what is considered legal JSON (the default)
-S, --nonstrict
Be loose in what is considered legal JSON
-f, --format
Reformat the JSON (if legal) to stdout
[...]
So you can see whether your JSON is valid by checking the return code of jsonlint. You can see it by running echo $? right afterwards (0=OK, 1=invalid), or by evaluating it using &&, || or if.
I tried jsonlint but it doesn't work.
jq . may-file.json work nice!
Hope this feedback is helpful.
I have yet to find a system where python -mjson.tool doesn't work. So you can do:
python -mjson.tool "$somefile" > /dev/null
The exit code will be nonzero and you get the parse error on stderr if the file is not valid JSON.
Note: The Python libraries don't follow the JSON spec and allow NaN and Infinity as values. So using json.tool will let some errors slip through. Still it's good enough for my use case of catching errors in human-written documents early.
The very simplest way is using json_pp (json pretty print) from perl.
$ echo {} | json_pp
{}
$ echo [] | json_pp
[]
$ echo 5 | json_pp
5
$ echo a | json_pp
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "a\n") at /usr/bin/json_pp line 59.
For file you can do
$ cat file.json | json_pp
For more info
$ man json_pp
You can use jq and send stdout to null
sudo apt-get install jq
jq . /path/to/yourfile.json 1> /dev/null
Example Execution
Here's some example runs
No Errors
If there are no errors in the json file, then there will be no output and the exit code is 0
user@host:~$ jq . updates/v1/meta.json 1> /dev/null
user@host:~$ echo $?
0
user@host:~$
Errors
If there are errors in the json file, then the output will tell you exactly which line & character, and the exit code will be non-zero
user@host:~$ jq . updates/v1/meta.json 1> /dev/null
parse error: Expected another array element at line 15, column 8
user@host:~$ echo $?
4
user@host:~$
sudo apt install jsonlint
and then do jsonlint-php {file}
Manual page:
OPTIONS
-q, --quiet
Cause jsonlint to be quiet when no errors are found
-h, --help
Show this message