Yes, JQ can do all that. Here is how I'd write it:
jq -nr '
def valid_key: test("^[A-Za-z_][0-9A-Za-z_]*$");
def valid_value:
type != "array" and type != "object" and (
type != "string" or (test("\u0000") | not)
);
def valid_input:
type == "object" and (
to_entries | all(
(.key | valid_key) and (.value | valid_value)
)
);
input |
if valid_input and isempty(inputs) then
to_entries[] | "\(.key)=\(.value | @sh)"
else
"bad input\n" | halt_error(1)
end'
Answer from oguz ismail on Stack OverflowYes, JQ can do all that. Here is how I'd write it:
jq -nr '
def valid_key: test("^[A-Za-z_][0-9A-Za-z_]*$");
def valid_value:
type != "array" and type != "object" and (
type != "string" or (test("\u0000") | not)
);
def valid_input:
type == "object" and (
to_entries | all(
(.key | valid_key) and (.value | valid_value)
)
);
input |
if valid_input and isempty(inputs) then
to_entries[] | "\(.key)=\(.value | @sh)"
else
"bad input\n" | halt_error(1)
end'
Here's a demo how to do the testing. For production, however, it'd obviously need more refinement.
if type != "object" then "\(@json) is not an object (#1)"
else to_entries[] |
if .key | test("\\A[_a-zA-Z]\\w*\\z") | not then "\(.key | @json) is illegal (#2)"
else .value |
if isempty(scalars) then "\(.) is not a scalar (#3)"
else tostring |
if test("\u0000") then "\(.[index("\u0000")+1:] | @json) is preceded by NUL (#4)"
else empty
end
end
end
end // (to_entries[] | .key + "=" + (.value | @sh))
"" is illegal (#2)
"illegal key shall fail" is illegal (#2)
[1,2] is not a scalar (#3)
{"k":"v"} is not a scalar (#3)
".txt" is preceded by NUL (#4)
task_uuid='5d7ea654-b649-452b-a8fd-002b56be9a59'
task_name='hello world'
user_email=null
min=7
max=100
ave=2.5
file_in='data.txt'
file_out='results.txt'
Demo
Videos
You can use jq and send stdout to null
sudo apt-get install jq
jq . /path/to/yourfile.json 1> /dev/null
Example Execution
Here's some example runs
No Errors
If there are no errors in the json file, then there will be no output and the exit code is 0
user@host:~$ jq . updates/v1/meta.json 1> /dev/null
user@host:~$ echo $?
0
user@host:~$
Errors
If there are errors in the json file, then the output will tell you exactly which line & character, and the exit code will be non-zero
user@host:~$ jq . updates/v1/meta.json 1> /dev/null
parse error: Expected another array element at line 15, column 8
user@host:~$ echo $?
4
user@host:~$
sudo apt install jsonlint
and then do jsonlint-php {file}
Manual page:
OPTIONS
-q, --quiet
Cause jsonlint to be quiet when no errors are found
-h, --help
Show this message
From the manual:
-e / --exit-status:
Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.
So you can use:
if jq -e . >/dev/null 2>&1 <<<"$json_string"; then
echo "Parsed JSON successfully and got something other than false/null"
else
echo "Failed to parse JSON, or got false/null"
fi
In fact, if you don't care about distinguishing between the different types of error, then you can just lose the -e switch. In this case, anything considered to be valid JSON (including false/null) will be parsed successfully by the filter . and the program will terminate successfully, so the if branch will be followed.
This is working for me
echo $json_string | jq -e . >/dev/null 2>&1 | echo ${PIPESTATUS[1]}
that returns return code:
- 0 - Success
- 1 - Failed
- 4 - Invalid
Then you can evaluate the return code by further code.
I'm not really familiar with jq in particular, nevertheless, I hope this answer will be useful.
Writing a parser (which parses a file according to a fixed grammar) is generally a complicated and painful task.
Writing a parser that can correct mistakes is on a whole another level. I can't even imagine how complex it could be!
Think about the different ways the input you've shown could be "corrected".
Maybe that [2023.06.07-21.58.47] was meant to have , instead of . and -, that is, the array of six items [2023,06,07,21,58,47]... but then what to do with the rest?
Maybe the whole [2023.06.07-21.58.47] StatManagerLog: is meant to be dropped? Then the file is not a single JSON but a couple of JSON files concatenated; something that jq seems to be able to cope with.
Maybe it's meant to be "[2023.06.07-21.58.47] StatManagerLog":, i.e. the double quotes are missing around the key, and a big outer { } pair encapsulating the entire file is missing too.
I'm sure there are also a few other possibilities where fixes of about this size could make the file a valid JSON. The parser would have no idea which one to choose.
This is similar to spoken languages. Think of how many times you don't clearly hear the entire sentence and ask back because your brain couldn't automatically correct the unheard part... possibly because you can imagine more than 1 way of correcting the sentence and they would result in different meaning.
Long story short: Only you know how exactly the data needs to be fixed, what the original intent was; the parser has no chance of figuring it out. Fix it using some text processing tools (sed, awk, cut, etc.) before feeding it to jq. Or fix whoever emits this data to emit proper JSON.
Pre-processing the file by simply replacing each line that starts with [ with a { would make it valid JSON, or at least concatenation of valid JSONs which jq is also able to handle:
$ sed 's/^\[.*/{/' file | jq -c .
{"RoundState":{"State":"Starting","Timestamp":"2023.06.07-21.58.47"}}
{"RoundState":{"State":"StandBy","Timestamp":"2023.06.07-21.58.47"}}
I assume you don't need the removed information as it comprises data already included in the output (the timestamp) and static strings (StatManagerLog).
You may then select the entries that you need, e.g., by state or timestamp:
$ sed 's/^\[.*/{/' file | jq -c --arg q 2023.06.07-21.58.47 'select(.RoundState.Timestamp == $q)'
{"RoundState":{"State":"Starting","Timestamp":"2023.06.07-21.58.47"}}
{"RoundState":{"State":"StandBy","Timestamp":"2023.06.07-21.58.47"}}
$ sed 's/^\[.*/{/' file | jq -c --arg q StandBy 'select(.RoundState.State == $q)'
{"RoundState":{"State":"StandBy","Timestamp":"2023.06.07-21.58.47"}}
... or by some more involved selection of any aggregate that you need.
Is it now possible to validate JSON I'm editing in a vim8 buffer with the new json_decode API? Or how does recommend I solve this use case?