jq . file.json
is what I was looking for. I didn't realize that the . is a filter and not a placeholder for the piped in content:
.The absolute simplest (and least interesting) filter is
.. This is a filter that takes its input and produces it unchanged as output.
And the man page makes it clear that the filter is a required argument.
Answer from k0pernikus on Stack ExchangeI did find a solution while I was writing the question: don't put the input inside string interpolation, output a stream of things:
echo '{"foo":"bar", "baz":[1,2,3]}' | jq -r '"My value is:", . , "Some other stuff"'
# .........................................................^^^^^
outputs
My value is:
{
"foo": "bar",
"baz": [
1,
2,
3
]
}
Some other stuff
This might not fit your actual use case, but instead of creating a single JSON string in jq, just use a shell command group.
echo '{"foo":"bar", "baz":[1,2,3]}' | { echo "My value is:"; jq .; echo "Some other stuff"; }
How to pretty print a JSON inside a string with jq? - Stack Overflow
jq does not pretty print when output is not a terminal
pretty print as the default
json - How to pretty print using jq, so that multiple values are on the same line? - Stack Overflow
Videos
This:
echo '{"foo": "bar"}' | jq '{other: .}' | jq -Rs '{json: .}'
produces:
{
"json": "{\n \"other\": {\n \"foo\": \"bar\"\n }\n}\n"
}
One way to remove the terminating "\n" would be to strip it:
echo '{"foo": "bar"}' | jq '{other: .}' | jq -Rs '{json: .[:-1]}'
I ended up writing a simple formatting function:
# 9 = \t
# 10 = \n
# 13 = \r
# 32 = (space)
# 34 = "
# 44 = ,
# 58 = :
# 91 = [
# 92 = \
# 93 = ]
# 123 = {
# 125 = }
def pretty:
explode | reduce .[] as $char (
{out: [], indent: [], string: false, escape: false};
if .string == true then
.out += [$char]
| if $char == 34 and .escape == false then .string = false else . end
| if $char == 92 and .escape == false then .escape = true else .escape = false end
elif $char == 91 or $char == 123 then
.indent += [32, 32] | .out += [$char, 10] + .indent
elif $char == 93 or $char == 125 then
.indent = .indent[2:] | .out += [10] + .indent + [$char]
elif $char == 34 then
.out += [$char] | .string = true
elif $char == 58 then
.out += [$char, 32]
elif $char == 44 then
.out += [$char, 10] + .indent
elif $char == 9 or $char == 10 or $char == 13 or $char == 32 then
.
else
.out += [$char]
end
) | .out | implode;
It adds unnecessary empty lines inside empty objects and arrays, but it's good enough for my purpose. For example (used on its own):
jq -Rr 'include "pretty"; pretty' test.json
where the function is saved in pretty.jq and test.json file is:
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"key":"string with \"quotes\" and \\"},"geometry":{"type":"Polygon","coordinates":[[[24.2578125,55.178867663281984],[22.67578125,50.958426723359935],[28.125,50.62507306341435],[30.322265625000004,53.80065082633023],[24.2578125,55.178867663281984]]]}}]}
gives:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"key": "string with \"quotes\" and \\"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
24.2578125,
55.178867663281984
],
[
22.67578125,
50.958426723359935
],
[
28.125,
50.62507306341435
],
[
30.322265625000004,
53.80065082633023
],
[
24.2578125,
55.178867663281984
]
]
]
}
}
]
}
While it is probably best to use a tool like the one peak suggested if your json isn't too complex you could use a second jq invocation to postprocess the output of the first. For example if your data is in data.json
$ jq -M . data.json | jq -MRsr 'gsub("\n +";"")|gsub("\n ]";"]")'
produces
{
"frameGrid": {
"size": [24,24],
"dimensions": [1,1],
"names": [["default"]]
}
}
As @jq170727 mentioned, postprocessing after a pretty-printing run of jq (e.g. jq .) is worth considering. In that vein, here is an awk script that might suffice:
#!/bin/bash
awk '
function ltrim(x) { sub(/^[ \t]*/, "", x); return x; }
s && NF > 1 && $NF == "[" { s=s $0; next}
s && NF == 1 && $1 == "]," { print s "],"; s=""; next}
s && NF == 1 && $1 == "[" { print s; s=$0; next}
s && NF == 1 && $1 == "{" { print s; print; s=""; next}
s && NF == 1 && $1 == "]" { print s $1; s=""; next}
s && NF == 1 && $1 == "}" { print s; s=$0; next}
s { s=s ltrim($0); next}
$NF == "[" { s=$0; next}
{print}
'
Examples
With the example input, the invocation:
jq . example.json | ./pp
produces:
{
"frameGrid": {
"size": [24,24],
"dimensions": [1,1],
"names": [
["default"]
]
}
}
The invocation:
jq -n '{a:[1,2,3,[1,2,3,4]],b:2,c:{d:[1,2,{e:[3,4]}]}}' | ./pp
produces:
{
"a": [1,2,3,
[1,2,3,4]
],
"b": 2,
"c": {
"d": [1,2,
{
"e": [3,4]
}
]
}
}
No, you can not process a file with jq and have it output the result to the original file.
You could use a temporary file like so:
cp file.json file.json.tmp &&
jq . file.json.tmp >file.json &&
rm file.json.tmp
This order of operations also retains the original file's metadata. Since each step depends on the successful completion of the previous step (due to &&), you will not lose the document if, for example, jq fails to run.
You may use a tool such as GNU sponge (part of the moreutils package) to hide the manual labour of handling a temporary file:
jq . file.json | sponge file.json
Note that this is still using a temporary file behind the scenes.
Out of these two variants, only the first set of three commands protects you from data loss in case your partition suddenly becomes full or jq fails to execute properly (due to being unavailable or because of an error in the input document).
json-beautify-inplace () {
temp=$(mktemp)
printf 'input = %s\n' "$1"
printf 'temp = %s\n' "$temp"
cp -- "$1" "$temp"
jq . "$temp" > "$1"
}
json-uglify-inplace () {
temp=$(mktemp)
printf 'input = %s\n' "$1"
printf 'temp = %s\n' "$temp"
cp -- "$1" "$temp"
jq -r tostring "$temp" > "$1"
}