If your jq has inputs then the simplest would probably be to use it:
jq -ncR '[inputs]' <<< "$groups"
["group1","group2","group3"]
Otherwise, here are three alternatives:
jq -c -n --arg groups "$groups" '$groups | split("\n")'
echo -n "$groups" | jq -cRs 'split("\n")'
echo "$groups" | jq -R -s -c 'split("\n") | map(select(length>0))'
In any case, the array can easily be incorporated into a JSON object, e.g. by extending the filter with | {groups: .}
If you really want to produce invalid JSON, consider:
printf "%s" "$groups" | jq -Rrsc 'split("\n") | "{ \(.) }"'
Output:
{ ["group_1","group_2","group_3"] }
Note on select(length>0)
Consider:
jq -Rsc 'split("\n")' <<< $'a\nb'
["a","b",""]
The reason for including select(length>0) is to avoid the trailing "".
If $groups contains consecutive newlines, and if it is important to retain the empty strings, then you might want to use [:-1], e.g.
jq -cRs 'split("\n")[:-1]' <<< "$groups"
["group1","group2","group3"]
If your jq does not support [:-1], make the 0 explicit: [0:-1]
Simply use printf to format the output into JSON
First, you have a very blatant typo in this part of your code right here:
echo "${array[3]:$var-3:4}
Note there is no closing straight quote: ". Fixed it in the rewrite I did below:
But more to the point, doing something like this (using printf) as suggested in this StackOverflow answer. Tested and works in CentOS 7.
#!/bin/bash
readarray -t array <<< "$(df -h)";
var=$(echo "${array[1]}"| grep -aob '%' | grep -oE '[0-9]+');
df_output="${array[3]:$var-3:4}";
manufacturer=$(cat /sys/class/dmi/id/chassis_vendor);
product_name=$(cat /sys/class/dmi/id/product_name);
version=$(cat /sys/class/dmi/id/bios_version);
serial_number=$(cat /sys/class/dmi/id/product_serial);
hostname=$(hostname);
operating_system=$(hostnamectl | grep "Operating System" | cut -d ' ' -f5-);
architecture=$(arch);
processor_name=$(awk -F':' '/^model name/ {print $2}' /proc/cpuinfo | uniq | sed -e 's/^[ \t]*//');
memory$(dmidecode -t 17 | grep "Size.*MB" | awk '{s+=$2} END {print s / 1024"GB"}');
hdd_model=$(cat /sys/block/sda/device/model);
system_main_ip=$(hostname -I);
printf '{"manufacturer":"%s","product_name":"%s","version":"%s","serial_number":"%s","hostname":"%s","operating_system":"%s","architecture":"%s","processor_name":"%s","memory":"%s","hdd_model":"%s","system_main_ip":"%s"}' "$manufacturer" "$product_name" "$version" "$serial_number" "$hostname" "$operating_system" "$architecture" "$processor_name" "$memory" "$hdd_model" "$system_main_ip"
The output I get is this:
{"manufacturer":"Oracle Corporation","product_name":"VirtualBox","version":"VirtualBox","serial_number":"","hostname":"sandbox-centos-7","operating_system":"CentOS Linux 7 (Core)","architecture":"x86_64","processor_name":"Intel(R) Core(TM) i5-1030NG7 CPU @ 1.10GHz","memory":"","hdd_model":"VBOX HARDDISK ","system_main_ip":"10.0.2.15 192.168.56.20 "}
And if you have jq installed, you can pipe the output of the shell script to jq to “pretty print” the output into some human readable format. Like let’s say your script is named my_script.sh, just pipe it to jq like this:
./my_script.sh | jq
And the output would look like this:
{
"manufacturer": "Oracle Corporation",
"product_name": "VirtualBox",
"version": "VirtualBox",
"serial_number": "",
"hostname": "sandbox-centos-7",
"operating_system": "CentOS Linux 7 (Core)",
"architecture": "x86_64",
"processor_name": "Intel(R) Core(TM) i5-1030NG7 CPU @ 1.10GHz",
"memory": "",
"hdd_model": "VBOX HARDDISK ",
"system_main_ip": "10.0.2.15 192.168.56.20 "
}
The following programs can output json:
lshw:
lshw -json
smartmontools v7+:
smartctl --json --all /dev/sda
lsblk:
lsblk --json
lsipc:
lsipc --json
sfdisk
sfdisk --json
I am writing a bash script for an Alfred Workflow. In this script I get a list (separated with newlines) from a command that I want to convert into a specific JSON format.
I tried storing the output into an array and parsing that in jq like that:
Command output:
$ piactl get regions auto france netherlands
Create array:
$ IFS=$'\n'
$ regions=($(piactl get regions))
$ echo "${regions[@]}"
auto france netherlandsParse to jq
$ jq -n --arg inarr "${regions}" '{ items: $inarr | split("\n") }'
{
"items": [
"auto"
]
}jq only outputs one item of the array and I don't know how to shape the JSON like shown in the wanted output below.
Wanted output:
{"items": [
{
"title": "auto",
"arg": "auto",
"icon": {
"path": "icon.png"
}
},
{
"title": "france",
"arg": "france",
"icon": {
"path": "icon.png"
}
},
{
"title": "netherlands",
"arg": "netherlands",
"icon": {
"path": "icon.png"
}
},
]}Can somebody help me craft the correct jq arguments for this task?
I have the following string in bash
"3.8,3.9,3.10"
Is there a way using shell to convert it into a json array, i.e.
["3.8", "3.9", "3.10"\]
If you know that no filename contains newlines, use jq:
ls | jq -R -s -c 'split("\n")[:-1]'
Short explanation of the flags to jq:
-Rtreats the input as string instead of JSON-sjoins all lines into an array-ccreates a compact output[:-1]removes the last empty string in the output array
This requires version 1.4 or later of jq. Try this if it doesn't work for you:
ls | jq -R '[.]' | jq -s -c 'add'
Yes, but the corner cases and Unicode handling will drive you up the wall. Better to delegate to a scripting language that supports it natively.
python -c 'import os, json; print json.dumps(os.listdir("."))'
["\u00e0", "\"a\"", "\u79c1", "a b", "\u3042", "a"]
Using jq :
readarray arr < <(jq '.[].item2' json)
printf '%s\n' "${arr[@]}"
If you need a more hardened way:
readarray -td '' arr
for inputs with newlines or other special characters, avoiding word splitting.
Output:
value2
value2_2
Check:
Process Substitution >(command ...) or <(...) is replaced by a temporary filename. Writing or reading that file causes bytes to get piped to the command inside. Often used in combination with file redirection: cmd1 2> >(cmd2).
See http://mywiki.wooledge.org/ProcessSubstitution http://mywiki.wooledge.org/BashFAQ/024
The following is actually buggy:
# BAD: Output line of * is replaced with list of local files; can't deal with whitespace
arr=( $( curl -k "$url" | jq -r '.[].item2' ) )
If you have bash 4.4 or newer, a best-of-all-worlds option is available:
# BEST: Supports bash 4.4+, with failure detection and newlines in data
{ readarray -t -d '' arr && wait "$!"; } < <(
set -o pipefail
curl --fail -k "$url" | jq -j '.[].item2 | (., "\u0000")'
)
...whereas with bash 4.0, you can have terseness at the cost of failure detection and literal newline support:
# OK (with bash 4.0), but can't detect failure and doesn't support values with newlines
readarray -t arr < <(curl -k "$url" | jq -r '.[].item2' )
...or bash 3.x compatibility and failure detection, but without newline support:
# OK: Supports bash 3.x; no support for newlines in values, but can detect failures
IFS=$'\n' read -r -d '' -a arr < <(
set -o pipefail
curl --fail -k "$url" | jq -r '.[].item2' && printf '\0'
)
...or bash 3.x compatibility and newline support, but without failure detection:
# OK: Supports bash 3.x and supports newlines in values; does not detect failures
arr=( )
while IFS= read -r -d '' item; do
arr+=( "$item" )
done < <(curl --fail -k "$url" | jq -j '.[] | (.item2, "\u0000")')
I was also trying to convert a bunch of lines into a JSON array, and was at a standstill until I realized that -s was the only way I could handle more than one line at a time in the jq expression, even if that meant I'd have to parse the newlines manually.
jq -R -s -c 'split("\n")' < just_lines.txt
-Rto read raw input-sto read all input as a single string-cto not pretty print the output
Easy peasy.
Edit: I'm on jq ≥ 1.4, which is apparently when the split built-in was introduced.
--raw-input, then --slurp
Just summarizing what the others have said in a hopefully quicker to understand form:
cat /etc/hosts | jq --raw-input . | jq --slurp .
will return you:
[
"fe00::0 ip6-localnet",
"ff00::0 ip6-mcastprefix",
"ff02::1 ip6-allnodes",
"ff02::2 ip6-allrouters"
]
Explanation
--raw-input/-R:
Don´t parse the input as JSON. Instead, each line of text is passed
to the filter as a string. If combined with --slurp, then the
entire input is passed to the filter as a single long string.
--slurp/-s:
Instead of running the filter for each JSON object in the input,
read the entire input stream into a large array and run the filter
just once.
free -k | { read
read TITLE TOTAL USED REST
echo "{\"Memory\":\"$(( 100 * $USED / $TOTAL ))\"}"
}
The output of free is piped to a compound command consisting of:
A first "read" which skips the first output line of "free".
A second "read" which reads the line we need, we need only the second and third value.
An echo which prints the line in the format you want including the calculation
Here's an example for memory:
echo {\"Memory\":\"$(awk '/^Mem/ {printf("%u", 100*$3/$2);}' <(free -m))\"} > mem.json
Putting that new file name into json:
echo {\"file\":\"$(ls mem.json)\"} > filename.json
Or:
echo {\"<paramName-here>\":\"$(<value-of-param-from-command-here>)\"} > mem.json
When it gets more complicated than this, you can continue to write line by line or more likely build strings in a variable.
Assuming you have the correct values in the shell variables label, last_year, and previous_year and that you want to delete all entries in the options array with the value value $previous_year, and add a new entry with the label value $label and value value $last_year.
jq --arg add_label "$label" \
--arg add_value "$last_year" \
--arg del_value "$previous_year" '
del(.options[]? | select(.value == $del_value)) |
.options += [{ label: $add_label, value: $add_value }]' file
This correctly combines the two operations in a single invocation of jq.
The command instantiates three internal jq string variables with values taken from the shell variables that you have previously created. Using --arg to create internal string variables from shell variables ensures that the values are correctly encoded (this avoids code injection vulnerabilities).
As in your code, I use .options[]? rather than just .options[] to avoid an error if the options array does not exist. If it does exist, any entry with a value value of $del_value (an internal variable with the value from the shell variable previous_year) is deleted. You tried to select using the label, but I believe this might have been a typo.
The (possibly) modified document is passed on to the next stage, which adds a new element to the options array. If the array did not previously exist, this would create it with a single element.
Also note that jq does not do in-place editing, which means you will need to redirect the output of the above command to a new name and then replace the original file with that new file. Alternatively, use GNU sponge:
jq ...as above... file | sponge file
(Assuming your file is called file.)
#!/bin/bash
file=temp.json
now="$(date)"
previous_year=$(date --date="$(date +'%Y') - 1 year" +%Y)
last_year=$((previous_year + 31))
label=$((last_year -2000))
echo "==============================================================================================================="
echo "| Started parsing json file at script at: $file"
echo "| Started at: $now"
echo "| previous year is: $previous_year"
echo "| last year to be at json array is: $last_year"
echo "| label assigned to the last year : $label"
echo "==============================================================================================================="
if cat $file | grep $previous_year
then
jq --arg key "$previous_year" 'del(.options[] | select(.value == $key))' $file >> new.json
jq --arg key "$label" --arg val "$last_year" '.options += [{
"label": $key,
"value": $val
}]' new.json >> new2.json
mv new2.json $file
rm new.json
echo "last year found and parsed"
else
echo "nothing to be done"
fi
now="$(date)"
echo "==============================================================================================================="
echo "| Ended script at: $now"
echo "==============================================================================================================="
´´´
You can do this:
X=("hello world" "goodnight moon")
printf '%s\n' "${X[@]}" | jq -R . | jq -s .
output
[
"hello world",
"goodnight moon"
]
Since jq 1.6 you can do this:
jq --compact-output --null-input '$ARGS.positional' --args -- "${X[@]}"
giving:
["hello world","goodnight moon"]
This has the benefit that no escaping is required at all. It handles strings containing newlines, tabs, double quotes, backslashes and other control characters. (Well, it doesn't handle NUL characters but you can't have them in a bash array in the first place.)