The problem is that jq is still just outputting lines of text; you can't necessarily preserve each array element as a single unit. That said, as long as a newline is not a valid character in any object, you can still output each object on a separate line.

get_json_array | jq -c '.[]' | while read object; do
    api_call "$object"
done

Under that assumption, you could use the readarray command in bash 4 to build an array:

readarray -t conversations < <(get_json_array | jq -c '.[]')
for conversation in "${conversations[@]}"; do
    api_call "$conversation"
done
Answer from chepner on Stack Overflow
Discussions

More fun with jq, getting results into a usable array
It's better practice to not chain together results within $() or backticks. Unless you're expecting a huge volume of data returned from your curl calls, it's better to capture it's result directly, so you can explicitly check for failure of the curl call itself, and failure of the request. If it looks valid, then proceed to parse it as you wish. It looks like you might not need to put your fields into arrays. You might be able to use jq -c to iterate through your initial curl call result, and then pull the values you need into separate variables, which you can then use to make the next requests. Something like: foo=$( curl something ) || { echo >&2 "curl failed"; exit 1; } # check $foo for http errors if $foo invalid; then exit 1; fi json_from_foo=$( filter_result "$foo" ) || { echo >&2 "failed to extract json from result"; exit 1; } for json_record in $( jq -c '.[]' <<< "$json_from_foo); do action=$(jq -r '.action' <<< $json_record) expr=$(jq -r '.expression' <<< $json_record) desc=$(jq -r '.description' <<< $json_record) process_record "$action" "$expr" "$desc" done More on reddit.com
🌐 r/bash
2
2
August 17, 2024
bash - jq: output array of json objects - Stack Overflow
44 How to convert a JSON object stream into an array with jq More on stackoverflow.com
🌐 stackoverflow.com
Assigning an Array Parsed With jq to Bash Script Array - Stack Overflow
To correctly parse values that ... to generate space-separated quoted strings, and Bash's declare -a to parse the quoted strings as array elements. (No pre-processing required) ... As of version 1.7, jq has a --raw-output0 option, enabling it to output null-terminated strings ... More on stackoverflow.com
🌐 stackoverflow.com
ubuntu - parse one field from an JSON array into bash array - Unix & Linux Stack Exchange
I was just hoping to avoid it. ... Great jq command, but please don't parse command output into an array with arr=( $(...) ) (even though it happens to work with the sample input): it doesn't work as intended with embedded or leading/trailing whitespace and can result in accidental globbing. More on unix.stackexchange.com
🌐 unix.stackexchange.com
People also ask

What does the jq command do in converting JSON arrays to Bash arrays?
The jq command extracts each element of the JSON array in converting such an array into Bash arrays by using the .[] notation within the full syntax echo "$json_array" | jq -r '.[]' and outputs the elements as separate lines. The -r option outputs strings without quotes to make them suitable for Bash array assignment.
🌐
linuxsimply.com
linuxsimply.com › home › bash scripting tutorial › a complete guide to bash array › array operations in bash › how to convert a json array into bash array [5 methods]
How to Convert a JSON Array into Bash Array [5 Methods] - LinuxSimply
How to convert a JSON array into a Bash array?
To convert a JSON array into a Bash array, you can use the jq command line tool with process substitution with the syntax bash_array=($(echo "$json_array" | jq -r '.[]')). This parses the JSON array stored in the variable json_array and assigns each element to the bash_array.
🌐
linuxsimply.com
linuxsimply.com › home › bash scripting tutorial › a complete guide to bash array › array operations in bash › how to convert a json array into bash array [5 methods]
How to Convert a JSON Array into Bash Array [5 Methods] - LinuxSimply
What is the jq command in Linux?
The Linux "jq" command is a flexible and lightweight tool to process and manipulate JSON data directly using the command line or within the Bash scripts. With the "jq" command, it is possible to parse data easily, filter based on specific fields, and transform JSON objects and arrays efficiently to work.
🌐
linuxsimply.com
linuxsimply.com › home › bash scripting tutorial › a complete guide to bash array › array operations in bash › how to convert a json array into bash array [5 methods]
How to Convert a JSON Array into Bash Array [5 Methods] - LinuxSimply
🌐
Eliatra
eliatra.com › home › blog › transform json data on bash using jq
Transform JSON Data on Bash Using jq
November 23, 2022 - First, we tell jq that we want to process an array of objects and that we want to add “2” to the “EmployedSince” field: ... The output of this command is our original JSON array of objects but with the incremented “EmployedSince” field.
🌐
Reddit
reddit.com › r/bash › more fun with jq, getting results into a usable array
r/bash on Reddit: More fun with jq, getting results into a usable array
August 17, 2024 -

I'm using this in a cURL to get the data from result[]:

foo=$(curl --request GET \
--silent \
--url https://example.com \
--header 'Content-Type: application/json' | jq -r '.result[]')

When I print $foo, this is what I have:

[key]
default

firewall_custom
zone
34
[
  {
    "id": "example",
    "version": "6",
    "action": "block",
    "expression": "lorem",
    "description": "ipsum",
    "last_updated": "2024-08-15T19:10:24.913784Z",
    "ref": "example",
    "enabled": true
  },
  {
    "id": "example2",
    "version": "7",
    "action": "block",
    "expression": "this",
    "description": "that",
    "last_updated": "2024-08-15T19:10:24.913784Z",
    "ref": "example2",
    "enabled": true
  }
]

What I need from this is to create a loop where, in a series of addtional cURLs, I can insert action, expression, and description.

I'm imagining that I would push these to 3 separate arrays (action, expression, and description), so that ${action[0]} would coincide with ${expression[0]} and ${description[0]}, and so on.

Something along the lines of:

# assuming that I have somehow created the following arrays:
# action=("block" "block")
# expression=("lorem" "this")
# description=("ipsum" "that")

for x in ${action[@]}; do
  bar=$(curl --request GET \
    --silent \
    --url https://example.com \
    --data '{
      "action": ${action[$x]},
      "expression": ${expression[$x]},
      "description": ${description[$x]}
    }' | jq '.success')

  if [[ $bar == true ]]
    then
      printf "$x succeeded\n"

    else
      printf "$x failed\n"
  fi

  # reset bar
  bar=''
done

The question is, how to create action, expression, and description arrays from the results of $foo (that original cURL)?

🌐
Ingernet
ingernet.github.io › bash › jq › json › 2020 › 04 › 16 › json-array-bash-array.html
Converting a JSON array to a bash array
April 16, 2020 - # using gcloud output as a source because why not use the hardest shit possible bork=$(gcloud --project=<project-id> container images list-tags us.gcr.io/<project-id>/<image-name> --filter='tags:DEPLOYED' --format=json | jq '.[0].tags') echo $bork [ "260", "61a1d7aef75421f5c209c42304716ba44e86ab7a", "DEPLOYED.2019-11-12T17.04.37.772145800Z", "DEPLOYED.2019-11-13T00.00.29.525908800Z" ] # ^ output is obviously not a bash array # strip out all the things you don't want - square brackets and commas borkstring=$(echo $bork | sed -e 's/\[ //g' -e 's/\ ]//g' -e 's/\,//g') arr=( $borkstring ) echo $arr ( "260" "61a1d7aef75421f5c209c42304716ba44e86ab7a" "DEPLOYED.2019-11-12T17.04.37.772145800Z" "DEPLOYED.2019-11-13T00.00.29.525908800Z" ) # ^ now THAT is a bash array
🌐
LinuxSimply
linuxsimply.com › home › bash scripting tutorial › a complete guide to bash array › array operations in bash › how to convert a json array into bash array [5 methods]
How to Convert a JSON Array into Bash Array [5 Methods] - LinuxSimply
March 17, 2024 - The “jq” command extracts each array element (item1 item2 item3) and -r flag outputs them without the quotes. Finally, the command substitution method within the syntax bash_array=($(echo "$json_array" | jq -r '.[]')) extracts the elements ...
Find elsewhere
Top answer
1 of 5
33

We can solve this problem by two ways. They are:

Input string:

// test.json
{
    "keys": ["key1","key2","key3"]
}

Approach 1:

1) Use jq -r (output raw strings, not JSON texts) .

KEYS=$(jq -r '.keys' test.json)
echo $KEYS
# Output: [ "key1", "key2", "key3" ]

2) Use @sh (Converts input string to a series of space-separated strings). It removes square brackets[], comma(,) from the string.

KEYS=$(<test.json jq -r '.keys | @sh')
echo $KEYS
# Output: 'key1' 'key2' 'key3'

3) Using tr to remove single quotes from the string output. To delete specific characters use the -d option in tr.

KEYS=$((<test.json jq -r '.keys | @sh')| tr -d \') 
echo $KEYS
# Output: key1 key2 key3

4) We can convert the comma-separated string to the array by placing our string output in a round bracket(). It also called compound Assignment, where we declare the array with a bunch of values.

ARRAYNAME=(value1 value2  .... valueN)
#!/bin/bash
KEYS=($((<test.json jq -r '.keys | @sh') | tr -d \'\"))

echo "Array size: " ${#KEYS[@]}
echo "Array elements: "${KEYS[@]}

# Output: 
# Array size:  3
# Array elements: key1 key2 key3

Approach 2:

1) Use jq -r to get the string output, then use tr to delete characters like square brackets, double quotes and comma.

#!/bin/bash
KEYS=$(jq -r '.keys' test.json  | tr -d '[],"')
echo $KEYS

# Output: key1 key2 key3

2) Then we can convert the comma-separated string to the array by placing our string output in a round bracket().

#!/bin/bash
KEYS=($(jq -r '.keys' test.json  | tr -d '[]," '))

echo "Array size: " ${#KEYS[@]}
echo "Array elements: "${KEYS[@]}

# Output:
# Array size:  3
# Array elements: key1 key2 key3
2 of 5
16

To correctly parse values that may have newlines (and any other arbitrary (non-NUL) characters) use jq's @sh filter to generate space-separated quoted strings, and Bash's declare -a to parse the quoted strings as array elements. (No pre-processing required)

foo.json:

{"data": ["$0", " \t\n", "*", "\"", ""]}
str=$(jq -r '.data | @sh' foo.json)
declare -a arr="($str)"   # must be quoted like this
declare -p arr
# declare -a arr=([0]="\$0" [1]=$' \t\n' [2]="*" [3]="\"" [4]="")

Update: jq 1.7 (2023-09)

As of version 1.7, jq has a --raw-output0 option, enabling it to output null-terminated strings which can be read into an array as usual:

mapfile -d '' arr < <(jq --raw-output0 '.data[]' foo.json)
wait "$!"  # use in bash-4.4+ to get exit status of the process substitution

Note on NUL characters in JSON strings

JSON strings may contain NUL characters while shell variables cannot. If your JSON input may contain NUL's, you may need to add some special handling.

  • When using the @sh filter, NUL characters from JSON strings will be silently replaced with the sequence \0. Note that this makes the JSON strings "\\0" and "\u0000" indistinguishable.

  • When using the --raw-output0 option, NUL characters will trigger an error and jq will terminate with an exit status of 5.

Reading multiple/nested arrays

The @sh filter can be combined with --raw-output0 to reliably read multiple arrays at once (or a single nested array) as it will produce a NUL-separated list of space-separated quoted strings.

json='[[1,2],[3,4]]' i=0
while read -r -d ''; do
    declare -a "arrREPLY)"
done < <(jq --raw-output0 '.[]|@sh' <<<$json)
for ((n=0; n<i; n++)); { declare -p "arr$n"; }
# declare -a arr0=([0]="1" [1]="2")
# declare -a arr1=([0]="3" [1]="4")
🌐
GitHub
gist.github.com › IAmStoxe › a36b6f043819fad1821e7cfd7e903a5b
This example shows you how to utilize jq to loop bash script through an array of JSON values. · GitHub
June 29, 2020 - echo '[{"name": "name #1", "value": "value #1"} , {"name": "name#2", "value": "value#2"}]' | jq --raw-output 'map("\(.name) = \(.value)")[]' ... Or if you really did want to drop down into the bash loop...
Top answer
1 of 6
36

Using jq :

readarray arr < <(jq '.[].item2' json)
printf '%s\n' "${arr[@]}"

If you need a more hardened way:

readarray -td '' arr

for inputs with newlines or other special characters, avoiding word splitting.

Output:

value2
value2_2

Check:

Process Substitution >(command ...) or <(...) is replaced by a temporary filename. Writing or reading that file causes bytes to get piped to the command inside. Often used in combination with file redirection: cmd1 2> >(cmd2). See http://mywiki.wooledge.org/ProcessSubstitution http://mywiki.wooledge.org/BashFAQ/024

2 of 6
9

The following is actually buggy:

# BAD: Output line of * is replaced with list of local files; can't deal with whitespace
arr=( $( curl -k "$url" | jq -r '.[].item2' ) )

If you have bash 4.4 or newer, a best-of-all-worlds option is available:

# BEST: Supports bash 4.4+, with failure detection and newlines in data
{ readarray -t -d '' arr && wait "$!"; } < <(
  set -o pipefail
  curl --fail -k "$url" | jq -j '.[].item2 | (., "\u0000")'
)

...whereas with bash 4.0, you can have terseness at the cost of failure detection and literal newline support:

# OK (with bash 4.0), but can't detect failure and doesn't support values with newlines
readarray -t arr < <(curl -k "$url" | jq -r '.[].item2' )

...or bash 3.x compatibility and failure detection, but without newline support:

# OK: Supports bash 3.x; no support for newlines in values, but can detect failures
IFS=$'\n' read -r -d '' -a arr < <(
  set -o pipefail
  curl --fail -k "$url" | jq -r '.[].item2' && printf '\0'
)

...or bash 3.x compatibility and newline support, but without failure detection:

# OK: Supports bash 3.x and supports newlines in values; does not detect failures
arr=( )
while IFS= read -r -d '' item; do
  arr+=( "$item" )
done < <(curl --fail -k "$url" | jq -j '.[] | (.item2, "\u0000")')
Top answer
1 of 4
5

To convert your JSON to a bash array, with help of jq:

$ readarray -t arr < <(jq '.value' file)
$ printf '%s\n' "${arr[@]}"
"1"
"3"
"4"

To fix your expanded example (the exact command), just don't use object construction {value: .Value}, but instead only .Value:

$ readarray -t arr < <(aws ec2 describe-instances --region=us-east-1 --filters --filters "Name=tag:NodeType,Values=worker" --query "Reservations[].Instances[].Tags[]" | jq -r '.[] | select(.Key == "NodeNumber") | .Value')
$ printf '%s\n' "${arr[@]}"
1
3
4

Notice the lack of double quotes, since the -r option now prints only raw string values, not raw JSON Objects.

After you get arr populated with values like this, you can easily iterate over it and perform tests, just as you described in your question.

2 of 4
2

First, Store the Data

Given your raw data stored as a string in a json variable, perhaps with a here-document:

json=$(
    cat <<- EOF
        {
          "value": "1"
        }
        {
          "value": "3"
        }
        {
          "value": "4"
        }
EOF
)

Bash itself will do a reasonable job of prettifying it:

$ echo $json
{ "value": "1" } { "value": "3" } { "value": "4" }

Parsing the Data Into a Bash Array

There's more than one way to do this. Two of the more obvious ways are to use use jq or grep to extract the values into a Bash array with the shell's simple array notation:

values=( `echo $json | jq .value` )
echo "${values[@]}"
"1" "3" "4"

unset values
values=$(egrep -o '[[:digit:]]+' <<< "$json")
echo "${values[@]}"
1
3
4

There are certainly other ways to accomplish this task, but this seems like the low-hanging fruit. Your mileage may vary.

Caveat

The main thing to remember about Bash arrays is that they need to use expansions such as "${foo[@]}" when quoted, or ${bar[*]} when unquoted, if you want to use them for loops rather than indexing into them directly. Once they're in an array, though, you can access the entire array easily as long as you understand the different expansions and quoting rules.

🌐
how.wtf
how.wtf › how-to-iterate-through-json-arrays-in-bash-using-jq.html
How to iterate through JSON arrays in Bash using jq | how.wtf
June 14, 2023 - To achieve this, you can use the ... do 2 echo "$i" 3done · The command uses jq to extract each object in the “projects” array and then uses a while loop to iterate through the extracted objects....
🌐
Brazil's Blog
blog.kellybrazil.com › home › blog feed › practical json at the command line
Practical JSON at the Command Line - Brazil's Blog
June 25, 2021 - ... The simplest scenario is to pull a single value from the JSON data we are interested in. If we run rpm -qia | jc --rpm-qi we will get a JSON array of rpm metadata objects to work with.
🌐
Cameronnokes
cameronnokes.com › blog › working-with-json-in-bash-using-jq
Working with JSON in bash using jq - Cameron Nokes
August 1, 2020 - Also, be sure to always wrap your jq selector in a single-quotes, otherwise bash tries to interpret all the symbols like ., whereas we want jq to do that. Now let’s see how iteration works. The array or object value iterator operator, .[] , is what makes it possible. ... That will output 1, 2, 3 on separate lines.
🌐
Reddit
reddit.com › r/bash › convert json array to bash array
r/bash on Reddit: Convert JSON array to bash array
December 25, 2024 -

Hi guys,

I am a linux noob and am trying to write a script to extract info from a mkv file using mkvmerge but am not able to convert the target json script to a bash array. I have tried a number of solutions from stack overflow but with no success.

here are some of my attempts

dir="/mnt/Anime/Series/KonoSuba/Season 2/[Nep_Blanc] KonoSuba II 10 .mkv"
*********************************************************************************
ARRAY_SIZE=$(mkvmerge -J  "$dir" | jq '.tracks | length')
count=0
arr=()

while [ $count -lt $ARRAY_SIZE ];
    do
        arr+=($(mkvmerge -J  "$dir" | jq '.tracks'[$count]))
        ((count++))
done
*********************************************************************************
readarray -t test_array < <(mkvmerge -J  "$dir" | jq '.tracks')
for element in "${test_array[@]}";
    do
        echo "$element"
done

*********************************************************************************
array=($(mkvmerge -J  "$dir" | jq '.tracks' | sed -e 's/^\[/(/' -e 's/\]$/)/'))

but the echo prints out lines instead of the specific objects.

Though now it is helpling me with my python, originally the project was to help me learn bash scripting. I would really like to have a bash implementation so any help overcoming this roadblock would be appreciated.

🌐
Alfred App Community
alfredforum.com › alfred workflows › workflow help & questions
Construct script filter JSON with jq from bash array - Workflow Help & Questions - Alfred App Community Forum
January 30, 2022 - Hi, I am creating a workflow and I want to create a script filter with strings from a bash array. Given I have an array called $regions that contains 3 strings. I want to create 3 list items from that using jq. echo $regions france netherlands denmark I tried this command but the output is obviou...
🌐
Reddit
reddit.com › r/bash › writing bash code for array with multiple values
r/bash on Reddit: Writing bash code for array with multiple values
October 24, 2022 -

I have a json file which i'm using bash to extract.

sample.json

{"extract": { "data": [ {"name": "John Smith", "id": 8752, "address": "1 Anywhere Street", "tel": 1234567890, "email": "john.smith@gmail.com" }, { "name": "Jane Smith", "id": 4568, "address": "719 Anywhere Street", "tel": 0987654321, "email": "janesmith@hotmail.com" } ] } }

and store the value within an array

id=($(cat sample.json | jq -r '.extract.data[] .name'))

so in the case of ${id[0]} will output John Smith and ${id[1]} will output Jane Smith.

I am intending to store the values in a database (this will be my first attempt) which will be in a similar to that of the json, each object needs to be relative to how it is in the json so it might be better to go with:

data1=($(cat sample.json | jq -r '.extract.data[0] | .[]))

Lets say i have 1000 names to save to my database along with their id's. I'm some advice whether if there a more sensible (more effective) approach on how:

- Pull the data from Json? will I need to write this 1000 times?e.g

data1=($(cat sample.json | jq -r '.extract.data[0] | .[]))
data2=($(cat sample.json | jq -r '.extract.data[1] | .[]))
data3=($(cat sample.json | jq -r '.extract.data[2] | .[]))
..
data1=($(cat sample.json | jq -r '.extract.data[1000] | .[]))

-Put the data into the DB from the first array? will the code need to reference the array as:

${data1[0]}
${data1[1]} 
${data1[2]}

Would be grateful for a steer in the right direction? - thanks.

🌐
CodeGenes
codegenes.net › blog › assigning-an-array-parsed-with-jq-to-bash-script-array
How to Assign a jq-Parsed JSON Array to a Bash Script Array: Fixing Bracket and Format Issues — codegenes.net
Now, we need to read these line-separated elements into a Bash array. The safest way to do this is with mapfile (or its synonym readarray), which reads lines from standard input into an array, preserving spaces and special characters.