With the key: value data in input.txt, and the following program in tojson.jq:
[inputs | select(length>0)
| [splits(": *")]
| {(.[0]): .[1]} ]
| add
the invocation:
jq -n -R -f tojson.jq input.txt
produces:
{
"k1": "v1",
"k2": "v2",
"k3": "v3"
}
Answer from peak on Stack OverflowWith the key: value data in input.txt, and the following program in tojson.jq:
[inputs | select(length>0)
| [splits(": *")]
| {(.[0]): .[1]} ]
| add
the invocation:
jq -n -R -f tojson.jq input.txt
produces:
{
"k1": "v1",
"k2": "v2",
"k3": "v3"
}
For those interested, here is a bash function which can create json dicts, by handling basic json types by adding a prefix to the bash values:
- string (prefix:
s:) - boolean (prefix:
b:) - number (prefix:
n:)
Here's the code:
#!/bin/bash
json_dict() {
for s in "$@"; do
echo -ne "$s\0"
done \
| jq -R 'splits("\u0000")' \
| jq -s -c '
_nwise(2)
| {
(.[0]):
(
if .[1]|startswith("s:") then .[1][2:]
elif .[1]|startswith("n:") then .[1][2:]|tonumber
elif .[1] == "b:true" then true
elif .[1] == "b:false" then false
else .[1]
end
)
}
' \
| jq -s add
}
key_values=(
k1 s:foo
k2 b:true
k3 n:123
)
json_dict "${key_values[@]}"
Output:
{
"k1": "foo",
"k2": true,
"k3": 123
}
json - Convert an array of strings to a dictionary with JQ? - Stack Overflow
json - Create single dictionary with jq - Stack Overflow
jq - Add key/value to json object - Unix & Linux Stack Exchange
question: how to concactenate values from list of dicts?
The following script uses the .ip_prefix as the key, thus perhaps avoiding the need for the sort|unique. It yields:
{
"35.180.0.0/16": "35.180.0.0/16",
"52.94.76.0/22": "52.94.76.0/22"
}
Script
#!/bin/bash
function data {
cat <<EOF
{
"syncToken": "1589917992",
"createDate": "2020-05-19-19-53-12",
"prefixes": [
{
"ip_prefix": "35.180.0.0/16",
"region": "eu-west-3",
"service": "AMAZON",
"network_border_group": "eu-west-3"
},
{
"ip_prefix": "52.94.76.0/22",
"region": "us-west-2",
"service": "AMAZON",
"network_border_group": "us-west-2"
}
]
}
EOF
}
data | jq '
.prefixes
| map(select(.region | test("west"))
| {(.ip_prefix): .ip_prefix} )
| add '
There's a better option to get at the AWS IP ranges data in Terraform, which is to use the aws_ip_ranges data source, instead of trying to mangle things with the external data source and jq.
The example in the above linked documentation shows a similar, but also slightly more complex, thing to what you're trying to do here:
data "aws_ip_ranges" "european_ec2" {
regions = ["eu-west-1", "eu-central-1"]
services = ["ec2"]
}
resource "aws_security_group" "from_europe" {
name = "from_europe"
ingress {
from_port = "443"
to_port = "443"
protocol = "tcp"
cidr_blocks = data.aws_ip_ranges.european_ec2.cidr_blocks
ipv6_cidr_blocks = data.aws_ip_ranges.european_ec2.ipv6_cidr_blocks
}
tags = {
CreateDate = data.aws_ip_ranges.european_ec2.create_date
SyncToken = data.aws_ip_ranges.european_ec2.sync_token
}
}
To do your exact thing you would do something like this:
data "aws_ip_ranges" "us_west_2_amazon" {
regions = ["us_west_2"]
services = ["amazon"]
}
resource "aws_default_security_group" "allow-mysql" {
vpc_id = aws_vpc.main.id
ingress {
description = "MySQL"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = data.aws_ip_ranges.us_west_2_amazon.cidr_blocks
}
}
However, there are 2 things that are bad here.
The first, and most important, is that you're allowing access to your database from every IP address that AWS has in US-West-2 across all services. That means that anyone in the world is able to spin up an EC2 instance or Lambda function in US-West-2 and then have network access to your database. This is a very bad idea.
The second is that if that returns more than 60 CIDR blocks you are going to end up with more than 60 rules in your security group. AWS security groups have a limit of 60 security group rules per IP address type (IPv4 vs IPv6) and per ingress/egress:
You can have 60 inbound and 60 outbound rules per security group (making a total of 120 rules). This quota is enforced separately for IPv4 rules and IPv6 rules; for example, a security group can have 60 inbound rules for IPv4 traffic and 60 inbound rules for IPv6 traffic. A rule that references a security group or prefix list ID counts as one rule for IPv4 and one rule for IPv6.
From https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html#vpc-limits-security-groups
This is technically a soft cap and you can ask AWS to raise this limit in exchange for reducing the amount of security groups that can be applied to a network interface to keep the maximum amount of security group rules at or below 1000 per network interface. It's probably not something you want to mess around with though.
Like this:
$ jq '.compilerOptions.skipLibCheck=true' file.json
{
"compileOnSave": false,
"compilerOptions": {
"baseUrl": "./",
"skipLibCheck": true
}
}
The easiest way to add the key with its value has been covered in another answer. That answer adds the key to the end of the list of keys in the compilerOptions object. Normally, the ordering of the keys does not matter, and if you need things ordered in a particular way, you will use an array. However, I'm noticing that you (for whatever reason) expect the key to be added first, before the existing baseUrl key.
We can add the key in that position by, instead of adding the new key to the existing object, instead, add the existing object's keys to the end of the new key. So given the existing JSON document,
{
"compileOnSave": false,
"compilerOptions": {
"baseUrl": "./",
"jerry": "Was a race car driver"
}
}
... we may want to use
jq --argjson skipLibCheck true '.compilerOptions = $ARGS.named + .compilerOptions' file
Given our example document above, this would generate
{
"compileOnSave": false,
"compilerOptions": {
"skipLibCheck": true,
"baseUrl": "./",
"jerry": "Was a race car driver"
}
}
The $ARGS.named thing is an object which contains the key-value pairs defined with --arg and/or --argjson on the command line. In the example above, this would be {"skipLibCheck":true}. Note that the $ARGS feature was introduced after release 1.5 of jq.
With the older 1.5 release of jq, you may use
jq --argjson skipLibCheck true '.compilerOptions = { skipLibCheck: $skipLibCheck } + .compilerOptions' file
Use --arg instead of --argjson if you want the value to be the string true rather than the special boolean value true.
The following gives an alternative way of adding the key at the end (to what's mentioned in the other answer), which follows the same pattern as the above command. Note that I'm also switching to using --arg here to insert true as a string, just to show how that looks.
jq --arg skipLibCheck true '.compilerOptions += $ARGS.named' file
... which would give you
{
"compileOnSave": false,
"compilerOptions": {
"baseUrl": "./",
"jerry": "Was a race car driver",
"skipLibCheck": "true"
}
}
With the older 1.5 release of jq, you may use
jq --arg skipLibCheck true '.compilerOptions += { skipLibCheck: $skipLibCheck }' file