NOTE: This solution assumes array1 has no duplicates.
Simple Explanation
The complexity of all these answers obscures understanding the principle. That's unfortunate because the principle is simple:
- array1 minus array2 returns:
- everything that's left in array1
- after removing everything that is in array2
- (and discarding the rest of array2)
Simple Demo
# From array1, subtract array2, leaving the remainder
$ jq --null-input '[1,2,3,4] - [2,4,6,8]'
[
1,
3
]
# Subtract the remainder from the original
$ jq --null-input '[1,2,3,4] - [1,3]'
[
2,
4
]
# Put it all together
$ jq --null-input '[1,2,3,4] - ([1,2,3,4] - [2,4,6,8])'
[
2,
4
]
comm Demo
def comm:
(.[0] - (.[0] - .[1])) as 
d, .[1]-
d]
;
With that understanding, I was able to imitate the behavior of the *nix comm command
With no options, produce three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and column three contains lines common to both files.
$ echo 'def comm: (.[0]-(.[0]-.[1])) as 
d,.[1]-
d];' > comm.jq
$ echo '{"a":101, "b":102, "c":103, "d":104}' > 1.json
$ echo '{ "b":202, "d":204, "f":206, "h":208}' > 2.json
$ jq --slurp '.' 1.json 2.json
[
{
"a": 101,
"b": 102,
"c": 103,
"d": 104
},
{
"b": 202,
"d": 204,
"f": 206,
"h": 208
}
]
$ jq --slurp '[.[] | keys | sort]' 1.json 2.json
[
[
"a",
"b",
"c",
"d"
],
[
"b",
"d",
"f",
"h"
]
]
$ jq --slurp 'include "comm"; [.[] | keys | sort] | comm' 1.json 2.json
[
[
"a",
"c"
],
[
"f",
"h"
],
[
"b",
"d"
]
]
$ jq --slurp 'include "comm"; [.[] | keys | sort] | comm[2]' 1.json 2.json
[
"b",
"d"
]
NOTE: This solution assumes array1 has no duplicates.
Simple Explanation
The complexity of all these answers obscures understanding the principle. That's unfortunate because the principle is simple:
- array1 minus array2 returns:
- everything that's left in array1
- after removing everything that is in array2
- (and discarding the rest of array2)
Simple Demo
# From array1, subtract array2, leaving the remainder
$ jq --null-input '[1,2,3,4] - [2,4,6,8]'
[
1,
3
]
# Subtract the remainder from the original
$ jq --null-input '[1,2,3,4] - [1,3]'
[
2,
4
]
# Put it all together
$ jq --null-input '[1,2,3,4] - ([1,2,3,4] - [2,4,6,8])'
[
2,
4
]
comm Demo
def comm:
(.[0] - (.[0] - .[1])) as d, .[1]-d]
;
With that understanding, I was able to imitate the behavior of the *nix comm command
With no options, produce three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and column three contains lines common to both files.
$ echo 'def comm: (.[0]-(.[0]-.[1])) as d,.[1]-d];' > comm.jq
$ echo '{"a":101, "b":102, "c":103, "d":104}' > 1.json
$ echo '{ "b":202, "d":204, "f":206, "h":208}' > 2.json
$ jq --slurp '.' 1.json 2.json
[
{
"a": 101,
"b": 102,
"c": 103,
"d": 104
},
{
"b": 202,
"d": 204,
"f": 206,
"h": 208
}
]
$ jq --slurp '[.[] | keys | sort]' 1.json 2.json
[
[
"a",
"b",
"c",
"d"
],
[
"b",
"d",
"f",
"h"
]
]
$ jq --slurp 'include "comm"; [.[] | keys | sort] | comm' 1.json 2.json
[
[
"a",
"c"
],
[
"f",
"h"
],
[
"b",
"d"
]
]
$ jq --slurp 'include "comm"; [.[] | keys | sort] | comm[2]' 1.json 2.json
[
"b",
"d"
]
A simple and quite fast (but somewhat naive) filter that probably does essentially what you want can be defined as follows:
# x and y are arrays
def intersection(x;y):
( (x|unique) + (y|unique) | sort) as $sorted
| reduce range(1; $sorted|length) as 
sorted[
sorted[$i-1] then . + [$sorted[$i]] else . end) ;
If x is provided as input on STDIN, and y is provided in some other way (e.g. def y: ...), then you could use this as: intersection(.;y)
Other ways to provide two distinct arrays as input include:
- using the
--slurpoption - using
--arg a v(or--argjson a vif available in your jq)
Here's a simpler but slower def that's nevertheless quite fast in practice:
def i(x;y):
if (y|length) == 0 then []
else (x|unique) as 
x - ($x - y)
end ;
Here's a standalone filter for finding the intersection of arbitrarily many arrays:
# Input: an array of arrays
def intersection:
def i(y): ((unique + (y|unique)) | sort) as $sorted
| reduce range(1; $sorted|length) as 
sorted[
sorted[$i-1] then . + [$sorted[$i]] else . end) ;
reduce .[1:][] as 
a)) ;
Examples:
[ [1,2,4], [2,4,5], [4,5,6]] #=> [4]
[[]] #=> []
[] #=> null
Of course if x and y are already known to be sorted and/or unique, more efficient solutions are possible. See in particular Finite Sets of JSON Entities
There are no set operations in javascript, but they are easy to define, for example:
intersection = function(a, b) {
return a.filter(function(x) { return b.indexOf(x) >= 0 })
}
For your specific task, first convert validTags to a list:
vtags = validTags.map(function(x) { return x.tag })
and then compute an intersection, converting each tag attribute in items into an array:
results = items.filter(function(x) {
return intersection(x.tag.split(/\s+/), vtags).length == vtags.length
})
Here is a solution using both objects and arrays:
validTags = [{"tag":"tag1"}, {"tag":"tag2"}];
items = [{"id":123456, "tag":"tag1"}, {"id":123456, "tag":"tag2"}, {"id":7890, "tag":"tag1"}];
accumulate = {};
// Make use of the hashing of JavaScript objects to merge the tags.
items.forEach(function(e) {
if(accumulate[e.id] == undefined) accumulate[e.id] = [e.tag];
else accumulate[e.id].push(e.tag);
});
// Convert the object into an array. The field 'tags' is still an array.
var result0 = [];
for(var id in accumulate) result0.push({"id": id, tags: accumulate[id]});
var result = result0.filter(
// First we cross out those do not contain every tag.
function(e) { return validTags.every(
function(e1) { return e.tags.indexOf(e1.tag) != -1; }); })
// Then we make the 'tags' array into a string.
.map(function(e) { return {"id": e.id, "tags": e.tags.join(" ")}; });
Assuming arrays do not contain duplicates (if they do, use unique to filter out duplicate results):
jq -cn '["a","b","e","c"] as $A | ["g","a","t","c"] as $B | $A - ($A - $B)'
echo '{ "group_a" : ["a","b","e","c"], "group_b" : ["g","a","t","c"] }' |
jq -c '.group_a - (.group_a - .group_b)'
They both yield:
["a","c"]
To get a general intersection operation for any number of arrays, here's my solution:
$ echo "[ [1, 2], [2, 3], [4, 2] ]" | \
jq 'reduce .[1:][] as $item(.[0]; . - (. - $item))'
[
2
]
Input: array of arrays. The inner arrays are those that should be intersected.
Output: array of all values that are present in all inner arrays of input.
The simplest solution would be to use sets, as long as you don't care that the elements in the result will have a different order, and that duplicates will be removed. The input arrays array1 and array2 are the Integer[] subarrays of the given int[] arrays corresponding to the number of elements that you intend to process:
Set<Integer> s1 = new HashSet<Integer>(Arrays.asList(array1));
Set<Integer> s2 = new HashSet<Integer>(Arrays.asList(array2));
s1.retainAll(s2);
Integer[] result = s1.toArray(new Integer[s1.size()]);
The above will return an Integer[], if needed it's simple to copy and convert its contents into an int[].
If you are fine with java-8, then the simplest solution I can think of is using streams and filter. An implementation is as follows:
public static int[] intersection(int[] a, int[] b) {
return Arrays.stream(a)
.distinct()
.filter(x -> Arrays.stream(b).anyMatch(y -> y == x))
.toArray();
}