If you care for performance, you should not combine a linear search with another linear search; with the resulting time complexity can’t be fixed with parallel processing when the lists get large.
You should built a data structure which allows efficient lookups first:
Map<List<?>,Map<String, Object>> inputKeys = inputRecords.stream()
.collect(Collectors.toMap(
m -> Arrays.asList(m.get("ID"),m.get("Email")),
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchinRecords = dbRecords.stream()
.filter(m -> inputKeys.containsKey(Arrays.asList(m.get("ID"),m.get("Email"))))
.collect(Collectors.toList());
matchinRecords.forEach(m -> inputKeys.remove(Arrays.asList(m.get("ID"),m.get("Email"))));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
This solution will keep the identity of the Maps.
If you are only interested in the values associated with the "Email" key, it would be much simpler:
Map<Object,Object> notMatchinRecords = inputRecords.stream()
.collect(Collectors.toMap(
m -> m.get("ID"),
m -> m.get("Email"),
(a,b) -> { throw new IllegalStateException("duplicate"); },
LinkedHashMap::new));
Object notPresent = new Object();
Map<Object,Object> matchinRecords = dbRecords.stream()
.filter(m -> notMatchinRecords.getOrDefault(m.get("ID"), notPresent)
.equals(m.get("Email")))
.collect(Collectors.toMap(
m -> m.get("ID"),
m -> m.get("Email"),
(a,b) -> { throw new IllegalStateException("duplicate"); },
LinkedHashMap::new));
notMatchinRecords.keySet().removeAll(matchinRecords.keySet());
System.out.println("Matching Records: " + matchinRecords.size());
matchinRecords.forEach((id,email) -> System.out.println(email));
System.out.println("Non Matching Records" + notMatchinRecords.size());
notMatchinRecords.forEach((id,email) -> System.out.println(email));
The first variant can get extended to support more/other map entries easily:
List<String> keys = Arrays.asList("ID", "Email");
Function<Map<String,Object>,List<?>> getKey
= m -> keys.stream().map(m::get).collect(Collectors.toList());
Map<List<?>,Map<String, Object>> inputKeys = inputRecords.stream()
.collect(Collectors.toMap(
getKey,
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchinRecords = dbRecords.stream()
.filter(m -> inputKeys.containsKey(getKey.apply(m)))
.collect(Collectors.toList());
matchinRecords.forEach(m -> inputKeys.remove(getKey.apply(m)));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
Answer from Holger on Stack OverflowI have two lists containing maps, the four keys are instrument, party, quantity, and entity.
If the values match between query1 and query2 results I want to add the matching maps into a list. So we can tell map1 in query1 matches map3 in query2. This is for reconciliation of data where there will be a plethora of records so essentially we want to go through each map in query1 results and compare to all maps in query2 results until there is a match. Once there is a match, we want to create a unique identifier or some sort of a pair to show that the two maps are equal and thus can be deemed a match. If we go through a full iteration and map1 in query1 does not equal any maps in query 2 we can then mark map1 in query1 a break, which would be adding the key to a break list.
Please let me know if you need more information. I've added the 3 different attempts I have tried, however none seem to make a match.
public void reconcile(LocalDate date) {
List<Map<String, Object>> query1Records;
List<Map<String, Object>> query2Records;
try (Stream<Map<String, Object>> stream = executeQuery1(date)) {
query1Records = stream.collect(Collectors.toList());
}
try (Stream<Map<String, Object>> stream = executeQuery2(date)) {
query2Records = stream.collect(Collectors.toList());
}
// ATTEMPT #1
List<Map<String,Object>> unMatchedRecords = query1Records.parallelStream().filter(searchData ->
query2Records.parallelStream().noneMatch( inputMap ->
searchData.entrySet().stream().noneMatch(value ->
inputMap.entrySet().stream().noneMatch(value1 ->
(value1.getKey().equals(value.getKey()) &&
value1.getValue().equals(value.getValue()))))
)).collect(Collectors.toList());
// ATTEMPT #2
List<Map<String, Object>> matching = query1Records.parallelStream().filter(searchData ->
query2Records.parallelStream().anyMatch(inputMap ->
String.valueOf(searchData.get("quantity")).equals(inputMap.get("quantity"))
&& searchData.get("instrument").equals(inputMap.get("instrument"))
&& searchData.get("entity").equals(inputMap.get("entity"))
&& searchData.get("party").equals(inputMap.get("party"))))
.collect(Collectors.toList());
// ATTEMPT #3
List<String> keysToReconcile = Arrays.asList("quantity", "instrument", "entity", "party");
Function<Map<String,Object>,List<?>> getKey
= m -> keysToReconcile.stream().map(m::get).collect(Collectors.toList());
Map<List<?>,Map<String, Object>> inputKeys = query1Records.stream()
.collect(Collectors.toMap(
getKey,
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchingRecords = query2Records.stream()
.filter(m -> inputKeys.containsKey(getKey.apply(m)))
.collect(Collectors.toList());
matchingRecords.forEach(m -> inputKeys.remove(getKey.apply(m)));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
}
}
If you care for performance, you should not combine a linear search with another linear search; with the resulting time complexity can’t be fixed with parallel processing when the lists get large.
You should built a data structure which allows efficient lookups first:
Map<List<?>,Map<String, Object>> inputKeys = inputRecords.stream()
.collect(Collectors.toMap(
m -> Arrays.asList(m.get("ID"),m.get("Email")),
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchinRecords = dbRecords.stream()
.filter(m -> inputKeys.containsKey(Arrays.asList(m.get("ID"),m.get("Email"))))
.collect(Collectors.toList());
matchinRecords.forEach(m -> inputKeys.remove(Arrays.asList(m.get("ID"),m.get("Email"))));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
This solution will keep the identity of the Maps.
If you are only interested in the values associated with the "Email" key, it would be much simpler:
Map<Object,Object> notMatchinRecords = inputRecords.stream()
.collect(Collectors.toMap(
m -> m.get("ID"),
m -> m.get("Email"),
(a,b) -> { throw new IllegalStateException("duplicate"); },
LinkedHashMap::new));
Object notPresent = new Object();
Map<Object,Object> matchinRecords = dbRecords.stream()
.filter(m -> notMatchinRecords.getOrDefault(m.get("ID"), notPresent)
.equals(m.get("Email")))
.collect(Collectors.toMap(
m -> m.get("ID"),
m -> m.get("Email"),
(a,b) -> { throw new IllegalStateException("duplicate"); },
LinkedHashMap::new));
notMatchinRecords.keySet().removeAll(matchinRecords.keySet());
System.out.println("Matching Records: " + matchinRecords.size());
matchinRecords.forEach((id,email) -> System.out.println(email));
System.out.println("Non Matching Records" + notMatchinRecords.size());
notMatchinRecords.forEach((id,email) -> System.out.println(email));
The first variant can get extended to support more/other map entries easily:
List<String> keys = Arrays.asList("ID", "Email");
Function<Map<String,Object>,List<?>> getKey
= m -> keys.stream().map(m::get).collect(Collectors.toList());
Map<List<?>,Map<String, Object>> inputKeys = inputRecords.stream()
.collect(Collectors.toMap(
getKey,
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchinRecords = dbRecords.stream()
.filter(m -> inputKeys.containsKey(getKey.apply(m)))
.collect(Collectors.toList());
matchinRecords.forEach(m -> inputKeys.remove(getKey.apply(m)));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
You just need to add a condition in the comparison
dbRecords.parallelStream().filter(searchData ->
inputRecords.parallelStream().anyMatch(inputMap ->
searchData.get("Email").equals(inputMap.get("Email"))
&& searchData.get("id").equals(inputMap.get("id"))))
.collect(Collectors.toList());
- Add the same in the
noneMatch(). - Compute the average time using
System.nanoTime(), it's more accurate - Try with and without
.parallelStream()(just.stream()) because not sure it helps you)
You could loop over the List one time and compare Maps in each list. This would cut your Big-O runtime in half -- assumes that the Maps are in the proper order (i.e., via LinkedList).
public boolean compare(List<Map<String, Object>> A, List<Map<String, Object>> B) {
// check size first
if (A.size() == B.size()) {
// if the Maps are abstracted into a POJO you could implement Comparator on that POJO. In the meantime you can sort manually
// sort A
Collections.sort(A, new Comparator<Map<String, String>>() {
public int compare(final Map<String, String> o1, final Map<String, String> o2) {
// optionally, you could use any method of sorting here... single field like an ID, multiple fields compared, sums, etc.
return o1.get("field").compareTo(o2.get("field"));
}
});
// sort B
Collections.sort(B, new Comparator<Map<String, String>>() {
public int compare(final Map<String, String> o1, final Map<String, String> o2) {
return o1.get("field").compareTo(o2.get("field"));
}
});
for (int i = 0; i < A.size(); i++) {
// get map from A & B
Map<String, Object> aMap = A.get(i);
Map<String, Object> bMap = B.get(i);
// check equality of Maps
if (!aMap.equals(bMap)) {
return false;
}
}
} else {
// Data reconciliation failed :: Data size mismatch
return false;
}
// if we get here then all was good
return true;
}
Here it is mate... The most efficient way to compare two List of Map to identify the matching and non matching records with multiple filter predicates in Java8 Streams is:
List<Map<String,String>> unMatchedRecords = dbRecords.parallelStream().filter(searchData ->
inputRecords.parallelStream().noneMatch( inputMap ->
searchData.entrySet().stream().noneMatch(value ->
inputMap.entrySet().stream().noneMatch(value1 ->
(value1.getKey().equals(value.getKey()) &&
value1.getValue().equals(value.getValue()))))
)).collect(Collectors.toList());
Note:
If <Map<String,String> used above is <Map<Object,Object> instead, don't forget to apply .toString() for .getKey() and value.getKey().
The unmatched records thus obtained, could be easily subtracted from either of the list (i.e., dbRecords or inputRecords) to retrieve the match results and the operation is swift.
Cheers,
Shubham Chauhan
just use equals:
if (!map1.equals(map2)) {
//they differ
}
you will have to implement equals (and hashCode!) on MyCustomObject, since (if I understand correctly) the objects will have the same content but different identity (the default equals just compares the pointers - i.e., the memory address in which the object live in).
Make sure that your MyCustomObject class implements an appropriate equals and hashcode methods and then it's as easy as getting the result of map1.equals(map2).
This is what the javadocs have to say about it:
Compares the specified object with this map for equality. Returns true if the given object is also a map and the two Maps represent the same mappings. More formally, two maps t1 and t2 represent the same mappings if t1.entrySet().equals(t2.entrySet()). This ensures that the equals method works properly across different implementations of the Map interface.
Let's say I have two hashmaps:
A:1 <-- B:2 C:3 X:1 <-- Y:2 Z:3
I want to go down each row one by one and confirm that the values are identical to each other (which they are). I don't care about the keys.
Is this possible? How would you even do this?
Edit: Trying to solve 'isomorphic strings' problem on LC
Quick Answer
You should use the equals method since this is implemented to perform the comparison you want. toString() itself uses an iterator just like equals but it is a more inefficient approach. Additionally, as @Teepeemm pointed out, toString is affected by order of elements (basically iterator return order) hence is not guaranteed to provide the same output for 2 different maps (especially if we compare two different maps).
Note/Warning: Your question and my answer assume that classes implementing the map interface respect expected toString and equals behavior. The default java classes do so, but a custom map class needs to be examined to verify expected behavior.
See: http://docs.oracle.com/javase/7/docs/api/java/util/Map.html
boolean equals(Object o)
Compares the specified object with this map for equality. Returns true if the given object is also a map and the two maps represent the same mappings. More formally, two maps m1 and m2 represent the same mappings if m1.entrySet().equals(m2.entrySet()). This ensures that the equals method works properly across different implementations of the Map interface.
Implementation in Java Source (java.util.AbstractMap)
Additionally, java itself takes care of iterating through all elements and making the comparison so you don't have to. Have a look at the implementation of AbstractMap which is used by classes such as HashMap:
// Comparison and hashing
/**
* Compares the specified object with this map for equality. Returns
* <tt>true</tt> if the given object is also a map and the two maps
* represent the same mappings. More formally, two maps <tt>m1</tt> and
* <tt>m2</tt> represent the same mappings if
* <tt>m1.entrySet().equals(m2.entrySet())</tt>. This ensures that the
* <tt>equals</tt> method works properly across different implementations
* of the <tt>Map</tt> interface.
*
* <p>This implementation first checks if the specified object is this map;
* if so it returns <tt>true</tt>. Then, it checks if the specified
* object is a map whose size is identical to the size of this map; if
* not, it returns <tt>false</tt>. If so, it iterates over this map's
* <tt>entrySet</tt> collection, and checks that the specified map
* contains each mapping that this map contains. If the specified map
* fails to contain such a mapping, <tt>false</tt> is returned. If the
* iteration completes, <tt>true</tt> is returned.
*
* @param o object to be compared for equality with this map
* @return <tt>true</tt> if the specified object is equal to this map
*/
public boolean equals(Object o) {
if (o == this)
return true;
if (!(o instanceof Map))
return false;
Map<K,V> m = (Map<K,V>) o;
if (m.size() != size())
return false;
try {
Iterator<Entry<K,V>> i = entrySet().iterator();
while (i.hasNext()) {
Entry<K,V> e = i.next();
K key = e.getKey();
V value = e.getValue();
if (value == null) {
if (!(m.get(key)==null && m.containsKey(key)))
return false;
} else {
if (!value.equals(m.get(key)))
return false;
}
}
} catch (ClassCastException unused) {
return false;
} catch (NullPointerException unused) {
return false;
}
return true;
}
Comparing two different types of Maps
toString fails miserably when comparing a TreeMap and HashMap though equals does compare contents correctly.
Code:
public static void main(String args[]) {
HashMap<String, Object> map = new HashMap<String, Object>();
map.put("2", "whatever2");
map.put("1", "whatever1");
TreeMap<String, Object> map2 = new TreeMap<String, Object>();
map2.put("2", "whatever2");
map2.put("1", "whatever1");
System.out.println("Are maps equal (using equals):" + map.equals(map2));
System.out.println("Are maps equal (using toString().equals()):"
+ map.toString().equals(map2.toString()));
System.out.println("Map1:"+map.toString());
System.out.println("Map2:"+map2.toString());
}
Output:
Are maps equal (using equals):true
Are maps equal (using toString().equals()):false
Map1:{2=whatever2, 1=whatever1}
Map2:{1=whatever1, 2=whatever2}
As long as you override equals() on each key and value contained in the map, then m1.equals(m2) should be reliable to check for maps equality.
The same result can be obtained also by comparing toString() of each map as you suggested, but using equals() is a more intuitive approach.
May not be your specific situation, but if you store arrays in the map, may be a little tricky, because they must be compared value by value, or using Arrays.equals(). More details about this see here.
I have two lists containing maps, the four keys are instrument, party, quantity, and entity.
If the values match between query1 and query2 results I want to add the matching maps into a list. So we can tell map1 in query1 matches map3 in query2. This is for reconciliation of data where there will be a plethora of records so essentially we want to go through each map in query1 results and compare to all maps in query2 results until there is a match. Once there is a match, we want to create a unique identifier or some sort of a pair to show that the two maps are equal and thus can be deemed a match. If we go through a full iteration and map1 in query1 does not equal any maps in query 2 we can then mark map1 in query1 a break, which would be adding the key to a break list.
Please let me know if you need more information. I've added the 3 different attempts I have tried, however none seem to make a match.
public void reconcile(LocalDate date) {
List<Map<String, Object>> query1Records;
List<Map<String, Object>> query2Records;
try (Stream<Map<String, Object>> stream = executeQuery1(date)) {
query1Records = stream.collect(Collectors.toList());
}
try (Stream<Map<String, Object>> stream = executeQuery2(date)) {
query2Records = stream.collect(Collectors.toList());
}
// ATTEMPT #1
List<Map<String,Object>> unMatchedRecords = query1Records.parallelStream().filter(searchData ->
query2Records.parallelStream().noneMatch( inputMap ->
searchData.entrySet().stream().noneMatch(value ->
inputMap.entrySet().stream().noneMatch(value1 ->
(value1.getKey().equals(value.getKey()) &&
value1.getValue().equals(value.getValue()))))
)).collect(Collectors.toList());
// ATTEMPT #2
List<Map<String, Object>> matching = query1Records.parallelStream().filter(searchData ->
query2Records.parallelStream().anyMatch(inputMap ->
String.valueOf(searchData.get("quantity")).equals(inputMap.get("quantity"))
&& searchData.get("instrument").equals(inputMap.get("instrument"))
&& searchData.get("entity").equals(inputMap.get("entity"))
&& searchData.get("party").equals(inputMap.get("party"))))
.collect(Collectors.toList());
// ATTEMPT #3
List<String> keysToReconcile = Arrays.asList("quantity", "instrument", "entity", "party");
Function<Map<String,Object>,List<?>> getKey
= m -> keysToReconcile.stream().map(m::get).collect(Collectors.toList());
Map<List<?>,Map<String, Object>> inputKeys = query1Records.stream()
.collect(Collectors.toMap(
getKey,
m -> m,
(a,b) -> { throw new IllegalStateException("duplicate "+a+" and "+b); },
LinkedHashMap::new));
List<Map<String,Object>> matchingRecords = query2Records.stream()
.filter(m -> inputKeys.containsKey(getKey.apply(m)))
.collect(Collectors.toList());
matchingRecords.forEach(m -> inputKeys.remove(getKey.apply(m)));
List<Map<String,Object>> notMatchinRecords = new ArrayList<>(inputKeys.values());
}
}
If both of your lists can contain unique items you could merge them together into a Set<> to remove duplicates. Then create a List<> again for your desired output:
Set<Map<String,String>> set = new HashSet<>();
set.addAll(customerDetails);
set.addAll(accountDetails);
List<Map<String,String>> combined = new ArrayList<>(set);
System.out.println(combined);
Output:
[{Account=1049, name=Tom, id=2}, {Account=1052, name=Mary, id=3}, {Account=1050, name=Andy, id=1}]
There are two lists of some objects, so basically one can remove the elements of one list from the other:
List<Map<String, String>> missing = new ArrayList<>(customerDetails);
missing.removeAll(accountDetails);
System.out.println("accountDetails missing " + missing);
But, as @Sweeper already stated, it's better to create a type for your data like e. g. Person.
Map#equals does not care about the order. As long as your 2 maps contain the same mapping it will return true.
Map#equals uses Set#equals method, applied to the entry set. Set#equals contract:
Returns true if the specified object is also a set, the two sets have the same size, and every member of the specified set is contained in this set (or equivalently, every member of this set is contained in the specified set).
Note: this assumes that your CourtCase objects have proper equals and hashcode methods to be compared.
Map implementations provides an equals method which do the trick. Map.equals