Such scenario is perfectly supported by my free StreamEx library which enhances standard Stream API. There's an intervalMap intermediate operation which is capable to collapse several adjacent stream elements to the single element. Here's complete example:
// Slot class and sample data are taken from @Andreas answer
List<Slot> slots = Arrays.asList(new Slot(3, 5), new Slot(5, 7),
new Slot(8, 10), new Slot(10, 11), new Slot(11, 13));
List<Slot> result = StreamEx.of(slots)
.intervalMap((s1, s2) -> s1.end == s2.start,
(s1, s2) -> new Slot(s1.start, s2.end))
.toList();
System.out.println(result);
// Output: [3-7, 8-13]
The intervalMap method takes two parameters. The first is a BiPredicate accepting two adjacent elements from the input stream and returns true if they must be merged (here the condition is s1.end == s2.start). The second parameter is a BiFunction which takes the first and the last elements from the merged series and produces the resulting element.
Note that if you have, for example 100 adjacent slots which should be combined into one, this solution does not create 100 intermediate objects (like in @Misha's answer, which is nevertheless very interesting), it tracks first and last slot in the series immediately forgetting about intermediate onces. Of course this solution is parallel friendly. If you have many thousands of input slots, using .parallel() may improve the performance.
Note that current implementation will recreate the Slot even if it's not merged with anything. In this case the BinaryOperator receives the same Slot parameter twice. If you want to optimize this case, you can make additional check like s1 == s2 ? s1 : ...:
List<Slot> result = StreamEx.of(slots)
.intervalMap((s1, s2) -> s1.end == s2.start,
(s1, s2) -> s1 == s2 ? s1 : new Slot(s1.start, s2.end))
.toList();
Answer from Tagir Valeev on Stack OverflowJava 8 Stream mixing two elements - Stack Overflow
How to merge two streams in Java without duplicates according to some property? - Stack Overflow
concatenation - Adding two Java 8 streams, or an extra element to a stream - Stack Overflow
Java Stream: is there a way to iterate taking two elements a time instead of one? - Stack Overflow
Videos
Such scenario is perfectly supported by my free StreamEx library which enhances standard Stream API. There's an intervalMap intermediate operation which is capable to collapse several adjacent stream elements to the single element. Here's complete example:
// Slot class and sample data are taken from @Andreas answer
List<Slot> slots = Arrays.asList(new Slot(3, 5), new Slot(5, 7),
new Slot(8, 10), new Slot(10, 11), new Slot(11, 13));
List<Slot> result = StreamEx.of(slots)
.intervalMap((s1, s2) -> s1.end == s2.start,
(s1, s2) -> new Slot(s1.start, s2.end))
.toList();
System.out.println(result);
// Output: [3-7, 8-13]
The intervalMap method takes two parameters. The first is a BiPredicate accepting two adjacent elements from the input stream and returns true if they must be merged (here the condition is s1.end == s2.start). The second parameter is a BiFunction which takes the first and the last elements from the merged series and produces the resulting element.
Note that if you have, for example 100 adjacent slots which should be combined into one, this solution does not create 100 intermediate objects (like in @Misha's answer, which is nevertheless very interesting), it tracks first and last slot in the series immediately forgetting about intermediate onces. Of course this solution is parallel friendly. If you have many thousands of input slots, using .parallel() may improve the performance.
Note that current implementation will recreate the Slot even if it's not merged with anything. In this case the BinaryOperator receives the same Slot parameter twice. If you want to optimize this case, you can make additional check like s1 == s2 ? s1 : ...:
List<Slot> result = StreamEx.of(slots)
.intervalMap((s1, s2) -> s1.end == s2.start,
(s1, s2) -> s1 == s2 ? s1 : new Slot(s1.start, s2.end))
.toList();
Since these types of questions come up a lot, I thought it might be an interesting exercise to write a collector that would group adjacent elements by a predicate.
Assuming we can add combining logic to the Slot class
boolean canCombine(Slot other) {
return this.end == other.start;
}
Slot combine(Slot other) {
if (!canCombine(other)) {
throw new IllegalArgumentException();
}
return new Slot(this.start, other.end);
}
the groupingAdjacent collector can then be used as follows:
List<Slot> combined = slots.stream()
.collect(groupingAdjacent(
Slot::canCombine, // test to determine if two adjacent elements go together
reducing(Slot::combine), // collector to use for combining the adjacent elements
mapping(Optional::get, toList()) // collector to group up combined elements
));
Alternatively, second parameter can be collectingAndThen(reducing(Slot::combine), Optional::get) and the third argument be toList()
Here's the source for groupingAdjacent. It can handle null elements and is parallel-friendly. With a bit more hassle, a similar thing can be done with a Spliterator.
public static <T, AI, I, AO, R> Collector<T, ?, R> groupingAdjacent(
BiPredicate<? super T, ? super T> keepTogether,
Collector<? super T, AI, ? extends I> inner,
Collector<I, AO, R> outer
) {
AI EMPTY = (AI) new Object();
// Container to accumulate adjacent possibly null elements. Adj can be in one of 3 states:
// - Before first element: curGrp == EMPTY
// - After first element but before first group boundary: firstGrp == EMPTY, curGrp != EMPTY
// - After at least one group boundary: firstGrp != EMPTY, curGrp != EMPTY
class Adj {
T first, last; // first and last elements added to this container
AI firstGrp = EMPTY, curGrp = EMPTY;
AO acc = outer.supplier().get(); // accumlator for completed groups
void add(T t) {
if (curGrp == EMPTY) /* first element */ {
first = t;
curGrp = inner.supplier().get();
} else if (!keepTogether.test(last, t)) /* group boundary */ {
addGroup(curGrp);
curGrp = inner.supplier().get();
}
inner.accumulator().accept(curGrp, last = t);
}
void addGroup(AI group) /* group can be EMPTY, in which case this should do nothing */ {
if (firstGrp == EMPTY) {
firstGrp = group;
} else if (group != EMPTY) {
outer.accumulator().accept(acc, inner.finisher().apply(group));
}
}
Adj merge(Adj other) {
if (other.curGrp == EMPTY) /* other is empty */ {
return this;
} else if (this.curGrp == EMPTY) /* this is empty */ {
return other;
} else if (!keepTogether.test(last, other.first)) /* boundary between this and other*/ {
addGroup(this.curGrp);
addGroup(other.firstGrp);
} else if (other.firstGrp == EMPTY) /* other container is single-group. prepend this.curGrp to other.curGrp*/ {
other.curGrp = inner.combiner().apply(this.curGrp, other.curGrp);
} else /* other Adj contains a boundary. this.curGrp+other.firstGrp form a complete group. */ {
addGroup(inner.combiner().apply(this.curGrp, other.firstGrp));
}
this.acc = outer.combiner().apply(this.acc, other.acc);
this.curGrp = other.curGrp;
this.last = other.last;
return this;
}
R finish() {
AO combined = outer.supplier().get();
if (curGrp != EMPTY) {
addGroup(curGrp);
assert firstGrp != EMPTY;
outer.accumulator().accept(combined, inner.finisher().apply(firstGrp));
}
return outer.finisher().apply(outer.combiner().apply(combined, acc));
}
}
return Collector.of(Adj::new, Adj::add, Adj::merge, Adj::finish);
}
@Jigar Joshi has answered the first part of your question which is "how to merge two IntStream's into one".
Your other question of "how to merge two Stream<T> without overwriting the equals() and hashCode() method?" can be done using the toMap collector, i.e.
assuming you don't want the result as a Stream<T>.
Example:
Stream.concat(stream1, stream2)
.collect(Collectors.toMap(Student::getNo,
Function.identity(),
(l, r) -> l,
LinkedHashMap::new)
).values();
if you want the result as a Stream<T> then one could do:
Stream.concat(stream1, stream2)
.collect(Collectors.collectingAndThen(
Collectors.toMap(Student::getNo,
Function.identity(),
(l, r) -> l,
LinkedHashMap::new),
f -> f.values().stream()));
This is possibly not as efficient as it can be but it's another way to return a Stream<T> where the T items are all distinct but without using overriding equals and hashcode as you've mentioned.
You can use concat()
IntStream.concat(stream1, stream2)
Unfortunately this answer is probably of little or no help whatsoever, but I did a forensics analysis of the Java Lambda Mailing list to see if I could find the cause of this design. This is what I found out.
In the beginning there was an instance method for Stream.concat(Stream)
In the mailing list I can clearly see the method was originally implemented as an instance method, as you can read in this thread by Paul Sandoz, about the concat operation.
In it they discuss the issues that could arise from those cases in which the stream could be infinite and what concatenation would mean in those cases, but I do not think that was the reason for the modification.
You see in this other thread that some early users of the JDK 8 questioned about the behavior of the concat instance method when used with null arguments.
This other thread reveals, though, that the design of the concat method was under discussion.
Refactored to Streams.concat(Stream,Stream)
But without any explanation, suddenly, the methods were changed to static methods, as you can see in this thread about combining streams. This is perhaps the only mail thread that sheds a bit of light about this change, but it was not clear enough for me to determine the reason for the refactoring. But we can see they did a commit in which they suggested to move the concat method out of Stream and into the helper class Streams.
Refactored to Stream.concat(Stream,Stream)
Later, it was moved again from Streams to Stream, but yet again, no explanation for that.
So, bottom line, the reason for the design is not entirely clear for me and I could not find a good explanation. I guess you could still ask the question in the mailing list.
Some Alternatives for Stream Concatenation
This other thread by Michael Hixson discusses/asks about other ways to combine/concat streams
To combine two streams, I should do this:
Stream.concat(s1, s2)not this:
Stream.of(s1, s2).flatMap(x -> x)... right?
To combine more than two streams, I should do this:
Stream.of(s1, s2, s3, ...).flatMap(x -> x)not this:
Stream.of(s1, s2, s3, ...).reduce(Stream.empty(), Stream::concat)... right?
If you add static imports for Stream.concat and Stream.of, the first example could be written as follows:
Stream<Foo> stream = concat(stream1, concat(stream2, of(element)));
Importing static methods with generic names can result in code that becomes difficult to read and maintain (namespace pollution). So, it might be better to create your own static methods with more meaningful names. However, for demonstration I will stick with this name.
public static <T> Stream<T> concat(Stream<? extends T> lhs, Stream<? extends T> rhs) {
return Stream.concat(lhs, rhs);
}
public static <T> Stream<T> concat(Stream<? extends T> lhs, T rhs) {
return Stream.concat(lhs, Stream.of(rhs));
}
With these two static methods (optionally in combination with static imports), the two examples could be written as follows:
Stream<Foo> stream = concat(stream1, concat(stream2, element));
Stream<Foo> stream = concat(
concat(stream1.filter(x -> x!=0), stream2).filter(x -> x!=1),
element)
.filter(x -> x!=2);
The code is now significantly shorter. However, I agree that the readability hasn't improved. So I have another solution.
In a lot of situations, Collectors can be used to extend the functionality of streams. With the two Collectors at the bottom, the two examples could be written as follows:
Stream<Foo> stream = stream1.collect(concat(stream2)).collect(concat(element));
Stream<Foo> stream = stream1
.filter(x -> x!=0)
.collect(concat(stream2))
.filter(x -> x!=1)
.collect(concat(element))
.filter(x -> x!=2);
The only difference between your desired syntax and the syntax above is, that you have to replace concat(...) with collect(concat(...)). The two static methods can be implemented as follows (optionally used in combination with static imports):
private static <T,A,R,S> Collector<T,?,S> combine(Collector<T,A,R> collector, Function<? super R, ? extends S> function) {
return Collector.of(
collector.supplier(),
collector.accumulator(),
collector.combiner(),
collector.finisher().andThen(function));
}
public static <T> Collector<T,?,Stream<T>> concat(Stream<? extends T> other) {
return combine(Collectors.toList(),
list -> Stream.concat(list.stream(), other));
}
public static <T> Collector<T,?,Stream<T>> concat(T element) {
return concat(Stream.of(element));
}
Of course there is a drawback with this solution that should be mentioned. collect is a final operation that consumes all elements of the stream. On top of that, the collector concat creates an intermediate ArrayList each time it is used in the chain. Both operations can have a significant impact on the behaviour of your program. However, if readability is more important than performance, it might still be a very helpful approach.
You can build a custom Collector for this task.
Map<String, String> map =
Stream.of("a", "b", "err1", "c", "d", "err2", "e", "f", "g", "h", "err3", "i", "j")
.collect(MappingErrors.collector());
with:
private static final class MappingErrors {
private Map<String, String> map = new HashMap<>();
private String first, second;
public void accept(String str) {
first = second;
second = str;
if (first != null && first.startsWith("err")) {
map.put(first, second);
}
}
public MappingErrors combine(MappingErrors other) {
throw new UnsupportedOperationException("Parallel Stream not supported");
}
public Map<String, String> finish() {
return map;
}
public static Collector<String, ?, Map<String, String>> collector() {
return Collector.of(MappingErrors::new, MappingErrors::accept, MappingErrors::combine, MappingErrors::finish);
}
}
In this collector, two running elements are kept. Each time a String is accepted, they are updated and if the first starts with "err", the two elements are added to a map.
Another solution is to use the StreamEx library which provides a pairMap method that applies a given function to the every adjacent pair of elements of this stream. In the following code, the operation returns a String array consisting of the first and second element of the pair if the first element starts with "err", null otherwise. null elements are then filtered out and the Stream is collected into a map.
Map<String, String> map =
StreamEx.of("a", "b", "err1", "c", "d", "err2", "e", "f", "g", "h", "err3", "i", "j")
.pairMap((s1, s2) -> s1.startsWith("err") ? new String[] { s1, s2 } : null)
.nonNull()
.toMap(a -> a[0], a -> a[1]);
System.out.println(map);
You can write a custom collector, or use the much simpler approach of streaming over the list's indexes:
Map<String, String> result = IntStream.range(0, data.size() - 1)
.filter(i -> data.get(i).startsWith("err"))
.boxed()
.collect(toMap(data::get, i -> data.get(i+1)));
This assumes that your data is in a random access friendly list or that you can temporarily dump it into one.
If you cannot randomly access the data or load it into a list or array for processing, you can always make a custom pairing collector so you can write
Map<String, String> result = data.stream()
.collect(pairing(
(a, b) -> a.startsWith("err"),
AbstractMap.SimpleImmutableEntry::new,
toMap(Map.Entry::getKey, Map.Entry::getValue)
));
Here's the source for the collector. It's parallel-friendly and might come in handy in other situations:
public static <T, V, A, R> Collector<T, ?, R> pairing(BiPredicate<T, T> filter, BiFunction<T, T, V> map, Collector<? super V, A, R> downstream) {
class Pairing {
T left, right;
A middle = downstream.supplier().get();
boolean empty = true;
void add(T t) {
if (empty) {
left = t;
empty = false;
} else if (filter.test(right, t)) {
downstream.accumulator().accept(middle, map.apply(right, t));
}
right = t;
}
Pairing combine(Pairing other) {
if (!other.empty) {
this.add(other.left);
this.middle = downstream.combiner().apply(this.middle, other.middle);
this.right = other.right;
}
return this;
}
R finish() {
return downstream.finisher().apply(middle);
}
}
return Collector.of(Pairing::new, Pairing::add, Pairing::combine, Pairing::finish);
}
This operation is usually called zip, and it doesn't exist as a function in the Stream API. It's kinda hard to implement it yourself, as you can see from this answer: https://stackoverflow.com/a/23529010/4137489
The function is implemented in the Google Guava library though:
https://guava.dev/releases/23.0/api/docs/com/google/common/collect/Streams.html#zip-java.util.stream.Stream-java.util.stream.Stream-java.util.function.BiFunction-
You can use it like this:
Streams.zip(a, b, (aElem, bElem) -> aElem + ":" + bElem));
You could use the IntStream.range() method:
IntStream.range(0, aList.size()).mapToObj(index -> aList.get(index) + ":" + bList.get(index)).collect(Collectors.toList());
List<Tag> combinedTags = Stream
.concat( // combine streams
tags.stream(),
tagIds.stream().map(Tag::new) // assuming constructor with id parameter
)
.distinct() // get rid of duplicates assuming correctly implemented equals method in Tag
.collect(Collectors.toList());
First of all, if you have enough data working with a Set will be faster, I assume a Tag can't have duplicate ids... And you can do everything in a few steps:
tagIds.removeAll(ids);
// assuming there is a Tag constructor that takes an Integer
List<Tag> newTags = tagIds.stream().map(Tag::new).collect(Collectors.toList())
tags.addAll(newTags);
If you want it to be lazy, you have to escape the Stream API through Stream.iterator() or Stream.spliterator().
Otherwise the way to do it is to call the terminal operation Stream.collect(Collector) with a custom collector, which will consume the whole stream.
@Test
public void test() {
Stream<Integer> input = Stream.of(0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1);
UniqCountSpliterator uniqCountSpliterator = new UniqCountSpliterator(input.spliterator());
long[] output = uniqCountSpliterator.stream()
.toArray();
long[] expected = {3, 2, 2, 4};
assertArrayEquals(expected, output);
}
import java.util.Spliterator;
import java.util.function.LongConsumer;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
public class UniqCountSpliterator implements Spliterator.OfLong {
private Spliterator wrapped;
private long count;
private Object previous;
private Object current;
public UniqCountSpliterator(Spliterator wrapped) {
this.wrapped = wrapped;
}
public LongStream stream() {
return StreamSupport.longStream(this, false);
}
@Override
public OfLong trySplit() {
return null;
}
@Override
public long estimateSize() {
return Long.MAX_VALUE;
}
@Override
public int characteristics() {
return NONNULL | IMMUTABLE;
}
@Override
public boolean tryAdvance(LongConsumer action) {
while (wrapped.tryAdvance(next -> current = next) && (null == previous || current.equals(previous))) {
count++;
previous = current;
}
if (previous == null) {
return false;
}
action.accept(count);
count = 1;
previous = null;
return true;
}
}
You can almost do it with flatMap. It would work for infinite streams, with finite stream I don't see a way to detect end of stream from within it.
Stream<Integer> stream = Stream.of(0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1);
Stream<Integer> flatMap = stream.flatMap(new Function<Integer, Stream<Integer>>() {
Integer prev = null;
int count;
public java.util.stream.Stream<Integer> apply(Integer i) {
if ( i.equals(prev)) {
count++;
return Stream.empty();
} else {
int c = count;
count = 1;
prev = i;
if ( c > 0 ) {
return Stream.of(c);
} else {
return Stream.empty();
}
}
};
});
flatMap.forEach(i -> {
System.out.println(i);
});
Said that, you could probably get a lot better mileage out of rxjava for such kind of things (where you could use Subject to emit values as you wish and be able to detect end of stream).
Of course, if you want to escape Stream boundaries, there are many options, as indicated by Christoffers answer.
You need to implement a Spliterator, rather than going through Stream.Builder. For this, you might even just go through an Iterator, since it's a fairly sequential operation. Using Guava lightly,
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(
Iterators.mergeSorted(
Arrays.asList(stream1.iterator(), stream2.iterator()),
comparator),
Spliterator.ORDERED),
false /* not parallel */ );
Iterables.mergeSorted() from Guava
public static <T> Iterable<T> mergeSorted(Iterable<? extends Iterable<? extends T>> iterables,
Comparator<? super T> comparator)
- https://javabot.evanchooly.com/javadoc/guava/22.0/com/google/common/collect/Iterables.html#mergeSorted-java.lang.Iterable-java.util.Comparator-