If you are interested in a regular expression that excludes duplicates, try this: (?([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,9}))(?!.*\).

It can be compared with DistinctBy:

Dim matches = Regex.Matches(Text, "your original expression...", RegexOptions.CultureInvariant Or RegexOptions.IgnoreCase Or RegexOptions.Multiline).DistinctBy(Function(m) m.Value, StringComparer.CurrentCultureIgnoreCase)

The experiments with typical data will show the fastest method.

Answer from Viorel on learn.microsoft.com
Discussions

formula - REGEX Validation Rule to prevent duplicate in set of numbers - Salesforce Stack Exchange
I'm working on a validation that would require a user to enter only numbers that have to be exactly 9 numbers and be separated by lines. The below is working for that, but is it possible with strai... More on salesforce.stackexchange.com
🌐 salesforce.stackexchange.com
Regex Expression for Duplicate Occurrences
What is a Regex expression that will check for any duplicates in a string, and stop when it finds the first occurrence of any upper case letter combination, any lower case combination, any special character combination, or any number combination. And the occurrence can be found in any consecutive or non... More on forum.bubble.io
🌐 forum.bubble.io
12
0
July 7, 2023
Regex: Finding non-duplicate characters - Stack Overflow
@Fede: In my opinion the OP want only to avoid consecutive characters, and to be clear, if it isn't the case (so, non duplicate characters whatever the position in the string), it is no more a job for regex. More on stackoverflow.com
🌐 stackoverflow.com
April 7, 2015
regex - How do I find and remove duplicate lines from a file using Regular Expressions? - Stack Overflow
Remove all occurrences of duplicate lines? Or remove all-but-one? If the latter, which one would you like to preserve, the first or the last? Or the order doesn't matter? ... @ebattulga: I agree with others. You cannot ask a regular expression question that is language agnostic. The underlying regex engine matters greatly. Jim G. – Jim G. 2009-10-16 20:03:52 +00:00 Commented Oct 16, 2009 at 20:03 ... I use regex search and replace in Notepad2 ... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Regular-Expressions.info
regular-expressions.info › duplicatelines.html
Regexp Example: Deleting Duplicate Lines or Items with Regular Expressions
If you have a file in which all ... duplicate lines. Simply open the file in your favorite text editor, and do a search-and-replace searching for ^(.*)(\R\1)+$ and replacing with \1 or $1. For this to work, the anchors need to match before and after line breaks (and not just at the start and the end of the file or string), and the dot must not match newlines. Here is how this works. The caret will match only at the start of a line. So the regex engine will ...
🌐
Medium
rvunabandi.medium.com › my-favorite-regular-expression-9b1bbf21b736
Regex Magic: Say Goodbye to Duplicate Lines | by Robert Vunabandi | Medium
July 2, 2025 - Here’s how I break dow the problem ... lines and get rid of the rest. ... Strategy A: Match one of the duplicate lines (typically the first or last), and ignore all the others....
🌐
Regex Tester
regextester.com › 97748
Remove Duplicate - Regex Tester/Debugger
Regular Expression to This will remove duplicates and only one the duplicates and will at least leave on instance
🌐
GeeksforGeeks
geeksforgeeks.org › dsa › remove-duplicate-words-from-sentence-using-regular-expression
Remove duplicate words from Sentence using Regular Expression - GeeksforGeeks
July 12, 2025 - "+" This quantifier ensures that the non-capturing group (?:\\W+\\1\\b) matches one or more times, effectively matching one or more repeated words. 3. Match the sentence with the Regex. In Java, this can be done using Pattern.matcher(). 4. return the modified sentence. Below is the implementation of the above approach: ... // C++ program to remove duplicate ...
Find elsewhere
🌐
Bubble
forum.bubble.io › need help
Regex Expression for Duplicate Occurrences - Need help - Bubble Forum
July 7, 2023 - What is a Regex expression that will check for any duplicates in a string, and stop when it finds the first occurrence of any upper case letter combination, any lower case combination, any special character combination, or any number combination. And the occurrence can be found in any consecutive or non...
🌐
Regex Pal
regexpal.com › 97748
Remove Duplicate - Regex Pal
RegEx for This will remove duplicates and only one the duplicates and will at least leave on instance
🌐
Alteryx Community
community.alteryx.com › t5 › Alteryx-Designer-Desktop-Discussions › Regex-Replace-Remove-Duplicates › td-p › 574857
Solved: Regex Replace - Remove Duplicates - Alteryx Community
June 13, 2024 - Difficult to say what the issue is with your code without access to your data to troubleshoot - the RegEx looks fine, and worked as expected including the specific example you gave in your question. Personally I find it simpler to split out the IDs to rows using Text to Columns, then remove duplicates with Unique, then concatenate back together - I've attached a sample workflow to illustrate what I mean.
Top answer
1 of 3
193

Regular-expressions.info has a page on Deleting Duplicate Lines From a File

This basically boils down to searching for this oneliner:

^(.*)(\r?\n\1)+$

... And replacing with \1.
Note: Dot must not match Newline

Explanation:

The caret will match only at the start of a line. So the regex engine will only attempt to match the remainder of the regex there. The dot and star combination simply matches an entire line, whatever its contents, if any. The parentheses store the matched line into the first backreference.

Next we will match the line separator. I put the question mark into \r?\n to make this regex work with both Windows (\r\n) and UNIX (\n) text files. So up to this point we matched a line and the following line break.

Now we need to check if this combination is followed by a duplicate of that same line. We do this simply with \1. This is the first backreference which holds the line we matched. The backreference will match that very same text.

If the backreference fails to match, the regex match and the backreference are discarded, and the regex engine tries again at the start of the next line. If the backreference succeeds, the plus symbol in the regular expression will try to match additional copies of the line. Finally, the dollar symbol forces the regex engine to check if the text matched by the backreference is a complete line. We already know the text matched by the backreference is preceded by a line break (matched by \r?\n). Therefore, we now check if it is also followed by a line break or if it is at the end of the file using the dollar sign.

The entire match becomes line\nline (or line\nline\nline etc.). Because we are doing a search and replace, the line, its duplicates, and the line breaks in between them, are all deleted from the file. Since we want to keep the original line, but not the duplicates, we use \1 as the replacement text to put the original line back in.

2 of 3
6

See my request for more info, I'm answering in the easy way now.

  1. If the order doesn't matter, just a

    sort -u

    will do the trick

  2. If the order does matter but you don't mind re-run multiple passes (this is vim syntax), you can use:

    %s/\(.*\)\(\_.*\)\(\1\)/\2\1/g

    to preserve the last occurrence, or

    %s/\(.*\)\(\_.*\)\(\1\)/\1\2/g

    to preserve the first occurrence.

If you do mind re-run multiple passes, than it's more difficult, so before we work on that, please say so in the question!

EDIT: in your edit you weren't very clear, but it looks like you want just a single-pass duplicate ADJACENT lines removal! Well, that's much easier!

A simple:

/(.*)\1*/\1/

(/\(.*\)\1*/\1/ in vim) i.e. searching for (.*)\1* and replacing it with just \1 will do the trick

🌐
regex101
regex101.com › library › QkBEpa
regex101: Remove Duplicates
Example input: someTitle SomeHeader anotherHeader HelloWorld Output: anotherHeader HelloWorld string pattern = $@"{ startP }((?'nested'{ openP })|{ closeP }(?'-nested')|\w\W]?){ closeP }"; *'StartP' (Must include open tag), example: <div id="target" *'openP' example: <div *'closeP' example: </div References: [In Depth with RegEx Matching Nested Constructions In Depth with .NET RegEx Balanced Grouping Regular expression matches closed HTML tags (nesting is supported) Submitted by w4po ... it detects among us references among us aming us mongus amogus sus suspect was not the impostor no imposters remain 3 impostors remainSubmitted by h
🌐
Alteryx Community
community.alteryx.com › t5 › Alteryx-Designer-Desktop-Discussions › regular-expression-to-remove-duplicates › td-p › 719203
Solved: regular expression to remove duplicates - Alteryx Community
June 13, 2024 - If so, you can use a unique with both total and value checked. This will remove the duplicates and your unique values will come out of the U output. From there, you can use regex to trim the last number.
🌐
GeeksforGeeks
geeksforgeeks.org › java › remove-duplicates-from-a-string-in-java-using-regex
How to Remove Duplicates from a String in Java Using Regex ? - GeeksforGeeks
July 23, 2025 - In the above java program, it demonstrates that how to remove the duplicate characters of the given string. It can be implements using Patter and Matchers. First define the regex expression to remove the duplicates((.)(?=.*\\1))