If you are interested in a regular expression that excludes duplicates, try this: (?([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,9}))(?!.*\).
It can be compared with DistinctBy:
Dim matches = Regex.Matches(Text, "your original expression...", RegexOptions.CultureInvariant Or RegexOptions.IgnoreCase Or RegexOptions.Multiline).DistinctBy(Function(m) m.Value, StringComparer.CurrentCultureIgnoreCase)
The experiments with typical data will show the fastest method.
Answer from Viorel on learn.microsoft.comIf you are interested in a regular expression that excludes duplicates, try this: (?([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,9}))(?!.*\).
It can be compared with DistinctBy:
Dim matches = Regex.Matches(Text, "your original expression...", RegexOptions.CultureInvariant Or RegexOptions.IgnoreCase Or RegexOptions.Multiline).DistinctBy(Function(m) m.Value, StringComparer.CurrentCultureIgnoreCase)
The experiments with typical data will show the fastest method.
Try the following
Dim matches As MatchCollection = Regex.Matches(text, "([a-zA-Z0-9_-.]+)@([a-zA-Z0-9_-.]+)\.([a-zA-Z]{2,9})", RegexOptions.CultureInvariant Or RegexOptions.IgnoreCase Or RegexOptions.Multiline)
Dim uniqueMatches As New HashSet(Of String)(StringComparer.OrdinalIgnoreCase)
For Each match As Match In matches
If match.Success Then
uniqueMatches.Add(match.Value)
End If
Next
' Now uniqueMatches contains only unique email addresses, case-insensitively
If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.
hth
Marcin
That's easy to do with a negative lookahead assertion:
^(?!.*(.).*\1)[ABC]+$
matches exactly as you described.
Test it live on regex101.com.
Explanation:
^ # Start of the string
(?! # Assert that it's impossible to match...
.* # Any number of characters (including zero)
(.) # followed by one character (remember this one in group 1)
.* # that's followed by any number of characters
\1 # and the same character as before
) # End of lookahead
[ABC]+ # Match one or more characters from this list
$ # until the end of the string
back referencing can be used. Here comes an example in PHP, which is compatible to Perl regular expressions:
$string = "A, B, C, AB, AC, B, BC, AABC";
if(preg_match('/([ABC])\1/', $string, $matches)) {
echo $matches[1] . " has been repeated\n";
} else {
echo "OK\n";
}
In the above pattern ([ABC]) is capturing group which can store one out of the characters A, B or C. \1 references this first capturing group, this makes the pattern matching if one those characters repeats.
formula - REGEX Validation Rule to prevent duplicate in set of numbers - Salesforce Stack Exchange
Regex Expression for Duplicate Occurrences
Regex: Finding non-duplicate characters - Stack Overflow
regex - How do I find and remove duplicate lines from a file using Regular Expressions? - Stack Overflow
You can use a regex like this to find the duplicates:
(.).*\1
Then you can use a replace over your main string by an empty string, so your resulting string will have all the characters non duplicated
Working demo
The substitution section contains your resulting string having the non duplicated characters:

Btw, if you just want to find the non duplicated letters you can change the regex to:
([A-Za-z]).*\1
This solution, works for consecutive characters but if you can have duplicated characters then you should use another solution. What I'd do is to split your string by characters and add them into a map, then store for each character the count for their ocurrences. So, there you have another approach without regex.
This is a workaround for not having advanced PCRE features in Javascript:
str = 'AAACDDBBK'
str.replace(new RegExp(str.match(/([A-Z])(?=.*?\1)/ig).join('|'), "g"), "");
//=> CK
str="AAPAACDDBBK";
str.replace(new RegExp(str.match(/([A-Z])(?=.*?\1)/ig).join('|'), "g"), "");
//=> PCK
Regular-expressions.info has a page on Deleting Duplicate Lines From a File
This basically boils down to searching for this oneliner:
^(.*)(\r?\n\1)+$
... And replacing with \1.
Note: Dot must not match Newline
Explanation:
The caret will match only at the start of a line. So the regex engine will only attempt to match the remainder of the regex there. The dot and star combination simply matches an entire line, whatever its contents, if any. The parentheses store the matched line into the first backreference.
Next we will match the line separator. I put the question mark into
\r?\nto make this regex work with both Windows (\r\n) and UNIX (\n) text files. So up to this point we matched a line and the following line break.Now we need to check if this combination is followed by a duplicate of that same line. We do this simply with
\1. This is the first backreference which holds the line we matched. The backreference will match that very same text.If the backreference fails to match, the regex match and the backreference are discarded, and the regex engine tries again at the start of the next line. If the backreference succeeds, the plus symbol in the regular expression will try to match additional copies of the line. Finally, the dollar symbol forces the regex engine to check if the text matched by the backreference is a complete line. We already know the text matched by the backreference is preceded by a line break (matched by \r?\n). Therefore, we now check if it is also followed by a line break or if it is at the end of the file using the dollar sign.
The entire match becomes
line\nline(orline\nline\nlineetc.). Because we are doing a search and replace, the line, its duplicates, and the line breaks in between them, are all deleted from the file. Since we want to keep the original line, but not the duplicates, we use\1as the replacement text to put the original line back in.
See my request for more info, I'm answering in the easy way now.
If the order doesn't matter, just a
sort -u
will do the trick
If the order does matter but you don't mind re-run multiple passes (this is vim syntax), you can use:
%s/\(.*\)\(\_.*\)\(\1\)/\2\1/g
to preserve the last occurrence, or
%s/\(.*\)\(\_.*\)\(\1\)/\1\2/g
to preserve the first occurrence.
If you do mind re-run multiple passes, than it's more difficult, so before we work on that, please say so in the question!
EDIT: in your edit you weren't very clear, but it looks like you want just a single-pass duplicate ADJACENT lines removal! Well, that's much easier!
A simple:
/(.*)\1*/\1/
(/\(.*\)\1*/\1/ in vim) i.e. searching for (.*)\1* and replacing it with just \1 will do the trick
To remove lines ending in duplicate values replace
([^\s]+\s(.*\n))([^\s]+\s\2)+
With
\2
I am assuming with this based on your example input that we can use the first space in a given line to delimit the boundary between the regex which should be checked for duplicates and the rest of the string. If this assumption is wrong you can modify the part [^\s]+\s to be any valid regex that matches the first (non duplicate) part of your string.
The first set of parentheses matches the first line which we will keep.
The second set of parentheses matches the string which we want to check as a duplicate values.
In the third set we check again for a string followed by a space followed by the same duplicate string which we captured in the 2nd set of parentheses.
The + checks for this multiple times.
\2 replaces the entire match with just the duplicated string.
SEARCH: ^(.*)(\r?\n\1)+$
REPLACE BY: \2\r\1\r
OR
SEARCH: ^(.*)(\r?\n\1)+$
REPLACE WITH: \1
OR
FIND: (?<=|^)([^,]*)(,\1)+(?=,|$)
OR
FIND: ^(.*?)$\s+?^(?=.*^\1$)