Use a CSV parser like super-csv.
Univocity provides a benchmark of CSV parsers. It says that univocity-parsers is fast, which is no surprise. You could give it a try.
Answer from jschnasse on Stack OverflowHello,
I'm working on some code that reads in millions of lines of data from a CSV and performs some rudimentary correlation analysis on that data. The code is working and running well how I have it implemented now BUT I am currently calling split() on every record I read in from the CSV and I'm aware that calling string splits in a loop can cause massive memory usage.
Anyone have any interesting/clever thoughts on how to achieve this without using split() to make it more efficient? I could write my own method to split the string but I'm just curious about any alternatives.
Edit: Just wanted to add that any clever ideas are appreciated but I'm more interested in being pointed in the right direction, rather than having someone solve this for me.
java - Fast alternative for String.split by whitespace - Code Review Stack Exchange
java - Something similar to split()? - Stack Overflow
Java split String performances - Stack Overflow
performance - Quickest way to split a delimited String in Java - Software Engineering Stack Exchange
Use a CSV parser like super-csv.
Univocity provides a benchmark of CSV parsers. It says that univocity-parsers is fast, which is no surprise. You could give it a try.
I would recommend you to take a look at opencsv library or try CSVParser from Apache Commons
Anyway, reinventing the wheel is not the best idea. Using 3rd party library would be less headache than writing it yourself :)
Bug #1
You have an off-by-one error here:
if(index1 < n - 1) { result.add(text.substring(index1)); }
The consequence of this is that "a b c" will be split to ["a", "b"], poor "c" is left out.
Bug #2
I would expect the same output for these calls:
fastSplitWS(""); fastSplitWS(" ");
But that's not the case. The first returns an empty list, the second a list with a single empty string in it.
This behavior is also inconsistent with String.split.
This is caused by this:
if(!y)
One way to fix:
if(!y && i > 0)
Another way is to initialize y to true.
(Notice that with this variable name,
it's hard to understand the logical meaning of this.)
Readability
@TopinFrassi was too kind. I find this code extremely hard to read, mostly due to the poor naming throughout. The unconventional indenting and brace placement don't help.
Performance
It's worth nothing the trade-offs of using .charAt and .toCharArray.
Take a look at those links, of the implementations in OpenJDK 8:
toCharArraydoes anarraycopy, using double the spacecharAtdoes a boundary check on the index parameter
Which one is more efficient? I've no idea...
Improvement ideas
The indentation levels can be reduced here:
if(text != null) { final int n = text.length(); if(n > 0) {
The first one can be reduced using an early return:
if(text == null) return result;
final int n = text.length();
if(n > 0)
{
The second can be reduced by eliminating the if(n > 0) completely,
as the loop logic takes care of the n == 0 case automatically.
Suggested implementation
Applying the above suggestions and bugfixes, the implementation becomes:
private List<String> fastSplitWS(final String text) {
if (text == null) {
throw new IllegalArgumentException("the text to split should not be null");
}
final List<String> result = new ArrayList<String>();
final int len = text.length();
int tokenStart = 0;
boolean prevCharIsSeparator = true; // "preceding char is separator" flag
char[] chars = text.toCharArray();
for (int pos = 0; pos < len; ++pos) {
char c = chars[pos];
if (c == '\t' || c == ' ') {
if (!prevCharIsSeparator) {
result.add(text.substring(tokenStart, pos));
prevCharIsSeparator = true;
}
tokenStart = pos + 1;
} else {
prevCharIsSeparator = false;
}
}
if (tokenStart < len) {
result.add(text.substring(tokenStart));
}
return result;
}
Your indentation is on the limit of being hard to read! :p Usually there's more indentation between two nested things. Ex :
Your code :
if(true)
{
if(true && 1==1)
{
}
}
Usually :
if(true)
{
if(true && 1==1)
{
}
}
It is much easier to read this way! Also, Java conventions point that the bracket's style should be "egyptian" meaning :
if(...) {
}
instead of
if(...)
{
}
You could reduce the nesting (and hence enhance readability) by reversing the logic of your first if. Instead of :
if(text != null) {
}
You should write :
if(text == null) {
return result;
}
This way there's less code in your if, and well, that's easier to read.
Another way to remove the else code block would be to introduce a variable that checks if the current character is a separator.
Same thing for n > 0. Although you could simply drop this condition. If n==0, then your for loop simply won't be executed and "voilΓ ", you return result.
Look at this : boolean y = false; // "preceding char is separator" flag
If you have a comment explaining what your variable does, then your variable isn't well named. Why wouldn't you name your boolean precedingCharIsSeparator? Is it too long? I don't think so, plus it would mean the comment is now useless, you could remove it. Remember that long variable names aren't longer to compute! :p
The overall naming needs to be reviewed. It won't help for performance, but it will sure as hell help for readability, which is the second biggest concern of your code.
There's something you need to understand about profiling. The final result will always be 100%. The fact that your code takes 58% of the time isn't abnormal. It could be 58% of 1 millisecond or 58% of 10 minutes. If the bottleneck of your application is split, your application is probably fine. Otherwise you maybe need to check if you can replace your split by something else.
Here's the final code, with better named variables, less nesting and the usage of a char[] which I think will speed up a little bit your code. (But I have no proof of what I'm saying! :p)
private static List<String> fastSplitWS(final String text)
{
final List<String> result = new ArrayList<String>();
if(text == null) {
return result;
}
final char[] characters = text.toCharArray();
final int length = characters.length;
int newWordIndex = 0;
boolean precedingCharIsSeparator = true;
for(int i = 0; i < length; i++) {
char current = characters[i];
boolean currentCharIsSeparator = current == '\t' || current == ' ';
if(currentCharIsSeparator && !precedingCharIsSeparator) {
result.add(new String(characters, newWordIndex, i - newWordIndex + 1));
} else if(precedingCharIsSeparator) {
newWordIndex = i;
}
precedingCharIsSeparator = currentCharIsSeparator;
}
if(newWordIndex < length - 1) {
result.add(new String(characters, newWordIndex, length - newWordIndex));
}
return result;
}
Finally, the method name isn't very good. I guess WS stands for WhiteSpace, well you should write it fully! splitOnWhitespace would be a better name.
Finally (again lol), you should wonder if you really want to return a List<String>? Usually split returns an array, I think it should be the same here.
I tried using split but that takes away the delimiter. I need the \n to be kept at the end of each line if it exists.
You can still use it and preserve the line break if you use look-ahead or look-behind in your regex. Check out the best regular expressions tutorial that I know of:
Regex Tutorial
Look-Around section of the Regex Tutorial.
For example:
public class RegexSplitPageBrk {
public static void main(String[] args) {
String text = "Hello world\nGoodbye cruel world!\nYeah this works!";
String regex = "(?<=\\n)"; // with look-behind!
String[] tokens = text.split(regex);
for (String token : tokens) {
System.out.print(token);
}
}
}
The look-ahead or look-behind (also called "look-around") is non-destructive to the characters they match.
Alternative to @Hovercraft solution with Lookahead assertion:
String[] result = s.split("(?=\n)");
Further details about Lookahead in http://www.regular-expressions.info/lookaround.html
String.split(String) won't create regexp if your pattern is only one character long. When splitting by single character, it will use specialized code which is pretty efficient. StringTokenizer is not much faster in this particular case.
This was introduced in OpenJDK7/OracleJDK7. Here's a bug report and a commit. I've made a simple benchmark here.
$ java -version
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
$ java Split
split_banthar: 1231
split_tskuzzy: 1464
split_tskuzzy2: 1742
string.split: 1291
StringTokenizer: 1517
If you can use third-party libraries, Guava's Splitter doesn't incur the overhead of regular expressions when you don't ask for it, and is very fast as a general rule. (Disclosure: I contribute to Guava.)
Iterable<String> split = Splitter.on('/').split(string);
(Also, Splitter is as a rule much more predictable than String.split.)
I've written a quick and dirty benchmark test for this. It compares 7 different methods, some of which require specific knowledge of the data being split.
For basic general purpose splitting, Guava Splitter is 3.5x faster than String#split() and I'd recommend using that. Stringtokenizer is slightly faster than that and splitting yourself with indexOf is twice as fast as again.
For the code and more info see https://web.archive.org/web/20210613074234/http://demeranville.com/battle-of-the-tokenizers-delimited-text-parser-performance (original link is dead and corresponding site does not appear to exist anymore)
As @Tom writes, an indexOf type approach is faster than String.split(), since the latter deals with regular expressions and has a lot of extra overhead for them.
However, one algorithm change that might give you a super speedup. Assuming that this Comparator is going to be used to sort your ~100,000 Strings, do not write the Comparator<String>. Because, in the course of your sort, the same String will likely be compared multiple times, so you will split it multiple times, etc...
Split all the Strings once into String[]s, and have a Comparator<String[]> sort the String[]. Then, at the end, you can combine them all together.
Alternatively, you could also use a Map to cache the String -> String[] or vice versa. e.g. (sketchy) Also note, you are trading memory for speed, hope you have lotsa RAM
HashMap<String, String[]> cache = new HashMap();
int compare(String s1, String s2) {
String[] cached1 = cache.get(s1);
if (cached1 == null) {
cached1 = mySuperSplitter(s1):
cache.put(s1, cached1);
}
String[] cached2 = cache.get(s2);
if (cached2 == null) {
cached2 = mySuperSplitter(s2):
cache.put(s2, cached2);
}
return compareAsArrays(cached1, cached2); // real comparison done here
}
Hello Night Owls,
I have seen a lot of frustration over Practice Assessment Lab question 20.6 and Lab 3.26 (Section 3 practice lab of a similar or exact same nature).
Students, including myself, were struggling to solve this using only what zyBooks provides us, and that does *not* include the .split() method. So, seeing this as the only suggestion was a little annoying, and it also shows a reliance on chatGPT, which some students may want to avoid for the sake of it.
So, I am providing my solution and breakdown (with full explanation) here to give back to this community for the help it provides, and to, hopefully, ease some frustrations.
ββββββββββββββββββ
The lab itself asks students to write a program that accepts a full name and formats it into βlastInitial., fullFirstName middleInitial.β
So βAye Bee Ceeβ formats to βC., Aye B.β and βAye Ceeβ formats to βC., Ayeβ.
To solve this, we are going to use .substring(), .indexOf(), and .lastIndexOf() from lesson 3.16.
ββββββββββββββββββ
Your final code will look something like this (aka TLDR;):
import java.util.Scanner;
public class LabProgram {
public static void main(String[] args) {
Scanner scnr = new Scanner(System.in);
String fullName = scnr.nextLine();
int firstSpaceIndex = fullName.indexOf(" ");
int lastSpaceIndex = fullName.lastIndexOf(" ");
String firstName = fullName.substring(0, firstSpaceIndex);
String middleInitial;
String lastInitial;
/* OR String lastInitial = fullName.substring(lastSpaceIndex + 1, lastSpaceIndex + 2); */
if (firstSpaceIndex != lastSpaceIndex) {
middleInitial = fullName.substring(firstSpaceIndex + 1, firstSpaceIndex + 2);
lastInitial = fullName.substring(lastSpaceIndex + 1, lastSpaceIndex + 2);
System.out.println(lastInitial + "., " + firstName + " " + middleInitial + ".");
}
else {
lastInitial = fullName.substring(firstSpaceIndex + 1, firstSpaceIndex + 2);
System.out.println(lastInitial + "., " + firstName);
}
}
}ββββββββββββββββββ
Scanner should already be established as βScanner scnr = new Scanner(System.in);β to take in user input.
ββββββββββββββββββ
String fullName = scnr.nextLine();
Start by establishing getting the user input through scanner.
We declare the data type as String, provide a variable name (fullName in this case) and initialize it with user input through scanner.
~Recall that .nextLine() takes in a full singular line as it is written out by the user. (Versus .next(), which only takes in each element up to the next whitespace β youβll see why weβre not using this).
ββββββββββββββββββ
int firstSpaceIndex = fullName.indexOf(β β);
Next, we need to establish where the whitespace is in our full string to help separate each section of the name. We do this by referencing the index in which the whitespace is to give us starting points.
So we declare the data type as int, provide a variable name (firstSpaceIndex in this case β the index number of the first space in our string) and initialize it with the method .indexOf(). We use variable fullName, since that is where the whitespace is coming from, and .indexOf() inherently locates the FIRST instance of whatever is referenced in the parentheses (in this case, a space, denoted as β β). We will later reference this as a starting point for substrings firstName, middle name, and last name.
ββββββββββββββββββ
int lastSpaceIndex = fullName.lastIndexOf(β β);
Next, we establish the index of the LAST instance of a whitespace in the string fullName. The last name is located after the last whitespace as seen in Aye Bee Cee (Aye_Bee**_**Cee).
Again, we declare the data type int (since index is an integer), provide a succinct variable name, and initialize it with the .lastIndexOf() method used on the fullName variable. .lastIndexOf() does what it says, inherently locating the last instance of whatever is in the parentheses.
So if the user input were to be a full name with multiple middle names, such βSalvador Felipe Jacinto Daliβ, .lastIndexOf(β β) would refer to the space just before Dali. Remember, this lab works under the assumption of 1 or 0 middle names, but this is to clarify how .lastIndexOf() works. It serves as a starting point for the input last name.
ββββββββββββββββββ
String firstName = fullName.substring(0, firstSpaceIndex);String middleInitial;String lastInitial;
Next, we make a few more necessary declarations. Firstly, our constant is the display of the full first name, so we declare it as a String, provide a variable name (firstName), and initialize it using the .substring() method because the firstName is a substring of the full String fullName. The substring firstName starts at index 0, always, because itβs -duh- the first name, and it goes up to the first whitespace in fullName (aka firstSpaceIndex).
The syntax for the substring method is βvariableName.substring(x, y);β with x being the index of the first element to be included in the result and y being the last element minus 1, meaning y is not included in the result; rather, the element before y is. **The βminus yβ is inherent in the method.
The other declared variables are both Strings (even though the result is a character, we need to reference the individual character in the full string of the middle and/or last name). You may name them as you see fit, but in order to apply the substring method, it must be applied to a String data type (it cannot be applied to a char data type).
Now that we have our variables declared, we are going to use an if branch to configure and output the properly formatted result being asked of us.
ββββββββββββββββββ
Our if-branch will determine: if the index for firstSpaceIndex is NOT the same as lastSpaceIndex (meaning there are minimally 2 white spaces, indicating the presence of a middle name), then we determine and output the first letter of both the middle name and last name as βC., Aye B.β.
if (firstSpaceIndex != lastSpaceIndex) {middleInitial = fullName.substring(firstSpaceIndex + 1, firstSpaceIndex + 2);
To find the first letter of the middle name, we reference the substring of the fullName string, starting at the index after the first whitespace (firstSpaceIndex + 1) and ending at firstSpaceIndex + 2, which is simply the ending point and not included in the result, making the only position to print the letter at index (firstSpaceIndex + 1).
So, with Aye Bee Cee:
Aye is from index 0 - 2. The first whitespace is at index 3. Bee is from index 4 - 6. The next/last whitespace is at index 7. Cee is from index 8 - 10.
ββββββββββββββββββ
Within the same if-section, we do the same with lastInitial as we did with middleInitial (using the reference point of lastSpaceIndex), then print the formatting.
lastInitial = fullName.substring(lastSpaceIndex + 1, lastSpaceIndex + 2);System.out.println(lastInitial + β., β + firstName + β β + middleInitial + β.β);}
ββββββββββββββββββ
Then, we finish off with the else branch, which will cover the formatting for if no middle name exists. It is implied that the else branch executes if firstSpaceIndex == lastSpaceIndex.
So, in the name βAye Ceeβ, there is one whitespace that serves as both the first (fullName.indexOf(β β)) and last instance (fullName.lastIndexOf(β β)) of the whitespace β β referenced in the parentheses.
else {lastInitial = fullName.substring(firstSpaceIndex + 1, firstSpaceIndex + 2);System.out.println(lastInitial + β., β + firstName);}
You can also use *lastInitial = fullName.substring(*lastSpaceIndex + 1, lastSpaceIndex + 2) in the else-branch, since it gives the same result.
You can also simplify your code a bit and just establish this in the declaration for lastInitial before your if-branch (keep in mind, this will only work using lastSpaceIndex, not firstSpaceIndex).
ββββββββββββββββββ
I hope this breaks it down in a way that makes sense for you guys.
If you find any issues with running the code, or any typos or syntax errors, let me know, and I will correct it.
Be sure to try this on your own to work out the concept of it and not just copy and paste.
If you choose to use the much simpler .split() method, more power to you.
Thanks for your time, everyone.
Use the appropriately named method String#split().
String string = "004-034556";
String[] parts = string.split("-");
String part1 = parts[0]; // 004
String part2 = parts[1]; // 034556
Note that split's argument is assumed to be a regular expression, so remember to escape special characters if necessary.
there are 12 characters with special meanings: the backslash
\, the caret^, the dollar sign$, the period or dot., the vertical bar or pipe symbol|, the question mark?, the asterisk or star*, the plus sign+, the opening parenthesis(, the closing parenthesis), and the opening square bracket[, the opening curly brace{, These special characters are often called "metacharacters".
For instance, to split on a period/dot . (which means "any character" in regex), use either backslash \ to escape the individual special character like so split("\\."), or use character class [] to represent literal character(s) like so split("[.]"), or use Pattern#quote() to escape the entire string like so split(Pattern.quote(".")).
String[] parts = string.split(Pattern.quote(".")); // Split on the exact string.
To test beforehand if the string contains certain character(s), just use String#contains().
if (string.contains("-")) {
// Split it.
} else {
throw new IllegalArgumentException("String " + string + " does not contain -");
}
Note, this does not take a regular expression. For that, use String#matches() instead.
If you'd like to retain the split character in the resulting parts, then make use of positive lookaround. In case you want to have the split character to end up in left hand side, use positive lookbehind by prefixing ?<= group on the pattern.
String string = "004-034556";
String[] parts = string.split("(?<=-)");
String part1 = parts[0]; // 004-
String part2 = parts[1]; // 034556
In case you want to have the split character to end up in right hand side, use positive lookahead by prefixing ?= group on the pattern.
String string = "004-034556";
String[] parts = string.split("(?=-)");
String part1 = parts[0]; // 004
String part2 = parts[1]; // -034556
If you'd like to limit the number of resulting parts, then you can supply the desired number as 2nd argument of split() method.
String string = "004-034556-42";
String[] parts = string.split("-", 2);
String part1 = parts[0]; // 004
String part2 = parts[1]; // 034556-42
An alternative to processing the string directly would be to use a regular expression with capturing groups. This has the advantage that it makes it straightforward to imply more sophisticated constraints on the input. For example, the following splits the string into two parts, and ensures that both consist only of digits:
import java.util.regex.Pattern;
import java.util.regex.Matcher;
class SplitExample
{
private static Pattern twopart = Pattern.compile("(\\d+)-(\\d+)");
public static void checkString(String s)
{
Matcher m = twopart.matcher(s);
if (m.matches()) {
System.out.println(s + " matches; first part is " + m.group(1) +
", second part is " + m.group(2) + ".");
} else {
System.out.println(s + " does not match.");
}
}
public static void main(String[] args) {
checkString("123-4567");
checkString("foo-bar");
checkString("123-");
checkString("-4567");
checkString("123-4567-890");
}
}
As the pattern is fixed in this instance, it can be compiled in advance and stored as a static member (initialised at class load time in the example). The regular expression is:
(\d+)-(\d+)
The parentheses denote the capturing groups; the string that matched that part of the regexp can be accessed by the Match.group() method, as shown. The \d matches and single decimal digit, and the + means "match one or more of the previous expression). The - has no special meaning, so just matches that character in the input. Note that you need to double-escape the backslashes when writing this as a Java string. Some other examples:
([A-Z]+)-([A-Z]+) // Each part consists of only capital letters
([^-]+)-([^-]+) // Each part consists of characters other than -
([A-Z]{2})-(\d+) // The first part is exactly two capital letters,
// the second consists of digits
Since this seems to be a task designed as coding practice, I'll only guide. No code for you, sir, though the logic and the code aren't that far separated.
You will need to loop through each character of the string, and determine whether or not the character is the delimiter (comma or semicolon, for instance). If not, add it to the last element of the array you plan to return. If it is the delimiter, create a new empty string as the array's last element to start feeding your characters into.
I'm going to assume that this is homework, so I will only give snippets as hints:
Finding indices of all occurrences of a given substring
Here's an example of using indexOf with the fromIndex parameter to find all occurrences of a substring within a larger string:
String text = "012ab567ab0123ab";
// finding all occurrences forward: Method #1
for (int i = text.indexOf("ab"); i != -1; i = text.indexOf("ab", i+1)) {
System.out.println(i);
} // prints "3", "8", "14"
// finding all occurrences forward: Method #2
for (int i = -1; (i = text.indexOf("ab", i+1)) != -1; ) {
System.out.println(i);
} // prints "3", "8", "14"
String API links
int indexOf(String, int fromIndex)- Returns the index within this string of the first occurrence of the specified substring, starting at the specified index. If no such occurrence exists, -1 is returned.
Related questions
- Searching for one string in another string
Extracting substrings at given indices out of a string
This snippet extracts substring at given indices out of a string and puts them into a List<String>:
String text = "0123456789abcdefghij";
List<String> parts = new ArrayList<String>();
parts.add(text.substring(0, 5));
parts.add(text.substring(3, 7));
parts.add(text.substring(9, 13));
parts.add(text.substring(18, 20));
System.out.println(parts); // prints "[01234, 3456, 9abc, ij]"
String[] partsArray = parts.toArray(new String[0]);
Some key ideas:
- Effective Java 2nd Edition, Item 25: Prefer lists to arrays
- Works especially nicely if you don't know how many parts there'll be in advance
String API links
String substring(int beginIndex, int endIndex)- Returns a new string that is a substring of this string. The substring begins at the specified
beginIndexand extends to the character at indexendIndex - 1.
- Returns a new string that is a substring of this string. The substring begins at the specified
Related questions
- Fill array with List data