You should be running these tests in separate methods as the optimisation of the first loop can interfere with the optimisation of the second i.e. the second one can be slower, just because it is second.
I suggest you run both tests at least 10 (or for 2 seconds) and use System.nanoTime() which has higher resolution.
If you do this and you still run out of memory I suggest you increase the maximum memory size. If you are running 32-bit windows, the default is very low. You can increase it with -Xmx1g on the command line
If you run the following, you can see that GCs have the biggest impact which is not surprising as the problem most produces garbage
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
public class Collectionss {
public static final int TO_ADD = 10000000;
public static void main(String[] args) {
for (int i = 0; i < 5; i++) {
long timeAL = timeAddToArrayList();
long timeLL = timeAddsToLinkedList();
System.out.printf("Time to add %,d Integer to ArrayList %.3f sec, LinkedList %.3f%n",
TO_ADD, timeAL / 1e9, timeLL / 1e9);
}
}
private static long timeAddToArrayList() {
long starttime = System.nanoTime();
List<Integer> l1 = new ArrayList<Integer>();
for (int i = 1; i <= TO_ADD; i++) {
l1.add(i);
}
assert TO_ADD == l1.size();
return System.nanoTime() - starttime;
}
private static long timeAddsToLinkedList() {
long starttime = System.nanoTime();
List<Integer> l2 = new LinkedList<Integer>();
for (int i = 1; i <= TO_ADD; i++) {
l2.add(i);
}
assert TO_ADD == l2.size();
return System.nanoTime() - starttime;
}
}
prints
Time to add 10,000,000 Integer to ArrayList 0.238 sec, LinkedList 1.326
Time to add 10,000,000 Integer to ArrayList 1.193 sec, LinkedList 0.971
Time to add 10,000,000 Integer to ArrayList 0.841 sec, LinkedList 0.048
Time to add 10,000,000 Integer to ArrayList 0.349 sec, LinkedList 1.128
Time to add 10,000,000 Integer to ArrayList 0.064 sec, LinkedList 0.048
However add System.gc() before each test and you get
Time to add 10,000,000 Integer to ArrayList 0.241 sec, LinkedList 2.130
Time to add 10,000,000 Integer to ArrayList 0.070 sec, LinkedList 0.072
Time to add 10,000,000 Integer to ArrayList 0.067 sec, LinkedList 0.053
Time to add 10,000,000 Integer to ArrayList 0.069 sec, LinkedList 0.048
Time to add 10,000,000 Integer to ArrayList 0.065 sec, LinkedList 0.051
Answer from Peter Lawrey on Stack OverflowYou should be running these tests in separate methods as the optimisation of the first loop can interfere with the optimisation of the second i.e. the second one can be slower, just because it is second.
I suggest you run both tests at least 10 (or for 2 seconds) and use System.nanoTime() which has higher resolution.
If you do this and you still run out of memory I suggest you increase the maximum memory size. If you are running 32-bit windows, the default is very low. You can increase it with -Xmx1g on the command line
If you run the following, you can see that GCs have the biggest impact which is not surprising as the problem most produces garbage
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
public class Collectionss {
public static final int TO_ADD = 10000000;
public static void main(String[] args) {
for (int i = 0; i < 5; i++) {
long timeAL = timeAddToArrayList();
long timeLL = timeAddsToLinkedList();
System.out.printf("Time to add %,d Integer to ArrayList %.3f sec, LinkedList %.3f%n",
TO_ADD, timeAL / 1e9, timeLL / 1e9);
}
}
private static long timeAddToArrayList() {
long starttime = System.nanoTime();
List<Integer> l1 = new ArrayList<Integer>();
for (int i = 1; i <= TO_ADD; i++) {
l1.add(i);
}
assert TO_ADD == l1.size();
return System.nanoTime() - starttime;
}
private static long timeAddsToLinkedList() {
long starttime = System.nanoTime();
List<Integer> l2 = new LinkedList<Integer>();
for (int i = 1; i <= TO_ADD; i++) {
l2.add(i);
}
assert TO_ADD == l2.size();
return System.nanoTime() - starttime;
}
}
prints
Time to add 10,000,000 Integer to ArrayList 0.238 sec, LinkedList 1.326
Time to add 10,000,000 Integer to ArrayList 1.193 sec, LinkedList 0.971
Time to add 10,000,000 Integer to ArrayList 0.841 sec, LinkedList 0.048
Time to add 10,000,000 Integer to ArrayList 0.349 sec, LinkedList 1.128
Time to add 10,000,000 Integer to ArrayList 0.064 sec, LinkedList 0.048
However add System.gc() before each test and you get
Time to add 10,000,000 Integer to ArrayList 0.241 sec, LinkedList 2.130
Time to add 10,000,000 Integer to ArrayList 0.070 sec, LinkedList 0.072
Time to add 10,000,000 Integer to ArrayList 0.067 sec, LinkedList 0.053
Time to add 10,000,000 Integer to ArrayList 0.069 sec, LinkedList 0.048
Time to add 10,000,000 Integer to ArrayList 0.065 sec, LinkedList 0.051
If you are dealing with large data sets, then its always better to initialise your ArrayList with the correct size, in your case.
List<Integer> l1 = new ArrayList<Integer>(10000000);, otherwise your ArrayList will be having a default size of 10, and every time once the size is exceeded, the add method will create a new array of increased size copying all the contents of the old array inside your arraylist.
See the source below from ArrayList ensureCapacity method.
if (minCapacity > oldCapacity) {
Object oldData[] = elementData;
int newCapacity = (oldCapacity * 3)/2 + 1;
if (newCapacity < minCapacity)
newCapacity = minCapacity;
// minCapacity is usually close to size, so this is a win:
**elementData = Arrays.copyOf(elementData, newCapacity);**
}
And if its still happens, even after initialising then increase your heap size as others suggested in the post.
arrays - ArrayList and java.lang.OutOfMemoryError - Stack Overflow
java - Behaviour of JVM during out of memory error? List s = new ArrayList<String>(); - Stack Overflow
java - Arraylist issue - Out of memory error - Stack Overflow
java - ArrayList Out of Memory Error - Stack Overflow
You never increment j. You are incrementing ii and val inside the for loop where as I think you actually want to do it in the while loop. Try the example below.
ArrayList<Integer> al = new ArrayList<Integer>();
Random rand = new Random();
int val=1;
int ii =0;
while(val < 400){
for (int j = 0; j<ii; j++) {
int pick = rand.nextInt(100);
al.add(pick);
}
ii++;
val++;
System.out.println("Contents of al: " + al);
al.clear();
System.out.print("Array has been cleared: " + al);
}
Your task would be much easier if you gave your variables meaningful names. e.g.
ArrayList<Integer> al = new ArrayList<Integer>();
Random rand = new Random();
int lineNumber = 1;
int lineLength = 0;
while(lineNumber < 400){
for (int randCount = 0; randCount < lineLength; randCount++) {
int pick = rand.nextInt(100);
al.add(pick);
}
lineLength++;
lineNumber++;
System.out.println("Contents of al: " + al);
al.clear();
System.out.print("Array has been cleared: " + al);
}
for (int j = 0; j<ii;) {
ii++;
val++;
pick = rand.nextInt(100);
al.add(pick);
}
is an infinite loop(j is always 0 and ii is always positive). You should fix it(according to the desired semantics).
Presumably the System.out.println call requires less memory than the s.add("Pradeep") call.
If s is an ArrayList for instance, the s.add call may cause the list to attempt to double up it's capacity. This is possibly a quite memory demanding operation, thus it is not very surprising that the JVM can continue executing even if it can't perform such relatively expensive task.
Here is more simple code that demonstrated better what happens:
try {
int[] a = new int[Integer.MAX_VALUE];
} catch( Exception e ) {
e.printStackTrace();
}
The allocation of the array fails but that doesn't mean Java has no free memory anymore. If you add items to a list, the list grows in jumps. At one time, the list will need more than half of the memory of the VM (about 64MB by default). The next add will try to allocate an array that is too big.
But that means the VM still has about 30MB unused heap left for other tasks.
If you want to get the VM into trouble, use a LinkedList because it grows linearly. When the last allocation fails, there will be only very little memory to handle the exception:
LinkedList<Integer> list = new LinkedList<Integer>();
try {
for(;;) {
list.add(0);
}
} catch( Exception e ) {
e.printStackTrace();
}
That program takes longer to terminate but it still terminates with an error. Maybe Java sets aside part of the heap for error handling or error handling doesn't need to allocate memory (allocation happens before the code is executed).
Hi, I'm doing some homework where I have to make a list of student objects and perform certain operations to them, and I need to have two classes, one using an Array and one using an ArrayList. My issue I'm having is I am inserting an object into the Arraylist at a specific index, but its throwing an error.
Here is my code:
public void insertStudent(String studentName, String newName, int q1, int q2, int q3, int q4, int q5) {
Student newStudent = new Student(newName, q1, q2, q3, q4, q5); //Creates the new student using the given values
for (int i = 0; i < myClass.size(); i++) //Goes through the list of students myClass
if (myClass.get(i).name.equals(studentName)) //Checks if the student has the name requested to replace
{
myClass.add(i, newStudent);
}and this is the error I'm getting:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3210) at java.util.Arrays.copyOf(Arrays.java:3181) at java.util.ArrayList.grow(ArrayList.java:261) at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235) at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227) at java.util.ArrayList.add(ArrayList.java:475) at TestStudent2.insertStudent(TestStudent2.java:42) at studentTester.main(studentTester.java:15) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
I'm honestly confused why I'm getting this, when I comment out
myClass.add(i, newStudent);
I don't get the error, but it looks to me like it should work. Any help would be appreciated.
For what it's worth this is the code I used in the Array portion of the assignment, it works fine.
public void insertStudent(String studentName, String newName, int q1, int q2, int q3, int q4, int q5) {
Student newStudent = new Student(newName, q1, q2, q3, q4, q5); //Creates the new student using the given values
Student[] tempClass = new Student[myClass.length + 1];
boolean t = false;
for (int i = 0; i < myClass.length; i++)
if (!myClass[i].name.equals(studentName) && !t)
tempClass[i] = myClass[i];
else if (myClass[i].name.equals(studentName)) {
t = true;
tempClass[i] = newStudent;
} else if (t)
tempClass[i] = myClass[i - 1];
tempClass[tempClass.length - 1] = myClass[myClass.length - 1];
myClass = tempClass;
}Also here are Pastebins for the entire project if you need some context.
Student.java
TestStudent.java (Using an Array)
TestStudent2.java (Using an ArrayList, the class with the issue)
studentTester.java (My tester class)
You need to increment your count
int count=0;
while (count<farray.size()){
FartsNameArray.add(farray.get(count).name);
count++;
}
You can also use a for loop too
for(int count = 0; count < farray.size(); count++) {
FartsNameArray.add(farray.get(count).name);
}
Or a foreach
foreach(Farts f in farray){
FartsNameArray.add(f.name);
}
i think you forgot to increment your count;
while (count<farray.size()){
FartsNameArray.add(farray.get(count).name);
count++;
}
Look at your loop:
while(j<campos.length){
nombres_reuniones.add(campos[j]);
}
How do you anticipate that ever finishing? You don't modify j. Given that you don't make any change to j after declaring it and assigning it a value of 0 right at the start, it would be much clearer as:
for (int j = 0; j < campos.length; j++) {
nombres_reuniones.add(campos[j]);
}
Or better:
for (String item : campos) {
nombres_reuniones.add(item);
}
Or even simpler:
nombres_reunions.addAll(Arrays.asList(campos));
Additionally, your earlier code can be simpler. Look at this:
String aux2 = null;
aux2 = aux.replace("[", "");
aux2= aux2.replace("]", "");
Why bother assigning aux2 an initial value of null which you then immediately overwrite? Additionally, you can easily chain the method calls. It would be neater as:
String aux2 = aux.replace("[", "").replace("]", "");
And in fact, you can just chain the whole of the string manipulation together from start to finish:
String[] campos = jsonReuniones.getString("nombres")
.replace("[", "")
.replace("]", "")
.split(",");
nombres_reunions.addAll(Arrays.asList(campos));
(I'd stop there, rather than inlining even that expression...)
You're not advancing the loop:
while (j < campos.length) {
nombres_reuniones.add(campos[j]);
j++; // without this, you'll loop forever!
}
What's happening is that you're adding an infinite amount of "campos" to the ArrayList, exhausting all the memory available to your program in the process.
Remember: a loop's condition must be false at some point for the loop to end. If you forget to advance the loop (in this case, by incrementing the j variable) the condition will always be true and the loop will never exit, therefore creating an infinite loop.
If you use the -Xmx flag appropriately when running java (see here for more info), you can increase your Heap size. This will allow you to use more memory if needed, in a controlled way. To my knowledge, there is not a way of "asking" for more heap memory from within a program itself (similarly to running the sbrk() or mmap() syscalls in C)
As the answer I linked to says:
For example, starting a JVM like so will start it with 256MB of memory, and will allow the process to use up to 2048MB of memory:
java -Xmx2048m -Xms256m
Also, you can use "k", "m", or "g" for Kilobytes, Megabytes and Gigabytes respectively. You cannot exceed 1GB (Heap size, that is), however, unless you are using the 64-bit JVM.
If you do the math with your particular use-case, assuming 64-bit longs * 100000000 costs about 800MB of space.
Does your program require you to store longs? If you were to use Integers instead of longs then you could store much more, either way, what requires you to store so many longs in an array?
Other solutions:
You could give your program more heap space when you start it with the argument -Xmx2G or some other length greater than the standard 512M or 1G
You could process a less number of array values, then save the array to hard drive. Then process the rest of the array and store it into the file.(Would require basic knowledge of Java's garbage collection)
To let your program use more memory from Eclipse:
Go to Run -> Run Configurations. You will see this window

Click on Arguments

Enter your arguments to the VM

Since you are using a lot of RAM besides setting the max heap parameter, make sure you use 64-bit Java. The 32-bit is limited to 2 Gigs or something like that.
Also, for large graphs you should consider using a database.
And last but not least, maybe you can rethink your algorithm, sometimes you just don't need all the nodes and edges.
This may sound glib, but your problem is that your application needs a List of 2.5 million rows as HashMaps.
This is an absurd, unreasonable and frankly ridiculous requirement; I can't imagine what use such a data structure would be good for.
Change the application to not require it.
You can try use byte[] instead of String object:
byte[] key = mappingKey.getBytes("UTF-8")
Each String object contains set of UTF-16 chars. It means 2-bytes per symbol in most case. UTF-8 encoding uses one byte for ASCII, two bytes for many europe languages.
Also each String object contains reference to char array. It means that you have two objects in memory heap: String and char array. Each object (even just new Object()) costs ~24 bytes (it depends from version Java VM and options).
So you can easy reduce count of objects by factor two (one byte[] instead of pair String + char[]), and array length of UTF-8 symbols usually less than length of UTF-16 chars.