Update: This answer was originally written in 2014 and is obsolete.
Peter 's answer is correct in that -Xms is allocated at startup and it will grow up to -Xmx (max heap size) but it's a little misleading in how he has worded his answer. (Sorry Peter I know you know this stuff cold).
Setting ms == mx effectively turns off this behavior. While this used to be a good idea in older JVMs, it is no longer the case. Growing and shrinking the heap allows the JVM to adapt to increases in pressure on memory yet reduce pause time by shrinking the heap when memory pressure is reduced. Sometimes this behavior doesn't give you the performance benefits you'd expect and in those cases it's best to set mx == ms.
OOME is thrown when heap is more than 98% of time is spent collecting and the collections cannot recover more than 2% of that. If you are not at max heaps size then the JVM will simply grow so that you're beyond that boundaries. You cannot have an OutOfMemoryError on startup unless your heap hits the max heap size and meets the other conditions that define an OutOfMemoryError.
For the comments that have come in since I posted. I don't know what the JMonitor blog entry is showing but this is from the PSYoung collector.
size_t desired_size = MAX2(MIN2(eden_plus_survivors, gen_size_limit()),
min_gen_size());
I could do more digging about but I'd bet I'd find code that serves the same purpose in the ParNew and PSOldGen and CMS Tenured implementations. In fact it's unlikely that CMS would be able to return memory unless there has been a Concurrent Mode Failure. In the case of a CMF the serial collector will run and that should include a compaction after which top of heap would most likely be clean and therefore eligible to be deallocated.
Update: This answer was originally written in 2014 and is obsolete.
Peter 's answer is correct in that -Xms is allocated at startup and it will grow up to -Xmx (max heap size) but it's a little misleading in how he has worded his answer. (Sorry Peter I know you know this stuff cold).
Setting ms == mx effectively turns off this behavior. While this used to be a good idea in older JVMs, it is no longer the case. Growing and shrinking the heap allows the JVM to adapt to increases in pressure on memory yet reduce pause time by shrinking the heap when memory pressure is reduced. Sometimes this behavior doesn't give you the performance benefits you'd expect and in those cases it's best to set mx == ms.
OOME is thrown when heap is more than 98% of time is spent collecting and the collections cannot recover more than 2% of that. If you are not at max heaps size then the JVM will simply grow so that you're beyond that boundaries. You cannot have an OutOfMemoryError on startup unless your heap hits the max heap size and meets the other conditions that define an OutOfMemoryError.
For the comments that have come in since I posted. I don't know what the JMonitor blog entry is showing but this is from the PSYoung collector.
size_t desired_size = MAX2(MIN2(eden_plus_survivors, gen_size_limit()),
min_gen_size());
I could do more digging about but I'd bet I'd find code that serves the same purpose in the ParNew and PSOldGen and CMS Tenured implementations. In fact it's unlikely that CMS would be able to return memory unless there has been a Concurrent Mode Failure. In the case of a CMF the serial collector will run and that should include a compaction after which top of heap would most likely be clean and therefore eligible to be deallocated.
Main reason to set the -Xms is for if you need a certain heap on start up. (Prevents OutOfMemoryErrors from happening on start up.) As mentioned above, if you need the startup heap to match the max heap is when you would match it. Otherwise you don't really need it. Just asks the application to take up more memory that it may ultimately need. Watching your memory use over time (profiling) while load testing and using your application should give you a good feel for what to need to set them to. But it isn't the worse thing to set them to the same on start up. For a lot of our apps, I actually start out with something like 128, 256, or 512 for min (startup) and one gigabyte for max (this is for non application server applications).
Just found this question on stack overflow which may also be helpful side-effect-for-increasing-maxpermsize-and-max-heap-size. Worth the look.
java - how to choose the jvm heap size? - Stack Overflow
How do I set Java's min and max heap size through environment variables? - Stack Overflow
JVM 1.8 Min and Max Heap Sizes
Best values for min-max RAM, Xms & Xmx arguments
Videos
My question is how to choose the min and max values, and the difference between the two (should max-min be small or big?)
Short answer: don't guess, profile your application.
jconsole can give you useful high-level data such as a feeling for the main resident set vs. the transient data that we normally allocate and garbage collect. What you'll see if you look at the memory tab of that display is usually something like a sawtooth. The lower corner of the sawteeth is about where I would normally set the heap minimum whereas I would use the peak or slope of the sawteeth to experiment with a heap maximum. If your teeth are very steep, you might consider a big heap just to delay the garbage collection. However, if they aren't, you could try a smaller heap maximum to see if that might leave more resources for other processes on your machine (for example).
You should also consider the server VM as that will cause different garbage collection behavior.
All that said, you should also use a more detailed tool such as jvisualvm to profile the memory usage of your process. It's possible that you have a memory leak or greedy allocator that you could tune or eliminate. That would completely change your heap needs.
You should enable GC logging and check to see where your OOM is ocurring.
-verbose:gc
-Xloggc:gc.log
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
You may be experiencing perm space limits, adjust via -XX:MaxPermSize=YYYm
Anyway to answer your question, I start with no minimums and set the maximum relatively high. I then graph the gc log and find out where my stead state is; visually choose an above-average size for the various generations. Read it like a financial chart, you'll want to see good spread in the new generations and a consistent growth and collection in the tenured generation. As mentioned also graph your perm space to make sure you're not constantly increasing.
GC tuning is an art, in no way a science.
You can't do it using environment variables directly. You need to use the set of "non standard" options that are passed to the java command. Run: java -X for details. The options you're looking for are -Xmx and -Xms (this is "initial" heap size, so probably what you're looking for.)
Some products like Ant or Tomcat might come with a batch script that looks for the JAVA_OPTS environment variable, but it's not part of the Java runtime. If you are using one of those products, you may be able to set the variable like:
set JAVA_OPTS="-Xms128m -Xmx256m"
You can also take this approach with your own command line like:
set JAVA_OPTS="-Xms128m -Xmx256m"
java ${JAVA_OPTS} MyClass
If you want any java process, not just ant or Tomcat, to pick up options like -Xmx use the environment variable _JAVA_OPTIONS.
In bash: export _JAVA_OPTIONS="-Xmx1g"