Thanks to @Andy Brown. In addition to what Andy Brown has answered, I'm posting the complete snippet

Complete Solution:

import java.text.DateFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class SampleDate {
    public static void main(String[] args) throws ParseException {
        DateFormat parseFormat = new SimpleDateFormat(
                 "yyyy-MM-dd'T'HH:mm:ss.SSSZ");
        Date date = parseFormat.parse("2012-03-16T00:00:00.000-0500");
        String strDate = parseFormat.format(date);
        System.out.println(strDate);

        // if you get date of type 'java.sql.Date' directly from database cursor like
         //rs.getDate("created_date"), just pass it directly to format()

        SimpleDateFormat dateFormat = new SimpleDateFormat(
                "dd-MMM-yyyy HH:mm:ss");
        String stringDate = dateFormat.format(date);
        System.out.println(stringDate);

    }
}

/*
Output:

2012-03-16T01:00:00.000-0400
16-Mar-2012 01:00:00

*/

you can also convert java.util.Date to java.sql.Date like this,

String dateString = "03-11-2012";
    SimpleDateFormat dateFormat = new SimpleDateFormat("MM-dd-yyyy");
    java.util.Date date = dateFormat.parse(dateString);
    java.sql.Date sqlDate = new Date(date.getTime());
// set the input param type as OracleTypes.DATE and pass the input param date as sqlDate

Date and time formats are well described below

SimpleDateFormat (Java Platform SE 7) - Date and Time Patterns

There could be n Number of formats you can possibly make. ex - dd/MM/yyyy or YYYY-'W'ww-u or you can mix and match the letters to achieve your required pattern. Pattern letters are as follow.

• G - Era designator (AD)
• y - Year (1996; 96)
• Y - Week Year (2009; 09)
• M - Month in year (July; Jul; 07)
• w - Week in year (27)
• W - Week in month (2)
• D - Day in year (189)
• d - Day in month (10)
• F - Day of week in month (2)
• E - Day name in week (Tuesday; Tue)
• u - Day number of week (1 = Monday, ..., 7 = Sunday)
• a - AM/PM marker
• H - Hour in day (0-23)
• k - Hour in day (1-24)
• K - Hour in am/pm (0-11)
• h - Hour in am/pm (1-12)
• m - Minute in hour (30)
• s - Second in minute (55)
• S - Millisecond (978)
• z - General time zone (Pacific Standard Time; PST; GMT-08:00)
• Z - RFC 822 time zone (-0800)
• X - ISO 8601 time zone (-08; -0800; -08:00)

To parse:

2000-01-23T04:56:07.000+0000

Use: new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");