I know this is a really old post but just in case someone is looking for how you could do this in a Java 8 way:

private String join(List<String> namesList) {
    return String.join(",", namesList
            .stream()
            .map(name -> ("'" + name + "'"))
            .collect(Collectors.toList()));
}

import org.apache.commons.lang.StringUtils;
...
...
String arr[] = new String[4];
arr[0] = "my";
arr[1] = "name";
arr[2] = "is";
arr[3] = "baybora";

String join = "'" + StringUtils.join(arr,"','") + "'";

Result:

'my','name','is','baybora'

You could count the total length of the string first, and pass it to the StringBuilder constructor. And you do not need to convert the Set first.

Set<String> abc = new HashSet<String>();
abc.add("A");
abc.add("B");
abc.add("C");

String separator = ", ";
int total = abc.size() * separator.length();
for (String s : abc) {
    total += s.length();
}

StringBuilder sb = new StringBuilder(total);
for (String s : abc) {
    sb.append(separator).append(s);
}

String result = sb.substring(separator.length()); // remove leading separator

String joined = s.stream()
    .map(plain -> '"' + StringEscapeUtils.escapeJava(plain) + '"')
    .collect(Collectors.joining(", "));

The idea is to first convert each of the strings to a properly quoted representation and then join them.

You have to decide for yourself which escaping function to use. And just in case you are generating a CSV file, you should really use some CsvWriter class.

With Java 8:

String csv = String.join(",", ids);

With Java 7-, there is a dirty way (note: it works only if you don't insert strings which contain ", " in your list) - obviously, List#toString will perform a loop to create idList but it does not appear in your code:

List<String> ids = new ArrayList<String>();
ids.add("1");
ids.add("2");
ids.add("3");
ids.add("4");
String idList = ids.toString();
String csv = idList.substring(1, idList.length() - 1).replace(", ", ",");