Python 3

From the docs:

Concatenating immutable sequences always results in a new object. This means that building up a sequence by repeated concatenation will have a quadratic runtime cost in the total sequence length. To get a linear runtime cost, you must switch to one of the alternatives below: if concatenating str objects, you can build a list and use str.join() at the end or else write to an io.StringIO instance and retrieve its value when complete

Experiment to compare runtime of several options:

import sys
import timeit
from io import StringIO
from array import array


def test_concat():
    out_str = ''
    for _ in range(loop_count):
        out_str += 'abc'
    return out_str


def test_join_list_loop():
    str_list = []
    for _ in range(loop_count):
        str_list.append('abc')
    return ''.join(str_list)


def test_array():
    char_array = array('b')
    for _ in range(loop_count):
        char_array.frombytes(b'abc')
    return str(char_array.tostring())


def test_string_io():
    file_str = StringIO()
    for _ in range(loop_count):
        file_str.write('abc')
    return file_str.getvalue()


def test_join_list_compr():
    return ''.join(['abc' for _ in range(loop_count)])


def test_join_gen_compr():
    return ''.join('abc' for _ in range(loop_count))


loop_count = 80000

print(sys.version)

res = {}

for k, v in dict(globals()).items():
    if k.startswith('test_'):
        res[k] = timeit.timeit(v, number=10)

for k, v in sorted(res.items(), key=lambda x: x[1]):
    print('{:.5f} {}'.format(v, k))

results

3.7.5 (default, Nov  1 2019, 02:16:32) 
[Clang 11.0.0 (clang-1100.0.33.8)]
0.03738 test_join_list_compr
0.05681 test_join_gen_compr
0.09425 test_string_io
0.09636 test_join_list_loop
0.11976 test_concat
0.19267 test_array

Python 2

Efficient String Concatenation in Python is a rather old article and its main statement that the naive concatenation is far slower than joining is not valid anymore, because this part has been optimized in CPython since then. From the docs:

CPython implementation detail: If s and t are both strings, some Python implementations such as CPython can usually perform an in-place optimization for assignments of the form s = s + t or s += t. When applicable, this optimization makes quadratic run-time much less likely. This optimization is both version and implementation dependent. For performance sensitive code, it is preferable to use the str.join() method which assures consistent linear concatenation performance across versions and implementations.

I've adapted their code a bit and got the following results on my machine:

from cStringIO import StringIO
from UserString import MutableString
from array import array

import sys, timeit

def method1():
    out_str = ''
    for num in xrange(loop_count):
        out_str += `num`
    return out_str

def method2():
    out_str = MutableString()
    for num in xrange(loop_count):
        out_str += `num`
    return out_str

def method3():
    char_array = array('c')
    for num in xrange(loop_count):
        char_array.fromstring(`num`)
    return char_array.tostring()

def method4():
    str_list = []
    for num in xrange(loop_count):
        str_list.append(`num`)
    out_str = ''.join(str_list)
    return out_str

def method5():
    file_str = StringIO()
    for num in xrange(loop_count):
        file_str.write(`num`)
    out_str = file_str.getvalue()
    return out_str

def method6():
    out_str = ''.join([`num` for num in xrange(loop_count)])
    return out_str

def method7():
    out_str = ''.join(`num` for num in xrange(loop_count))
    return out_str


loop_count = 80000

print sys.version

print 'method1=', timeit.timeit(method1, number=10)
print 'method2=', timeit.timeit(method2, number=10)
print 'method3=', timeit.timeit(method3, number=10)
print 'method4=', timeit.timeit(method4, number=10)
print 'method5=', timeit.timeit(method5, number=10)
print 'method6=', timeit.timeit(method6, number=10)
print 'method7=', timeit.timeit(method7, number=10)

Results:

2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]
method1= 0.171155929565
method2= 16.7158739567
method3= 0.420584917068
method4= 0.231794118881
method5= 0.323612928391
method6= 0.120429992676
method7= 0.145267963409

Conclusions:

• join still wins over concat, but marginally
• list comprehensions are faster than loops (when building a list)
• joining generators is slower than joining lists
• other methods are of no use (unless you're doing something special)

It would be a mistake to judge the value of String against StringBuffer on a single point. Yes they are very similar in functionality when taking substrings. No this does not contribute to the argument as to whether to use one or the other.

You should always use the best structure. String is immutable and should be used when immutability is required and in some edge cases. If you need to make changes to a CharSequence then it would be better/safer/faster/more memory efficient to use StringBuilder.

Consider removing a part of a string. Compare how you can do it using a String and using a StringBuilder - see how much easier it is. It isn't just easier, it is more efficient. The String mechanism creates two new objects and then adds them together to make a third. The StringBuilder one just removes the section of the string.

public void test() {
    String a = "0123456789";
    StringBuilder b = new StringBuilder(a);
    System.out.println("a=" + a + " b=" + b);
    a = a.substring(0, 2) + a.substring(5);
    b.delete(2, 5);
    System.out.println("a=" + a + " b=" + b);
}

The issue is that

s.setLength(64);

changes the length to 64, filling the new space with \0 characters: its contents will be:

[' ', '\0', '\0', ... '\0'] (1 space, 63 \0s)

After your second replace, the contents will be:

['H', 'e', 'j', ... 't', 'ø', 'j', '!', '\0' (x10), 'f', 'o', 'o', '\0', ...]

So there is whitespace there, it's just non-printed whitespace.

If you want to fill your string with spaces, rather than '\0', you can either loop through the string with a plain for loop

for (int i = 0; i < 64; ++i) s.setCharAt(i, ' ');

or hard-code a string of spaces with the correct length:

StringBuffer s = new StringBuffer("           "); // Not 64 chars - count carefully!

or you can "simply" initialize with a char[] filled with spaces:

char[] fill = new char[64];
Arrays.fill(fill, ' ');
StringBuffer s = new StringBuffer(new String(fill));

>>> x = "Hello World!"
>>> x[2:]
'llo World!'
>>> x[:2]
'He'
>>> x[:-2]
'Hello Worl'
>>> x[-2:]
'd!'
>>> x[2:-2]
'llo Worl'

Python calls this concept "slicing" and it works on more than just strings. Take a look here for a comprehensive introduction.