There are 26 letters in the English language. There are more than 170.000 words.

That's why the entropy is way greater with words, and thus fewer words are needed.

"The computer knows all the words"..? ...so? The computer doesn't know the letters?

One official way to estimate the strength of a user selected password such as "Tr0ub4dor&3" is to look at NIST recommendations. Granted that this is now deprecated, but the relevant publication was NIST Special Publication 800-63 Version 1.0.2, Electronic Authentication Guideline.

Table A.1 (reproduced below in case of link rot):-

The reasoning behind this table is within the document at § A.2.1 Guessing Entropy Estimate. NIST therefor estimates that the entropy is 33 bits if we interpolate for 11 characters and use dictionary and composition rules.

The difficulty of assessing the entropy of short sequences, particularly human produced ones is the take away from this question. The two current answers diverge in strength by a factor of 32. If we compare NIST's estimate to Blafasel's original query on 50 bits, the entropy diverges 131,072 times. NIST says of the above, "Readers are cautioned against interpreting the following rules as anything more than a very rough rule of thumb method". True.

Another take away is that very few sites will allow the stronger and easier to remember technique of choice from a word list, such as "correcthorsebatterystaple". The on-line version of the UK government doesn't, no bank I'm aware of does, and stackexchange.com doesn't.

first off (FYI) computer guesses are based of how common and how long a phrase is, but human memory is based on how complex and long something is.

Basically what this comic is making fun of is that common 'good password criteria' has made passwords more complicated but it has to be short to be remembered. But the 'good password' criteria has made more complicated sequences more common. This means that four common medium length words are less common and longer than normal meaning a better password. Basically what was rare(short complex patterns) is now common making what was common (med-long length phrases) rare.

Here is a thorough explanation of the mathematics in this comic:

The little boxes in the comic represent entropy in a logarithmic scale, i.e. "bits". Each box means one extra bit of entropy. Entropy is a measure of the average cost of hitting the right password in a brute force attack. We assume that the attacker knows the exact password generation method, including probability distributions for random choices in the method. An entropy of n bits means that, on average, the attacker will try 2^n-1 passwords before finding the right one. When the random choices are equiprobable, you have n bits of entropy when there are 2ⁿ possible passwords, which means that the attacker will, on average, try half of them. The definition with the average cost is more generic, in that it captures the cases where random choices taken during the password generation process (the one which usually occurs in the head of the human user) are not uniform. We'll see an example below.

The point of using "bits" is that they add up. If you have two password halves that you generate independently of each other, one with 10 bits of entropy and the other with 12 bits, then the total entropy is 22 bits. If we were to use a non-logarithmic scale, we would have to multiply: 2¹⁰ uniform choices for the first half and 2¹² uniform choices for the other half make up for 2¹⁰·2¹² = 2²² uniform choices. Additions are easier to convey graphically with little boxes, hence our using bits.

That being said, let's see the two methods described in the comic. We'll begin with the second one, which is easier to analyze.

The "correct horse" method

The password generation process for this method is: take a given (public) list of 2048 words (supposedly common words, easy to remember). Choose four random words in this list, uniformly and independently of each other: select one word at random, then select again a word at random (which could be the same as the first word), and so on for a third and then a fourth words. Concatenate all four words together, and voila! you have your password.

Each random word selection is worth 11 bits, because 2¹¹ = 2048, and, crucially, each word is selected uniformly (all 2048 words have the same probability of 1/2048 of being selected) and independently of the other words (you don't choose a word so that it matches or non-matches the previous words, and, in particular, you do not reject a word if it happens to be the same choice as a previous word). Since humans are not good at all at doing random choices in their head, we have to assume that the random word selection is done with a physical device (dice, coin flips, computers...).

The total entropy is then 44 bits, matching the 44 boxes in the comic.

The "troubador" method

For this one, the rules are more complex:

1. Select a random word in a given big list of meaningful words.
2. Decide randomly whether to capitalize the first letter, or not.
3. For the letters which are eligible to "traditional substitutions", apply or not apply the substitution (decide randomly for each letter). These traditional substitutions can be, for instance: "o" -> "0", "a" -> "4", "i" -> "!", "e" -> "3", "l" -> "1" (the rules give a publicly known exhaustive list).
4. Append a punctuation sign and a digit.

The random word is rated to 16 bits by the comic, meaning uniform selection in a list of 65536 words (or non-uniform in a longer list). There are more words than that in English, apparently about 228000, but some of them are very long or very short, others are so uncommon that people would not remember them at all. "16 bits" seem to be a plausible count.

Uppercasing or not uppercasing the first letter is, nominally, 1 bit of entropy (two choices). If the user makes that choice in his head, then this will be a balance between user's feeling of safety ("uppercase is obviously more secure !") and user's laziness ("lowercase is easier to type"). There again, "1 bit" is plausible.

"Traditional substitutions" are more complex because the number of eligible letters depends on the base word; here, three letters, hence 3 bits of entropy. Other words could have other counts, but it seems plausible that, on average, we'll find about 3 eligible letters. This depends on the list of "traditional substitutions", which are assumed to be a given convention.

For the extra punctuation sign and digit, the comic gives 1 bit for the choice of which comes first (the digit or the punctuation sign), then 4 bits for the sign and 3 bits for the digit. The count for digits deserves an explanation: this is because humans, when asked to choose a random digit, are not at all uniform; the digit "1" will have about 5 to 10 times more chances of being selected than "0". Among psychological factors, "0" has a bad connotation (void, dark, death), while "1" is viewed positively (winner, champion, top). In south China, "8" is very popular because the word for "eight" is pronounced the same way as the word for "luck"; and, similarly, "4" is shunned because of homophony with the word for "death". The attacker will first try passwords where the digit is a "1", allowing him to benefit from the non-uniformity of the user choices.

If the choice of digit is not made by a human brain but by an impartial device, then we get 3.32 bits of entropy, not 3 bits. But that's close enough for illustration purposes (I quite understand that Randall Munroe did not want to draw partial boxes).

Four bits for punctuation are a bit understated; there are 32 punctuation signs in ASCII, all relatively easy to type on a common keyboard. This would mean 5 bits, not 4. There again, if the sign is chosen by a human, then some signs will be more common than others, because humans rarely think of '#' or '|' as "punctuation".

The grand total of 28 bits is then about right, although it depends on the precise details of some random selections, and the list of "traditional substitutions" (which impacts the average number of eligible letters). With a computer-generated password, we may hope for about 30 bits. That's still low with regards to the 44 bits of the "correct horse" method.

Applicability

The paragraphs above show that the maths in the comic are correct (at least with the precision that can be expected in these conditions -- that's a webcomic, not a research article). It still requires the following conditions:

• The "password generation method" is known by the attacker. This is the part which @Jeff does not believe. But it makes sense. In big organizations, security officers publish such guidelines for password generation. Even when they don't, people have Google and colleagues, and will tend to use one of about a dozen or so sets of rules. The comic includes provisions for that: "You can add a few more bits to account for the fact that this is only one of a few common formats".

  Bottom-line: even if you keep your method "secret", it won't be that secret because you will more or less consciously follow a "classic" method, and there are not that many of those.

• Random choices are random and uniform. This is hard to achieve with human users. You must convince them to use a device for good randomness (a coin, not a brain), and to accept the result. This is the gist of my original answer (reproduced below). If the users alter the choices, if only by generating another password if the one they got "does not please them", then they depart from random uniformity, and the entropy can only be lowered (maximum entropy is achieved with uniform randomness; you cannot get better, but you can get much worse).

The right answer is of course that of @AviD. The maths in the comic are correct, but the important point is that good passwords must be both hard to guess and easy to remember. The main message of the comic is to show that common "password generation rules" fail at both points: they make hard to remember passwords, which are nonetheless not that hard to guess.

It also illustrates the failure of human minds at evaluating security. "Tr0ub4dor&3" looks more randomish than "correcthorsebatterystaple"; and the same minds will give good points to the latter only because of the wrong reason, i.e. the widespread (but misguided) belief that password length makes strength. It does not. A password is not strong because it is long; it is strong because it includes a lot of randomness (all the entropy bits we have been discussing all along). Extra length just allows for more strength, by giving more room for randomness; in particular, by allowing "gentle" randomness that is easy to remember, like the electric horse thing. On the other hand, a very short password is necessarily weak, because there is only so much entropy you can fit in 5 characters.

Note that "hard to guess" and "easy to remember" do not cover all that is to say about password generation; there is also "easy to use", which usually means "easy to type". Long passwords are a problem on smartphones, but passwords with digits and punctuation signs and mixed casing are arguably even worse.

Original answer:

The comic assumes that the selection of a random "common" word yields an entropy of about 11 bits -- which means that there are about 2000 common words. This is a plausible count. The trick, of course, is to have a really random selection. For instance, the following activities:

• select four words randomly, then remember them in the order which makes most sense;
• if the four words look too hard to remember, scrap them and select four others;
• replace one of the words with the name of a footballer (the attacker will never guess that !);

... all reduce the entropy. It is not easy to get your users to actually use true randomness and accept the result.

The same users will probably complain about the hassle of typing a long password (if the typing involves a smartphone, I must say that I quite understand them). An unhappy user is never a good thing, because he will begin to look for countermeasures which will make his life easier, such as keeping the password in a file and "typing" it with a copy&paste. Users can often be surprisingly creative that way. Therefore long passwords have a tendency to backfire, security-wise.

Schneier writes this:

This is why the oft-cited XKCD scheme for generating passwords -- string together individual words like "correcthorsebatterystaple" -- is no longer good advice. The password crackers are on to this trick.

but the key to understanding what he is really after is a little further in his essay:

There's still one scheme that works. Back in 2008, I described the "Schneier scheme"

so that's it. Ole' Bruce wants to assert that his scheme is the One and Only, the best, the winner, the ultimate scheme. Therefore, he needs to say disparaging things about the "competitors", regardless of whether such assertions are scientifically sound or not.

In this case, it has always been assumed that the password generation method is known to the attacker. That's the whole point of entropy computations; see the analysis. That attackers are "on to this trick" changes nothing at all (when an attacker knows the password generation method, the entropy computation describes exactly the password strength; when the attacker is incompetent and does not know the password generation method, the password strength is only higher, by an amount which is nigh impossible to quantify).

The quip about "passwords in memory" is just more incoherent ramblings. Passwords necessarily go to RAM at some point, whether you type them or copy-paste them from a password safe, or anything similar.

My guess is that Bruce was drunk.

Update Schneier was specifically asked to comment about his passphrase condemnation in a Reddit AMA (via archive.org, original link) that took place August 2, 2016. He continued to advocate for his password creation system as a superior alternative to random word passphrases. Schneier did say his scheme "gives you more entropy per memorizable character than other methods" which is true when compared to characters making up words. But this is also irrelevant when a system relies on memorizing words rather than characters, and you're allowed to combine enough words to generate adequate 'entropy' for your passphrase as a whole.