Definitely take Thorium's answer seriously. However, I figured I might as well try to address your actual question too.

You'll hear this all the time on a security board like this, but I'll say it anyway: the answer always depends on your anticipated threat vector. I'll focus on brute-force attacks by people who aren't specifically targeting you (because that sounds like your primary concern), but the situation is much different if someone is specifically targeting you. Let's keep it simple though.

Untargeted Offline brute-force attack

A big reason for high-entropy passwords is to defeat offline brute-force attacks. Obviously offline brute-force attacks are trivially easy if the hacked service uses plain-text passwords (which is a very important reason why you should never reuse passwords across sites). However, what if your password ends up in a dump from a service that uses MD5 for passwords? There are rigs out there that can test hundreds of billions of passwords per second against MD5. The best defense against such an attack is simply password length, and making sure your password isn't on a password list or a simple variation of common password schemes people use.

Even with an offline brute-force against an MD5 password list, an attacker isn't going to just exhaustively search password space. They will start by downloading lists of previously-cracked passwords and trying all of those. Then they'll take a list of the most commonly used password-generation schemes and try those. The "combination of dictionary words" generation phrase is common enough that they may even try that. If so, the question is how long can you hold up? Depends on how many words you have and how many words are in your list. There are 7776 words in the diceware list, so let's use that. That means a 4 word passphrase has approximately 3.66e15 different passwords combinations it can make. At a rate of 200 billion passwords per second (a top-of-the-line hashing rig) it will take 5 hours to search that password space. Here is the search time depending on the number of words in your password:

• 4 words: 5 hours
• 5 words: 4.5 years
• 6 words: 35,000 years
• 7 words: 270 million years
• 8 words: 2 trillion years

Of course MD5 is terrible. If your password was leaked from a system that uses more modern hashing methods, even a 4 word password will be effectively uncrackable. However, it's best to assume the worst and, for important services, assume the person on the other end is using the worst possible security and choose accordingly (i.e. assume plain-text passwords or MD5). There are plenty of systems out there that are still using MD5 for passwords.

The unknowns

There are lots of unknowns though, which makes this hard to answer. We've assumed that an attacker has tried to brute force a diceware-like password and is using the exact same password list that you used. Those are a lot of assumptions, and a hacker might not bother or might not have your word list. What if they don't and instead just try an exhaustive search? Assuming an average word length of 5 characters, a 4 word diceware password is 20 characters long. They are doing an exhaustive search so must check all letters and numbers even though you have only lowecase letters (we'll be nice and ignore special characters). Now there are 7e35 password combinations to try (if they want to search all passwords up to 20 characters long), or 1e17 years of computation with a top-notch hashing rig before exhaustively searching the necessary password space. In other words, there is absolutely no chance of your password being cracked. Obviously, no one would even bother trying that. Which is what it really comes down to. Most people who are just trying to crack as many passwords as they can are going to try the obvious answers first. Past a certain level of complexity there is some safety in the simple fact that you are no longer the low-hanging fruit. Of course if someone is specifically targeting you, then all bets are off (another obligatory xkcd).

Still, I'd probably opt for 6 or more words. Also, don't reuse it anywhere.

Regarding disk encryption/password managers

In a comment you mention that your interest may primarily be in selecting a master password for a password manager, or a password for disk encryption. This is a slightly different use-case. Modern password hashing algorithms are designed to be slow and therefore hard to brute-force. However, encryption algorithms work a bit differently and "slowness" is not as important for encryption as it is for password hashing (to some extent too much slowness is even a bad thing). How "hard" it is to brute force an encryption key varies wildly depending on the exact details of the encryption method (so I can't really guess at what that would look like in practice), but comparing against something like MD5 might not be a bad reference point.

An important difference (h/t Michael Kjörling) is that with local disk encryption or password manager you may have control over the cost factor for the key generation process. In this case you can crank up the "hardness", decrease your password length, and find your own acceptable compromise between security, ease of memory, and "how long I have to wait until this thing opens" (i.e. ease of use).