I know this question is quite old, but I was getting confused by the amount of solutions provided - each one of them claiming to be faster and better. I put together a project on github to compare the base64 encoders and decoders: https://github.com/gaspardpetit/base64-benchmark/

At this point, I have not limited myself to C algorithms - if one implementation performs well in C++, it can easily be backported to C. Also tests were conducted using Visual Studio 2015. If somebody wants to update this answer with results from clang/gcc, be my guest.

FASTEST ENCODERS: The two fastest encoder implementations I found were Jouni Malinen's at http://web.mit.edu/freebsd/head/contrib/wpa/src/utils/base64.c and the Apache at https://github.com/apple-oss-distributions/apache1/blob/apache1-697/apache1/src/ap/ap_base64.c.

Here is the time (in microseconds) to encode 32K of data using the different algorithms I have tested up to now:

jounimalinen                25.1544
apache                      25.5309
NibbleAndAHalf              38.4165
internetsoftwareconsortium  48.2879
polfosol                    48.7955
wikibooks_org_c             51.9659
gnome                       74.8188
elegantdice                 118.899
libb64                      120.601
manuelmartinez              120.801
arduino                     126.262
daedalusalpha               126.473
CppCodec                    151.866
wikibooks_org_cpp           343.2
adp_gmbh                    381.523
LihO                        406.693
libcurl                     3246.39
user152949                  4828.21

(René Nyffenegger's solution, credited in another answer to this question, is listed here as adp_gmbh).

Here is the one from Jouni Malinen that I slightly modified to return a std::string:

/*
* Base64 encoding/decoding (RFC1341)
* Copyright (c) 2005-2011, Jouni Malinen <j@w1.fi>
*
* This software may be distributed under the terms of the BSD license.
* See README for more details.
*/

// 2016-12-12 - Gaspard Petit : Slightly modified to return a std::string 
// instead of a buffer allocated with malloc.

#include <string>

static const unsigned char base64_table[65] =
    "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

/**
* base64_encode - Base64 encode
* @src: Data to be encoded
* @len: Length of the data to be encoded
* @out_len: Pointer to output length variable, or %NULL if not used
* Returns: Allocated buffer of out_len bytes of encoded data,
* or empty string on failure
*/
std::string base64_encode(const unsigned char *src, size_t len)
{
    unsigned char *out, *pos;
    const unsigned char *end, *in;

    size_t olen;

    olen = 4*((len + 2) / 3); /* 3-byte blocks to 4-byte */

    if (olen < len)
        return std::string(); /* integer overflow */

    std::string outStr;
    outStr.resize(olen);
    out = (unsigned char*)&outStr[0];

    end = src + len;
    in = src;
    pos = out;
    while (end - in >= 3) {
        *pos++ = base64_table[in[0] >> 2];
        *pos++ = base64_table[((in[0] & 0x03) << 4) | (in[1] >> 4)];
        *pos++ = base64_table[((in[1] & 0x0f) << 2) | (in[2] >> 6)];
        *pos++ = base64_table[in[2] & 0x3f];
        in += 3;
    }

    if (end - in) {
        *pos++ = base64_table[in[0] >> 2];
        if (end - in == 1) {
            *pos++ = base64_table[(in[0] & 0x03) << 4];
            *pos++ = '=';
        }
        else {
            *pos++ = base64_table[((in[0] & 0x03) << 4) |
                (in[1] >> 4)];
            *pos++ = base64_table[(in[1] & 0x0f) << 2];
        }
        *pos++ = '=';
    }

    return outStr;
}

FASTEST DECODERS: Here are the decoding results and I must admit that I am a bit surprised:

polfosol                    45.2335
wikibooks_org_c             74.7347
apache                      77.1438
libb64                      100.332
gnome                       114.511
manuelmartinez              126.579
elegantdice                 138.514
daedalusalpha               151.561
jounimalinen                206.163
arduino                     335.95
wikibooks_org_cpp           350.437
CppCodec                    526.187
internetsoftwareconsortium  862.833
libcurl                     1280.27
LihO                        1852.4
adp_gmbh                    1934.43
user152949                  5332.87

Polfosol's snippet from Base64 decode snippet in C++ is the fastest by a factor of almost 2x.

Here is the code for the sake of completeness:

static const int B64index[256] = { 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 62, 63, 62, 62, 63, 52, 53, 54, 55,
56, 57, 58, 59, 60, 61,  0,  0,  0,  0,  0,  0,  0,  0,  1,  2,  3,  4,  5,  6,
7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,  0,
0,  0,  0, 63,  0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51 };

std::string b64decode(const void* data, const size_t len)
{
    unsigned char* p = (unsigned char*)data;
    int pad = len > 0 && (len % 4 || p[len - 1] == '=');
    const size_t L = ((len + 3) / 4 - pad) * 4;
    std::string str(L / 4 * 3 + pad, '\0');

    for (size_t i = 0, j = 0; i < L; i += 4)
    {
        int n = B64index[p[i]] << 18 | B64index[p[i + 1]] << 12 | B64index[p[i + 2]] << 6 | B64index[p[i + 3]];
        str[j++] = n >> 16;
        str[j++] = n >> 8 & 0xFF;
        str[j++] = n & 0xFF;
    }
    if (pad)
    {
        int n = B64index[p[L]] << 18 | B64index[p[L + 1]] << 12;
        str[str.size() - 1] = n >> 16;

        if (len > L + 2 && p[L + 2] != '=')
        {
            n |= B64index[p[L + 2]] << 6;
            str.push_back(n >> 8 & 0xFF);
        }
    }
    return str;
}