Using TextEncoder and TextDecoder

var uint8array = new TextEncoder("utf-8").encode("Plain Text");
var string = new TextDecoder().decode(uint8array);
console.log(uint8array ,string )
Run code snippetEdit code snippet Hide Results Copy to answer Expand

Answer from PPB on Stack Overflow
Top answer
1 of 16
352

Update 2016 - five years on there are now new methods in the specs (see support below) to convert between strings and typed arrays using proper encoding.

##TextEncoder

The TextEncoder represents:

The TextEncoder interface represents an encoder for a specific method, that is a specific character encoding, liarraybufferke utf-8, iso-8859-2, koi8, cp1261, gbk, ... An encoder takes a stream of code points as input and emits a stream of bytes.

Change note since the above was written: (ibid.)

Note: Firefox, Chrome and Opera used to have support for encoding types other than utf-8 (such as utf-16, iso-8859-2, koi8, cp1261, and gbk). As of Firefox 48 [...], Chrome 54 [...] and Opera 41, no other encoding types are available other than utf-8, in order to match the spec.*

*) Updated specs (W3) and here (whatwg).

After creating an instance of the TextEncoder it will take a string and encode it using a given encoding parameter:

if (!("TextEncoder" in window)) 
  alert("Sorry, this browser does not support TextEncoder...");

var enc = new TextEncoder(); // always utf-8
console.log(enc.encode("This is a a string to be converted to a Uint8Array"));

You then of course use the .buffer parameter on the resulting Uint8Array to convert the underlaying ArrayBuffer to a different view if needed.

Just make sure that the characters in the string adhere to the encoding schema, for example, if you use characters outside the UTF-8 range in the example they will be encoded to two bytes instead of one.

For general use you would use UTF-16 encoding for things like localStorage.

##TextDecoder

Likewise, the opposite process uses the TextDecoder:

The TextDecoder interface represents a decoder for a specific method, that is a specific character encoding, like utf-8, iso-8859-2, koi8, cp1261, gbk, ... A decoder takes a stream of bytes as input and emits a stream of code points.

All available decoding types can be found here.

if (!("TextDecoder" in window))
  alert("Sorry, this browser does not support TextDecoder...");

var enc = new TextDecoder("utf-8");
var arr = new Uint8Array([84,104,105,115,32,105,115,32,97,32,85,105,110,116,
                          56,65,114,114,97,121,32,99,111,110,118,101,114,116,
                          101,100,32,116,111,32,97,32,115,116,114,105,110,103]);
console.log(enc.decode(arr));

##The MDN StringView library

An alternative to these is to use the StringView library (licensed as lgpl-3.0) which goal is:

  • to create a C-like interface for strings (i.e., an array of character codes — an ArrayBufferView in JavaScript) based upon the JavaScript ArrayBuffer interface
  • to create a highly extensible library that anyone can extend by adding methods to the object StringView.prototype
  • to create a collection of methods for such string-like objects (since now: stringViews) which work strictly on arrays of numbers rather than on creating new immutable JavaScript strings
  • to work with Unicode encodings other than JavaScript's default UTF-16 DOMStrings

giving much more flexibility. However, it would require us to link to or embed this library while TextEncoder/TextDecoder is being built-in in modern browsers.

#Support

As of July/2018:

TextEncoder (Experimental, On Standard Track)

 Chrome    | Edge      | Firefox   | IE        | Opera     | Safari
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |    19°    |     -     |     25    |     -

 Chrome/A  | Edge/mob  | Firefox/A | Opera/A   |Safari/iOS | Webview/A
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |    19°    |     ?     |     -     |     38

°) 18: Firefox 18 implemented an earlier and slightly different version
of the specification.

WEB WORKER SUPPORT:

Experimental, On Standard Track

 Chrome    | Edge      | Firefox   | IE        | Opera     | Safari
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |     20    |     -     |     25    |     -

 Chrome/A  | Edge/mob  | Firefox/A | Opera/A   |Safari/iOS | Webview/A
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |     20    |     ?     |     -     |     38

Data from MDN - `npm i -g mdncomp` by epistemex
2 of 16
207

Although Dennis and gengkev solutions of using Blob/FileReader work, I wouldn't suggest taking that approach. It is an async approach to a simple problem, and it is much slower than a direct solution. I've made a post in html5rocks with a simpler and (much faster) solution: http://updates.html5rocks.com/2012/06/How-to-convert-ArrayBuffer-to-and-from-String

And the solution is:

function ab2str(buf) {
  return String.fromCharCode.apply(null, new Uint16Array(buf));
}

function str2ab(str) {
  var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i=0, strLen=str.length; i<strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}

EDIT:

The Encoding API helps solving the string conversion problem. Check out the response from Jeff Posnik on Html5Rocks.com to the above original article.

Excerpt:

The Encoding API makes it simple to translate between raw bytes and native JavaScript strings, regardless of which of the many standard encodings you need to work with.

<pre id="results"></pre>

<script>
  if ('TextDecoder' in window) {
    // The local files to be fetched, mapped to the encoding that they're using.
    var filesToEncoding = {
      'utf8.bin': 'utf-8',
      'utf16le.bin': 'utf-16le',
      'macintosh.bin': 'macintosh'
    };

    Object.keys(filesToEncoding).forEach(function(file) {
      fetchAndDecode(file, filesToEncoding[file]);
    });
  } else {
    document.querySelector('#results').textContent = 'Your browser does not support the Encoding API.'
  }

  // Use XHR to fetch `file` and interpret its contents as being encoded with `encoding`.
  function fetchAndDecode(file, encoding) {
    var xhr = new XMLHttpRequest();
    xhr.open('GET', file);
    // Using 'arraybuffer' as the responseType ensures that the raw data is returned,
    // rather than letting XMLHttpRequest decode the data first.
    xhr.responseType = 'arraybuffer';
    xhr.onload = function() {
      if (this.status == 200) {
        // The decode() method takes a DataView as a parameter, which is a wrapper on top of the ArrayBuffer.
        var dataView = new DataView(this.response);
        // The TextDecoder interface is documented at http://encoding.spec.whatwg.org/#interface-textdecoder
        var decoder = new TextDecoder(encoding);
        var decodedString = decoder.decode(dataView);
        // Add the decoded file's text to the <pre> element on the page.
        document.querySelector('#results').textContent += decodedString + '\n';
      } else {
        console.error('Error while requesting', file, this);
      }
    };
    xhr.send();
  }
</script>
🌐
Java2s
java2s.com › example › javascript › language-basics › conversion-between-utf8-arraybuffer-and-string.html
Conversion between UTF-8 ArrayBuffer and String - Javascript Language Basics
<html> <head> <meta name="viewport" content="width=device-width, initial-scale=1"> <script type="text/javascript"> window.onload=function(){/* ww w. j av a 2 s. com*/ var array = new Uint8Array(3); array[0] = 0xe2; array[1] = 0x82; array[2] = 0xac; function uintToString(uintArray) { var encodedString = String.fromCharCode.apply(null, uintArray), decodedString = decodeURIComponent(escape(encodedString)); return decodedString; }
🌐
GitHub
github.com › dy › arraybuffer-to-string
GitHub - dy/arraybuffer-to-string: Convert ArrayBuffer to string · GitHub
var ab2str = require('arraybuffer-to-string') var uint8 = new Uint8Array([ 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33 ]) ab2str(uint8) // 'Hello World!' ab2str(uint8, 'base64') // 'SGVsbG8gV29ybGQh' ab2str(uint8, 'hex') // '48656c6c6f20576f726c6421' ab2str(uint8, 'iso-8859-2') // 'Hello World!'
Author   dy
🌐
GitHub
gist.github.com › boushley › 5471599
A JavaScript UTF-8 decoding function for ArrayBuffers. Credit for most of the heavy lifting goes to "bob" http://ciaranj.blogspot.com/2007/11/utf8-characters-encoding-in-javascript.html · GitHub
A JavaScript UTF-8 decoding function for ArrayBuffers. Credit for most of the heavy lifting goes to "bob" http://ciaranj.blogspot.com/2007/11/utf8-characters-encoding-in-javascrip...
🌐
GitHub
github.com › samirkumardas › dataviewUTF-8
GitHub - samirkumardas/dataviewUTF-8: Javascript encoding method from ArrayBuffer to utf-8 String
Javascript method for converting ArrayBuffer to utf-8 String · You will have to add utf8.js file in your project.
Author   samirkumardas
🌐
GeeksforGeeks
geeksforgeeks.org › node-js-npm-arraybuffer-to-string-module
Node.js NPM arraybuffer-to-string Module | GeeksforGeeks
January 11, 2022 - There is an NPM package called arraybuffer-to-string used to decode array buffers in actual strings. The package not only converts the buffer to 'utf8' string but also it converts the buffer to many forms like base64 encoded string, a hex-encoded ...
🌐
Restack
restack.io › p › javascript-ai-search-optimization-techniques-answer-arraybuffer-to-string-utf8-cat-ai
Where product teams design, test and optimize agents at Enterprise Scale — Restack
The open-source stack enabling product teams to improve their agent experience while engineers make them reliable at scale on Kubernetes.
🌐
GitHub
github.com › nfroidure › utf-8
GitHub - nfroidure/utf-8: A simple JavaScript library to manage UTF8 strings contained in ArrayBuffers
A simple JavaScript library to manage UTF8 strings contained in ArrayBuffers - nfroidure/utf-8
Starred by 19 users
Forked by 6 users
Languages   TypeScript 96.4% | JavaScript 3.6% | TypeScript 96.4% | JavaScript 3.6%
Find elsewhere
🌐
amanhimself.dev
amanhimself.dev › blog › converting-a-buffer-to-json-and-utf8-strings-in-nodejs
Converting a Buffer to JSON and Utf8 Strings in Nodejs | amanhimself.dev
August 10, 2017 - console.log(bufferOriginal.toString('utf8')); // Output: This is a buffer example. .toString() is not the only way to convert a buffer to a string.
🌐
Coolaj86
coolaj86.com › articles › unicode-string-to-a-utf-8-typed-array-buffer-in-javascript
Unicode String to a UTF-8 TypedArray Buffer in JavaScript
June 26, 2015 - 'use strict'; // uint array to string function typedArrayToUnicodeString(ua) { var binstr = Array.prototype.map.call(ua, function (ch) { return String.fromCharCode(ch); }).join(''); var escstr = binstr.replace(/(.)/g, function (m, p) { var code = p.charCodeAt(p).toString(16).toUpperCase(); if (code.length < 2) { code = '0' + code; } return '%' + code; }); return decodeURIComponent(escstr); }
Top answer
1 of 2
3

Why not use the TextDecoder interface instead of rolling your own? Are you limited to a browser that doesn't support it?

const decoder = new TextDecoder('UTF-8')
const dataStr = decoder.decode(dataBuf) 
2 of 2
2

We need some assumptions to understand what happened:

1. JS uses UTF-16

First of all, js uses UTF-16 for storing symbols as it mention here in section unicode strings: https://developer.mozilla.org/en-US/docs/Web/API/btoa

2. UTF-16 and UTF-8

UTF-8 and UTF-16 doesn't mean that one symbol represents by one byte or two bytes. UTF-8 such as utf-16 is a variable-length encoding.

3. ArrayBuffer and encodings

"hello" by one byte (Uint8Array): [104, 101, 108, 108, 111]
the same by two bytes (Uint16Array): [0, 104, 0, 101, 0, 108, 0, 108, 0, 111]

There is no encoding in ArrayBuffer because ArrayBuffer represent numbers.

Iteration over second array will be different from iteration over the first array. You know that the two-byte number cannot be pack onto one-byte number.


When you receive response from server in utf-8 - you receive it as sequence of bytes and if data you receive are stored by one-byte per symbol your code will work fine - it works with symbols like [a-zA-Z0-9] and common punctuation symbols. But if you receive a symbol that stores with two bytes in UTF-8 the transcription onto UTF-16 will be incorrect:

0xC3 0xA5 (one symbol å) -> 0x00 0xC3 0x00 0xA5 (two symbols "Ã¥")

So if you will not transfer symbols outside the range of latin symbols, digits and punctuation you can use your code and it will work fine even though it is not correct.

🌐
npm
npmjs.com › package › arraybuffer-to-string
arraybuffer-to-string - npm
Convert ArrayBuffer/ArrayBufferView/Array buffer to string with defined encoding. Available encoding: utf8, binary, base64, hex, ascii, latin1, ucs2, utf16 and many others.
      » npm install arraybuffer-to-string
    
Published   Feb 23, 2018
Version   1.0.2
🌐
Chrome Developers
developer.chrome.com › blog › easier arraybuffer to string conversion with the encoding api
Easier ArrayBuffer to String conversion with the Encoding API | Blog | Chrome for Developers
As demonstrated by this live sample, excerpted below, the Encoding API makes it simple to translate between raw bytes and native JavaScript strings, regardless of which of the many standard encodings you need to work with. <pre id="results"></pre> <script> if ('TextDecoder' in window) { // The local files to be fetched, mapped to the encoding that they're using. var filesToEncoding = { 'utf8.bin': 'utf-8', 'utf16le.bin': 'utf-16le', 'macintosh.bin': 'macintosh' }; Object.keys(filesToEncoding).forEach(function(file) { fetchAndDecode(file, filesToEncoding[file]); }); } else { document.querySelector('#results').textContent = 'Your browser does not support the Encoding API.'
🌐
Medium
medium.com › hackernoon › https-medium-com-amanhimself-converting-a-buffer-to-json-and-utf8-strings-in-nodejs-2150b1e3de57
Converting a Buffer to JSON and Utf8 Strings in Nodejs | by Aman Mittal | HackerNoon.com | Medium
April 1, 2019 - Converting a Buffer to JSON and Utf8 Strings in Nodejs Nodejs and browser based JavaScript differ because Node has a way to handle binary data even before the ES6 draft came up with ArrayBuffer. In …
Top answer
1 of 1
15

is this enough?

function stringToArrayBuffer(str){
    if(/[\u0080-\uffff]/.test(str)){
        throw new Error("this needs encoding, like UTF-8");
    }
    var arr = new Uint8Array(str.length);
    for(var i=str.length; i--; )
        arr[i] = str.charCodeAt(i);
    return arr.buffer;
}

function arrayBufferToString(buffer){
    var arr = new Uint8Array(buffer);
    var str = String.fromCharCode.apply(String, arr);
    if(/[\u0080-\uffff]/.test(str)){
        throw new Error("this string seems to contain (still encoded) multibytes");
    }
    return str;
}

or do you need real UTF-8 encoding

Edit: full UTF-8 support

Beware/Disclaimer: this code is not tested against some foreign implementaion of an UTF-8 encoder or decoder. It may produce wrong results.

TEST IT YOURSELF, before you use it in production!

function stringToArrayBuffer(str){
    if(/[\u0080-\uffff]/.test(str)){
        var arr = new Array(str.length);
        for(var i=0, j=0, len=str.length; i<len; ++i){
            var cc = str.charCodeAt(i);
            if(cc < 128){
                //single byte
                arr[j++] = cc;
            }else{
                //UTF-8 multibyte
                if(cc < 2048){
                    arr[j++] = (cc >> 6) | 192;
                }else{
                    arr[j++] = (cc >> 12) | 224;
                    arr[j++] = ((cc >> 6) & 63) | 128;
                }
                arr[j++] = (cc & 63) | 128;
            }
        }
        var byteArray = new Uint8Array(arr);
    }else{
        var byteArray = new Uint8Array(str.length);
        for(var i = str.length; i--; )
            byteArray[i] = str.charCodeAt(i);
    }
    return byteArray.buffer;
}

function arrayBufferToString(buffer){
    var byteArray = new Uint8Array(buffer);
    var str = "", cc = 0, numBytes = 0;
    for(var i=0, len = byteArray.length; i<len; ++i){
        var v = byteArray[i];
        if(numBytes > 0){
            //2 bit determining that this is a tailing byte + 6 bit of payload
            if((cc&192) === 192){
                //processing tailing-bytes
                cc = (cc << 6) | (v & 63);
            }else{
                throw new Error("this is no tailing-byte");
            }
        }else if(v < 128){
            //single-byte
            numBytes = 1;
            cc = v;
        }else if(v < 192){
            //these are tailing-bytes
            throw new Error("invalid byte, this is a tailing-byte")
        }else if(v < 224){
            //3 bits of header + 5bits of payload
            numBytes = 2;
            cc = v & 31;
        }else if(v < 240){
            //4 bits of header + 4bit of payload
            numBytes = 3;
            cc = v & 15;
        }else{
            //UTF-8 theoretically supports up to 8 bytes containing up to 42bit of payload
            //but JS can only handle 16bit.
            throw new Error("invalid encoding, value out of range")
        }

        if(--numBytes === 0){
            str += String.fromCharCode(cc);
        }
    }
    if(numBytes){
        throw new Error("the bytes don't sum up");
    }
    return str;
}
🌐
Toptal
toptal.com › external-blogs › adeva › convert-node-js-buffer-to-string
How to Convert Node.js Buffer to String | Toptal®
May 26, 2026 - const b = Buffer.alloc(10); b.write('test', 4, 'utf8'); console.log(b); // <Buffer 00 00 00 00 74 65 73 74 00 00> In a nutshell, it’s easy to convert a Buffer object to a string using the toString() method.
🌐
Chrome Developers
developer.chrome.com › blog › how to convert arraybuffer to and from string
How to convert ArrayBuffer to and from String | Blog | Chrome for Developers
June 14, 2012 - See Easier ArrayBuffer <-> String conversion with the Encoding API for more details. ArrayBuffers are used to transport raw data and several new APIs rely on them, including WebSockets, Web Intents 2](https://www.html5rocks.com/en/tutorials/file/xhr2/) and WebWorkers. However, because they recently landed in the JavaScript ...
🌐
haikel-fazzani
haikel-fazzani.eu.org › snippet › string-uint8array-conversion
Converting String to Uint8Array & Vice-Versa
October 23, 2024 - In Node.js, the Buffer class provides a simple way to handle binary data. const str = 'Hello, World!'; const uint8Array = Buffer.from(str, 'utf-8'); console.log(uint8Array); // Output: <Buffer 48 65 6c 6c 6f 2c 20 57 6f 72 6c 64 21> ... Optimized ...