Ok, so following a chat with Google engineering and after reading the code I've reached the following conclusions.

Passing binary data efficiently is impossible

It is impossible to pass binary data efficiently between JavaScript and Java through a @JavascriptInterface:

On the Java side:

@JavascriptInterface
void onBytes(byte[] bytes) {
   // bytes available here
}

And on the JavaScript side:

var byteArray = new Uint8Array(buffer);
var arr = new Uint8Array(byteArray.length);
for(var i = 0; i < byteArray.length; i++) {
  arr[i] = byteArray[i];
}
javaObject.onBytes(arr);

In the code above (from my old answer) and in Alex's - the conversion performed for the array is brutal:

case JavaType::TypeArray:
  if (value->IsType(base::Value::Type::DICTIONARY)) {
    result.l = CoerceJavaScriptDictionaryToArray(
        env, value, target_type, object_refs, error);
  } else if (value->IsType(base::Value::Type::LIST)) {
    result.l = CoerceJavaScriptListToArray(
        env, value, target_type, object_refs, error);
  } else {
    result.l = NULL;
  }
  break;

Which in turn coerces every array element to a Java object:

for (jsize i = 0; i < length; ++i) {
    const base::Value* value_element = null_value.get();
    list_value->Get(i, &value_element);
    jvalue element = CoerceJavaScriptValueToJavaValue(
        env, value_element, target_inner_type, false, object_refs, error);
    SetArrayElement(env, result, target_inner_type, i, element);

So, for a 1024 * 1024 * 10 Uint8Array - ten million Java objects are created and destroyed on each pass resulting in 10 seconds of CPU time on my emulator.

Creating an HTTP server

One thing we tried was creating an HTTP server and POSTing the result to it via an XMLHttpRequest. This worked - but ended up costing about 200ms of latency and also introduced a nasty memory leak.

MessageChannels are slow

Android API 23 added support for MessageChannels, which can be used via createWebMessageChannel() as shown in this answer. This is very slow, still serializes with GIN (like the @JavascriptInterface method) and incurs additional latency. I was not able to get this to work with reasonable performance.

It is worth mentioning that Google said they believe this is the way forward and hopes to promote message channels over @JavascriptInterface at some point.

Passing a string works

After reading the conversion code - one can see (and this was confirmed by Google) that the only way to avoid many conversions is to pass a String value. This only goes through:

case JavaType::TypeString: {
  std::string string_result;
  value->GetAsString(&string_result);
  result.l = ConvertUTF8ToJavaString(env, string_result).Release();
  break;
}

Which converts the result once to UTF8 and then again to a Java string. This still means the data (10MB in this case) is copied three times - but it is possible to pass 10MB of data in "only" 60ms - which is a lot more reasonable than the 10 seconds the above array method takes.

Petka came up with the idea of using 8859 encoding which can convert a single byte to a single letter. Unfortunately it is not supported in JavaScript's TextDecoder API - so Windows-1252 which is another 1 byte encoding can be used instead.

On the JavaScript side one can do:

var a = new Uint8Array(1024 * 1024 * 10); // your buffer
var b = a.buffer
// actually windows-1252 - but called iso-8859 in TextDecoder
var e = new TextDecoder("iso-8859-1"); 
var dec = e.decode(b);
proxy.onBytes(dec); // this is in the Java side.

Then, in the Java side:

@JavascriptInterface
public void onBytes(String dec) throws UnsupportedEncodingException
    byte[] bytes = dec.getBytes("windows-1252");
    // work with bytes here
}

Which runs in about 1/8th the time of direct serialization. It's still not very fast (since the string is padded to 16 bits instead of 8, then through UTF8 and then to UTF16 again). However, it runs in reasonable speed compared to the alternative.

After speaking with the relevant parties who are maintaining this code - they told me that it's as good as it can get with the current API. I was told I'm the first person to ask for this (fast JavaScript to Java serialization).

Java

using import com.sun.org.apache.xml.internal.security.utils.Base64;

byte[] b = new byte[] { 12, 3, 4, 5, 12 , 34, 100 };
String encoded = Base64.encode(b);

produces:

"DAMEBQwiZA=="

JavaScript

use atob and btoa

var stringToByteArray = function(str) {
    var bytes = [];
    for (var i = 0; i < str.length; ++i) {
        bytes.push(str.charCodeAt(i));
    }
    return bytes;
};
var decoded = stringToByteArray(atob("DAMEBQwiZA=="));

produces:

[ 12, 3, 4, 5, 12 , 34, 100 ]

Note: If you are doing this in NodeJS then take a look at the atob package.

You could convert the bytes to a string list and then parse them back into binary bytes.

class ArrayBufferUtil {
  static toString(buffer) {
    return new Uint8Array(buffer).toString()
  }
  
  static parse(s) {
    return new Uint8Array(
      s.split(',').map(i => parseInt(i, 10))
    ).buffer
  }
}

Test

console.log( ArrayBufferUtil.toString( new ArrayBuffer(8) ))
console.log( ArrayBufferUtil.parse( '0,0,0,0,0,0,0,0' ))

It turns out that as long as you don't look too closely at it, a JS Int8Array (and so a elemental2.core.Int8Array) behaves like you'd expect a GWT/Java byte[] to do - it will only contain values from -127 to 128. Unlike a "real" byte[], it will not correctly cast to byte[] and anything calling .getClass() or other Java methods on it will not work correctly either. It also can't be cast (by generics or explicitly) to byte[] either.

So if you need to treat the contents as a real Java array, you have to copy it first. Your answer will do what you want, but it might be more clearly correct to not "convert it from Double" along the way:

import elemental2.core.ArrayBuffer;
import elemental2.core.Int8Array;
import jsinterop.base.Js;

public static byte[] toBytes(ArrayBuffer buffer) {
    // Use uncheckedCast since we are outright lying to the compiler
    byte[] arr = Js.uncheckedCast(new Int8Array(buffer));
    byte[] result = new byte[arr.length];
    for (int i = 0; i < arr.length; i++) {
        result[i] = arr[i];
    }
    return result;
}

Your solution is also correct - a java.lang.Double in GWT and J2CL is exactly the same as a JS Number type, which is what your Int8Array is going to return when you query its values. The byteValue() call you're making will ask the compiler to make certain that the values are what is expected - it will truncate (probably via value & 0xFF?) the value, which should be cheap, but not entirely free.

Finally as long as you are just going to treat the resulting byte[] as a collection you can iterate, and won't be dealing with it in some way that the actual type will be checked, we can get away with not copying it at all:

import elemental2.core.ArrayBuffer;
import elemental2.core.Int8Array;
import jsinterop.base.Js;

public static byte[] toBytes(ArrayBuffer buffer) {
    byte[] result = Js.uncheckedCast(new Int8Array(buffer));
    return result;
}

This really is cheating though, and has other downsides, like if the original buffer is modified the array will be changed too, and vice versa. But it is very fast!

If you look into the Scala source code or ArrayBuffer, you'll see the following implementation of remove:

/** Removes the element on a given index position. It takes time linear in
   *  the buffer size.
   *
   *  @param n       the index which refers to the first element to delete.
   *  @param count   the number of elements to delete
   *  @throws Predef.IndexOutOfBoundsException if `n` is out of bounds.
   */
  override def remove(n: Int, count: Int) {
    require(count >= 0, "removing negative number of elements")
    if (n < 0 || n > size0 - count) throw new IndexOutOfBoundsException(n.toString)
    copy(n + count, n, size0 - (n + count))
    reduceToSize(size0 - count)
  }

So the removed element will no longer be present in the array, but indeed, the removal means copying all elements ("shifting" in your definition), and it will be slow for longer arrays.