You can use newer OpenCV python interface (if I'm not mistaken it is available since OpenCV 2.2). It natively uses numpy arrays:

import cv2
im = cv2.imread("abc.tiff")
print(type(im))

result:

<type 'numpy.ndarray'>

I created a 2x2 JPEG image to test this. The image has white, red, green and purple pixels. I used cv2.imdecode and numpy.frombuffer

import cv2
import numpy as np

f = open('image.jpg', 'rb')
image_bytes = f.read()  # b'\xff\xd8\xff\xe0\x00\x10...'

decoded = cv2.imdecode(np.frombuffer(image_bytes, np.uint8), -1)

print('OpenCV:\n', decoded)

# your Pillow code
import io
from PIL import Image
image = np.array(Image.open(io.BytesIO(image_bytes))) 
print('PIL:\n', image)

This seems to work, although the channel order is BGR and not RGB as in PIL.Image. There are probably some flags you might use to tune this. Test results:

OpenCV:
 [[[255 254 255]
  [  0   0 254]]

 [[  1 255   0]
  [254   0 255]]]
PIL:
 [[[255 254 255]
  [254   0   0]]

 [[  0 255   1]
  [255   0 254]]]

If you effectively have the contents of a JPEG/PNG file in your variable called response, I think you can do:

frame = cv2.imdecode(response)

Or, you may need to do:

frame = cv2.imdecode(np.fromstring(response, np.uint8), cv2.IMREAD_COLOR)

Failing that, another way is like this:

from io import BytesIO
from scipy import misc

frame = misc.imread(BytesIO(response))

Your code can be fixed as follows:

import numpy as np, cv
vis = np.zeros((384, 836), np.float32)
h,w = vis.shape
vis2 = cv.CreateMat(h, w, cv.CV_32FC3)
vis0 = cv.fromarray(vis)
cv.CvtColor(vis0, vis2, cv.CV_GRAY2BGR)

Short explanation:

1. np.uint32 data type is not supported by OpenCV (it supports uint8, int8, uint16, int16, int32, float32, float64)
2. cv.CvtColor can't handle numpy arrays so both arguments has to be converted to OpenCV type. cv.fromarray do this conversion.
3. Both arguments of cv.CvtColor must have the same depth. So I've changed source type to 32bit float to match the ddestination.

Also I recommend you use newer version of OpenCV python API because it uses numpy arrays as primary data type:

import numpy as np, cv2
vis = np.zeros((384, 836), np.float32)
vis2 = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)

You don't need to convert NumPy array to Mat because OpenCV cv2 module can accept NumPyarray. The only thing you need to care for is that {0,1} is mapped to {0,255} and any value bigger than 1 in NumPy array is equal to 255. So you should divide by 255 in your code, as shown below.

img = numpy.zeros([5,5,3])

img[:,:,0] = numpy.ones([5,5])*64/255.0
img[:,:,1] = numpy.ones([5,5])*128/255.0
img[:,:,2] = numpy.ones([5,5])*192/255.0

cv2.imwrite('color_img.jpg', img)
cv2.imshow("image", img)
cv2.waitKey()