img_file = request.session.get('image_file')
json.dumps(str(my_imagefield))
this should work for storing the image in the memory
or using base64
import base64
img_file = request.session.get('image_file')
with open(img_file , "wb") as fh:
fh.write(base64.decodebytes(img_data))
Answer from Exprator on Stack Overflowimg_file = request.session.get('image_file')
json.dumps(str(my_imagefield))
this should work for storing the image in the memory
or using base64
import base64
img_file = request.session.get('image_file')
with open(img_file , "wb") as fh:
fh.write(base64.decodebytes(img_data))
You have a few options and which you use will vary on your use case.
You can get the raw bytes of the image and return the serialized byte array.
You can get the raw bytes of the image and convert the bytes to hex and return the hex string.
Or you can get the raw bytes of the image and convert the bytes to a base64 string and return the base64 string.
As a side note InMemoryUploadedFile is a wrapper to to get the actual file you need to use .file
Error saving image via JSON in Python 3
Serialize an image, resize, and send back for Server Modules and REST APIs - Show and Tell - Anvil Community Forum
python - Serializing image stream using protobuf - Stack Overflow
python - How do I serialize an ImageField in Django? - Stack Overflow
Any ideas why the following doesn't work in Python 3, but does in 2.7?
def output_image(name, format, bytes):
image_start = "BEGIN_IMAGE_f9825uweof8jw9fj4r8"
image_end = "END_IMAGE_0238jfw08fjsiufhw8frs"
data = {}
data['name'] = name
data['format'] = format
data['bytes'] = base64.encodestring(bytes)
print(image_start+json.dumps(data)+image_end) It is called as follows:
output_image("test.png", "png", open("test.png", "rb").read())
In Python 2.7, the above, when called, produces and shows an image. In Python 3.6, I get the following error, which seems to be about JSON serialization. Any ideas? And what can I do to make it work?
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-c7be2be27290> in <module>()
36 ### draw the decision boundary with the text points overlaid
37 prettyPicture(clf, features_test, labels_test)
---> 38 output_image("test.png", "png", open("test.png", "rb").read())
~/class_vis.py in output_image(name, format, bytes)
57 data['format'] = format
58 data['bytes'] = base64.encodestring(bytes)
---> 59 print(image_start+json.dumps(data)+image_end)
~/anaconda/lib/python3.6/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
229 cls is None and indent is None and separators is None and
230 default is None and not sort_keys and not kw):
--> 231 return _default_encoder.encode(obj)
232 if cls is None:
233 cls = JSONEncoder
~/anaconda/lib/python3.6/json/encoder.py in encode(self, o)
197 # exceptions aren't as detailed. The list call should be roughly
198 # equivalent to the PySequence_Fast that ''.join() would do.
--> 199 chunks = self.iterencode(o, _one_shot=True)
200 if not isinstance(chunks, (list, tuple)):
201 chunks = list(chunks)
~/anaconda/lib/python3.6/json/encoder.py in iterencode(self, o, _one_shot)
255 self.key_separator, self.item_separator, self.sort_keys,
256 self.skipkeys, _one_shot)
--> 257 return _iterencode(o, 0)
258
259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
~/anaconda/lib/python3.6/json/encoder.py in default(self, o)
178 """
179 raise TypeError("Object of type '%s' is not JSON serializable" %
--> 180 o.__class__.__name__)
181
182 def encode(self, o):
TypeError: Object of type 'bytes' is not JSON serializableIf I had to do this, I would use one of:
message image {
int width = 1;
int height = 2;
bytes image_data = 3;
}
message image {
int width = 1;
int height = 2;
bytes red_data = 3;
bytes green_data = 4;
bytes blue_data = 5;
}
Or possibly use an intermediate ScanRow message, composed either of interleaved R, G, B bytes or separated R, G, B bytes. The first version is likely going to be fastest to generate and display.
According to this blog post : https://medium.com/@brynmathias/kafka-and-google-protobuf-a-match-made-in-python-a1bc3381da1a
you could use the following schema :
syntax = 'proto3';
package protobuf.data;
import "protobuf/common/messageData.proto";
message Image {
bytes image_data = 1;
int32 height = 2;
int32 width = 3;
int64 frame = 4;
protobuf.common.messageData _message_data = 5;
}
you can't serialize the object, because it's an Image. You have to serialize the string representation of it's path.
The easiest way of achiving it is to call it's str() method when you what to serialize it.
json.dumps(unicode(my_imagefield)) # py2
json.dumps(str(my_imagefield)) # py3
should work.
I wrote an extension to the simplejson encoder. Instead to serializing the image to base643, it returns the path of the image. Here's a snippet:
def encode_datetime(obj):
"""
Extended encoder function that helps to serialize dates and images
"""
if isinstance(obj, datetime.date):
try:
return obj.strftime('%Y-%m-%d')
except ValueError, e:
return ''
if isinstance(obj, ImageFieldFile):
try:
return obj.path
except ValueError, e:
return ''
raise TypeError(repr(obj) + " is not JSON serializable")
This might get you started:
import json
import base64
data = {}
with open('some.gif', mode='rb') as file:
img = file.read()
data['img'] = base64.encodebytes(img).decode('utf-8')
print(json.dumps(data))
Python 2
As the base64.encodebytes() has been deprecated in base64, the code snippet above can be modified as follows:
import json
import base64
data = {}
with open('some.gif', mode='rb') as file:
img = file.read()
data['img'] = base64.b64encode(img)
print(json.dumps(data))
Then, use base64.b64decode(data['img']) to convert back.
Description for the issue
The origin of the exception was a KeyError, because of this statement
images_data = validated_data.pop('images')
This is because the validated data has no key images. This means the images input doesn't validate the image inputs from postman.
Django post request store InMemmoryUpload in request.FILES, so we use it for fetching files. also, you want multiple image upload at once. So, you have to use different image_names while your image upload (in postman).
Change your serializer to like this:
class TaskSerializer(serializers.HyperlinkedModelSerializer):
user = serializers.ReadOnlyField(source='user.username')
images = TaskImageSerializer(source='taskimage_set', many=True, read_only=True)
class Meta:
model = Task
fields = ('id', 'title', 'user', 'images')
def create(self, validated_data):
images_data = self.context.get('view').request.FILES
task = Task.objects.create(title=validated_data.get('title', 'no-title'),
user_id=1)
for image_data in images_data.values():
TaskImage.objects.create(task=task, image=image_data)
return task
I don't know about your view, but I'd like to use ModelViewSet preferrable view class
class Upload(ModelViewSet):
serializer_class = TaskSerializer
queryset = Task.objects.all()
Postman console:

DRF result:
{
"id": 12,
"title": "This Is Task Title",
"user": "admin",
"images": [
{
"image": "http://127.0.0.1:8000/media/Screenshot_from_2017-12-20_07-18-43_tNIbUXV.png"
},
{
"image": "http://127.0.0.1:8000/media/game-of-thrones-season-valar-morghulis-wallpaper-1366x768_3bkMk78.jpg"
},
{
"image": "http://127.0.0.1:8000/media/IMG_212433_lZ2Mijj.jpg"
}
]
}
UPDATE
This is the answer for your comment.
In django reverse foreignKey are capturing using _set. see this official doc. Here, Task and TaskImage are in OneToMany relationship, so if you have one Task instance, you could get all related TaskImage instance by this reverse look-up feature.
Here is the example:
task_instance = Task.objects.get(id=1)
task_img_set_all = task_instance.taskimage_set.all()
Here this task_img_set_all will be equal to TaskImage.objects.filter(task_id=1)
You have read_only set to true in TaskImageSerializer nested field. So there will be no validated_data there.
You must be careful about the datatypes.
If you read a binary image, you get bytes. If you encode these bytes in base64, you get ... bytes again! (see documentation on b64encode)
json can't handle raw bytes, that's why you get the error.
I have just written some example, with comments, I hope it helps:
from base64 import b64encode
from json import dumps
ENCODING = 'utf-8'
IMAGE_NAME = 'spam.jpg'
JSON_NAME = 'output.json'
# first: reading the binary stuff
# note the 'rb' flag
# result: bytes
with open(IMAGE_NAME, 'rb') as open_file:
byte_content = open_file.read()
# second: base64 encode read data
# result: bytes (again)
base64_bytes = b64encode(byte_content)
# third: decode these bytes to text
# result: string (in utf-8)
base64_string = base64_bytes.decode(ENCODING)
# optional: doing stuff with the data
# result here: some dict
raw_data = {IMAGE_NAME: base64_string}
# now: encoding the data to json
# result: string
json_data = dumps(raw_data, indent=2)
# finally: writing the json string to disk
# note the 'w' flag, no 'b' needed as we deal with text here
with open(JSON_NAME, 'w') as another_open_file:
another_open_file.write(json_data)
Alternative solution would be encoding stuff on the fly with a custom encoder:
import json
from base64 import b64encode
class Base64Encoder(json.JSONEncoder):
# pylint: disable=method-hidden
def default(self, o):
if isinstance(o, bytes):
return b64encode(o).decode()
return json.JSONEncoder.default(self, o)
Having that defined you can do:
m = {'key': b'\x9c\x13\xff\x00'}
json.dumps(m, cls=Base64Encoder)
It will produce:
'{"key": "nBP/AA=="}'
Image-processing is CPU-expensive. So, the performance first:
ZeroMQ shall allow one to enjoy a Zero-Copy modus operandi, so prevent any adverse operations, that spoil that.
Having used just a generic OpenCV Camera, not the RPi / PiCamera, I always prefer to take individual Camera-frames ( not a sequence ) on the acquisition side under a controlled event-loop.
Camera gets a known, fixed-geometry picture ( in OpenCV a numpy.ndarray 3D-structure [X,Y,[B,G,R]] ), so the fastest and the most straightforward serialisation is using a struct.pack( CONST_FRAME_STRUCT_MASK, aFrame ) on sender-side and struct.unpack( CONST_FRAME_STRUCT_MASK, aMessage ) on the receiver(s)-side(s).
Yes, struct.pack() was so far a fastest way, even when documentation offers other means ( a flexibility comes at an additional cost, which is not justified ):
import numpy
def send_array( socket, A, flags = 0, copy = True, track = False ):
"""send a numpy array with metadata"""
md = dict( dtype = str( A.dtype ),
shape = A.shape,
)
pass; socket.send_json( md, flags | zmq.SNDMORE )
return socket.send( A, flags, copy = copy, track = track )
def recv_array( socket, flags = 0, copy = True, track = False ):
"""recv a numpy array"""
md = socket.recv_json( flags = flags )
msg = socket.recv( flags = flags, copy = copy, track = track )
buf = buffer( msg )
pass; A = numpy.frombuffer( buf, dtype = md['dtype'] )
return A.reshape( md['shape'] )
Any color-conversion and similar source-side transformations may consume +150 ~ 180 [ms], so try to avoid any and all un-necessary color-space or reshape or similar non-core conversions, as these adversely increase the accumulated pipeline latency envelope.
Using struct.pack() also avoids any kind of size-mismatches, so what you load onto a binary payload landing pad, is exactly what you receive on the receiver(s) side(s).
If indeed keen to have also a JSON-related overheads around the message core-data, then rather setup a two-socket paradigm, both having ZMQ_CONFLATE == 1, where the first moves struct-payloads and the second JSON-decorated telemetry.
If RPi permits, the zmq.Context( nIOthreads ) may further increase the data-pumping throughput on both sides with nIOthreads >= 2, and additional JSON_socket.setsockopt( ZMQ_AFFINITY, 1 ); VIDEO_socket.setsockopt( ZMQ_AFFINITY, 0 ) mapping can separate / distribute the workload to ride each on a different, separate IOthread.
Check out the code below . I've used Nlohmann json and some minor tweaks,(from various sources) to send images,vetors,strings etc.
Client code
#include <zmq.hpp>
#include <string>
#include <iostream>
#include <sstream>
#include <nlohmann/json.hpp>
#include <opencv2/opencv.hpp>
#include "opencv2/imgproc/imgproc_c.h"
#include "opencv2/imgproc/imgproc.hpp"
#include <typeinfo>
using json = nlohmann::json;
class image_test
{
public:
void client1(){
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REQ);
socket.connect ("tcp://localhost:5555");
while (true){
// create an empty structure (null)
json j;
std::string data;
float f = 3.12;
cv::Mat mat = cv::imread("cat.jpg",CV_LOAD_IMAGE_COLOR);
// std::cout<<Imgdata;
std::vector<uchar> array;
if (mat.isContinuous())
{
array.assign(mat.datastart, mat.dataend);
}
else
{
for (int i = 0; i < mat.rows; ++i)
{
array.insert(array.end(), mat.ptr<uchar>(i), mat.ptr<uchar>(i)+mat.cols);
}
}
std::vector<uint> v = {1,5,9};
j["Type"] = f;
j["vec"] = v;
j["Image"]["rows"] = mat.rows;
j["Image"]["cols"] = mat.cols;
j["Image"]["channels"] = mat.channels();
j["Image"]["data"] = array;
// add a Boolean that is stored as bool
j["Parameter"] = "Frequency";
// add a string that is stored as std::string
j["Value"] = "5.17e9";
// explicit conversion to string
std::string s = j.dump();
zmq::message_t request (s.size());
memcpy (request.data (), (s.c_str()), (s.size()));
socket.send(request);
zmq::message_t reply;
socket.recv (&reply);
std::string rpl = std::string(static_cast<char*>(reply.data()), reply.size());
json second = json::parse(rpl);
std::cout << second["num"] << std::endl;
}
}
};
int main (void)
{
image_test caller;
caller.client1();
}
Server code
import zmq
import json
import numpy as np
import matplotlib.pyplot as plt
import cv2
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")
while True:
json_str = socket.recv()
data_print = json.loads(json_str)
img = np.array(data_print["Image"]["data"])
img = img.reshape(data_print["Image"]["rows"],data_print["Image"]["cols"], data_print["Image"]["channels"])
b,g,r = cv2.split(img)
img = cv2.merge((r,g,b))
print(img.shape)
# plt.imshow(img)
# plt.show()
Type = data_print['Type']
Parameter = data_print['Parameter']
Value = data_print['Value']
a = {"info": "hello", "num":1}
socket.send(json.dumps(a))
The include package from nlohmann git should be included. Or yoy can directly download the source code and the link from : https://github.com/zsfVishnu/zmq.git. Also, if you're using g++ or any other compiler which doesn't include the include folder from nlohmann, just specify at the CLI ie add -I/path-to-the-include-folder/
You can convert the Image object into data then you can pickle it:
image = {
'pixels': im.tostring(),
'size': im.size,
'mode': im.mode,
}
And back to an Image:
im = Image.fromstring(image['mode'], image['size'], image['pixels'])
NOTE: As astex mentioned, if you're using Pillow (which is recommended instead of PIL), the tostring() method is deprecated for tobytes(). Likewise with fromstring() for frombytes().
Slight variation of Gerald's answer using keyword args
create pickleable object
image = {'data': im.tostring(), 'size':im.size, 'mode':im.mode}
or
image = dict(data=im.tostring(), size=im.size, mode=im.mode)
unpickle back to image
im = Image.fromstring(**image)