img_file = request.session.get('image_file')
json.dumps(str(my_imagefield)) 

this should work for storing the image in the memory

or using base64

import base64
img_file = request.session.get('image_file')
with open(img_file , "wb") as fh:
    fh.write(base64.decodebytes(img_data))
Answer from Exprator on Stack Overflow
๐ŸŒ
GitHub
github.com โ€บ zylo117 โ€บ msgpack-image
GitHub - zylo117/msgpack-image: Serialize numpy arrays using msgpack and convert images arrays in JPEG and back.
Serialize numpy arrays using msgpack and convert images arrays in JPEG and back. - zylo117/msgpack-image
Author ย  zylo117
Discussions

Error saving image via JSON in Python 3
format, bytes are python keywords. try changing the argument names. if it does not work, print the types in python 2.7 and in python 3 and compare. More on reddit.com
๐ŸŒ r/Python
6
0
December 7, 2017
Serialize an image, resize, and send back for Server Modules and REST APIs - Show and Tell - Anvil Community Forum
(moved to Show & Tell) One of the really cool features of Anvil is that it has a built-in file loader and a way to save binary media objects in datatables. One the occasion that you might need to serialize a binary image for transmission via a server function or REST API, the process is a bit ... More on anvil.works
๐ŸŒ anvil.works
1
1
June 1, 2019
python - Serializing image stream using protobuf - Stack Overflow
I have two programs in Ubuntu: a C++ program (TORCS game) and a python program. The C++ program always generates images. I want to transfer these real-time images into python(maybe the numpy.ndarray More on stackoverflow.com
๐ŸŒ stackoverflow.com
python - How do I serialize an ImageField in Django? - Stack Overflow
I'm trying to serialize one of my models which has an ImageField. The inbuilt serializer can't seem to serialize this and therefore I thought of writing a custom serializer. Could you tell me how I More on stackoverflow.com
๐ŸŒ stackoverflow.com
๐ŸŒ
Stack Overflow
stackoverflow.com โ€บ questions โ€บ 54471341 โ€บ how-to-serialize-a-image-into-str-and-deserialize-it-as-image
python - How to serialize a image into str and deserialize it as image? - Stack Overflow
Copy from PIL import Image from tinydb import TinyDB, Query import base64 import io from pdb import set_trace as bp # note: with 'encoding' in name, it is always a bytes obj in_jpg_encoding = None # open some randome image with open('rkt2.jpg', 'rb') as f: # The file content is a jpeg encoded bytes object in_jpg_encoding = f.read() # output is a bytes object in_b64_encoding = base64.b64encode(in_jpg_encoding) # interpret above bytes as str, because json value need to be string in_str = in_b64_encoding.decode(encoding='utf-8') # in_str = str(in_b64_encoding) # alternative way of above statement # simulates a transmission, e.g.
๐ŸŒ
Reddit
reddit.com โ€บ r/python โ€บ error saving image via json in python 3
r/Python on Reddit: Error saving image via JSON in Python 3
December 7, 2017 -

Any ideas why the following doesn't work in Python 3, but does in 2.7?

def output_image(name, format, bytes):
    image_start = "BEGIN_IMAGE_f9825uweof8jw9fj4r8"
    image_end = "END_IMAGE_0238jfw08fjsiufhw8frs"
    data = {}
    data['name'] = name
    data['format'] = format
    data['bytes'] = base64.encodestring(bytes)
    print(image_start+json.dumps(data)+image_end) 

It is called as follows:

output_image("test.png", "png", open("test.png", "rb").read())

In Python 2.7, the above, when called, produces and shows an image. In Python 3.6, I get the following error, which seems to be about JSON serialization. Any ideas? And what can I do to make it work?

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-c7be2be27290> in <module>()
     36 ### draw the decision boundary with the text points overlaid
     37 prettyPicture(clf, features_test, labels_test)
---> 38 output_image("test.png", "png", open("test.png", "rb").read())

~/class_vis.py in output_image(name, format, bytes)
     57     data['format'] = format
     58     data['bytes'] = base64.encodestring(bytes)
---> 59     print(image_start+json.dumps(data)+image_end)

~/anaconda/lib/python3.6/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    229         cls is None and indent is None and separators is None and
    230         default is None and not sort_keys and not kw):
--> 231         return _default_encoder.encode(obj)
    232     if cls is None:
    233         cls = JSONEncoder

~/anaconda/lib/python3.6/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

~/anaconda/lib/python3.6/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

~/anaconda/lib/python3.6/json/encoder.py in default(self, o)
    178         """
    179         raise TypeError("Object of type '%s' is not JSON serializable" %
--> 180                         o.__class__.__name__)
    181 
    182     def encode(self, o):

TypeError: Object of type 'bytes' is not JSON serializable
๐ŸŒ
PyPI
pypi.org โ€บ project โ€บ serialize-images
Pypi
September 2, 2020 - JavaScript is disabled in your browser. Please enable JavaScript to proceed ยท A required part of this site couldnโ€™t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser
๐ŸŒ
Anvil
anvil.works โ€บ show and tell
Serialize an image, resize, and send back for Server Modules and REST APIs - Show and Tell - Anvil Community Forum
June 1, 2019 - (moved to Show & Tell) One of the really cool features of Anvil is that it has a built-in file loader and a way to save binary media objects in datatables. One the occasion that you might need to serialize a binary image for transmission via a server function or REST API, the process is a bit ...
๐ŸŒ
Real Python
realpython.com โ€บ storing-images-in-python
Three Ways of Storing and Accessing Lots of Images in Python โ€“ Real Python
October 21, 2023 - Both the keys and values are expected to be strings, so the common usage is to serialize the value as a string, and then unserialize it when reading it back out. You can use pickle for the serializing. Any Python object can be serialized, so ...
Find elsewhere
๐ŸŒ
The Hitchhiker's Guide to Python
docs.python-guide.org โ€บ scenarios โ€บ serialization
Data Serialization โ€” The Hitchhiker's Guide to Python
If the data to be serialized is located in a file and contains flat data, Python offers two methods to serialize data.
๐ŸŒ
Stack Overflow
stackoverflow.com โ€บ questions โ€บ 45464064 โ€บ save-image-from-url-with-serializer
python - Save image from URL with serializer - Stack Overflow
August 2, 2017 - class Picture(Media): image = models.ImageField(upload_to=picture_path, blank=True, null=True) entity = models.CharField(max_length=20, choices=ENTITIES, default=ENTITY_CLASIFICADO) ... class PictureSerializer(serializers.ModelSerializer): image = serializers.ImageField(max_length=None, use_url=True) class Meta: model = Picture fields = ("pk", "image", "entity",)
๐ŸŒ
Bogotobogo
bogotobogo.com โ€บ python โ€บ python_serialization_pickle_json.php
Python Tutorial: serialization with pickle - 2020
Wrap it in a with statement to ensure the file is closed automatically when we're done with it. The dump() function in the pickle module takes a serializable Python data structure, serializes it into a binary, Python-specific format using the latest version of the pickle protocol, and saves it to an open file.
๐ŸŒ
SAP Blogs
blogs.sap.com โ€บ 2020 โ€บ 01 โ€บ 23 โ€บ how-to-use-python-3-to-save-an-image-via-a-json-payload-to-a-sap-hana-odata-endpoint
How to use Python 3 to save an image via a JSON payload to a SAP HANA OData endpoint
January 23, 2020 - Previously, I showed how to use Python 3 to get data from a SAP HANA OData endpoint. Since we can quickly build OData endpoints to back the data needed for our application on SAP HANA, it can be useful to be able to save an image via a JSON payload. For example, we can build an image similarity ...
Top answer
1 of 2
23

Description for the issue

The origin of the exception was a KeyError, because of this statement

images_data = validated_data.pop('images')

This is because the validated data has no key images. This means the images input doesn't validate the image inputs from postman.

Django post request store InMemmoryUpload in request.FILES, so we use it for fetching files. also, you want multiple image upload at once. So, you have to use different image_names while your image upload (in postman).

Change your serializer to like this:

class TaskSerializer(serializers.HyperlinkedModelSerializer):
    user = serializers.ReadOnlyField(source='user.username')
    images = TaskImageSerializer(source='taskimage_set', many=True, read_only=True)

    class Meta:
        model = Task
        fields = ('id', 'title', 'user', 'images')

    def create(self, validated_data):
        images_data = self.context.get('view').request.FILES
        task = Task.objects.create(title=validated_data.get('title', 'no-title'),
                                   user_id=1)
        for image_data in images_data.values():
            TaskImage.objects.create(task=task, image=image_data)
        return task

I don't know about your view, but I'd like to use ModelViewSet preferrable view class

class Upload(ModelViewSet):
    serializer_class = TaskSerializer
    queryset = Task.objects.all()

Postman console:

DRF result:

{
        "id": 12,
        "title": "This Is Task Title",
        "user": "admin",
        "images": [
            {
                "image": "http://127.0.0.1:8000/media/Screenshot_from_2017-12-20_07-18-43_tNIbUXV.png"
            },
            {
                "image": "http://127.0.0.1:8000/media/game-of-thrones-season-valar-morghulis-wallpaper-1366x768_3bkMk78.jpg"
            },
            {
                "image": "http://127.0.0.1:8000/media/IMG_212433_lZ2Mijj.jpg"
            }
        ]
    }

UPDATE

This is the answer for your comment.

In django reverse foreignKey are capturing using _set. see this official doc. Here, Task and TaskImage are in OneToMany relationship, so if you have one Task instance, you could get all related TaskImage instance by this reverse look-up feature.

Here is the example:

task_instance = Task.objects.get(id=1)
task_img_set_all = task_instance.taskimage_set.all()

Here this task_img_set_all will be equal to TaskImage.objects.filter(task_id=1)

2 of 2
1

You have read_only set to true in TaskImageSerializer nested field. So there will be no validated_data there.

Top answer
1 of 3
144

You must be careful about the datatypes.

If you read a binary image, you get bytes. If you encode these bytes in base64, you get ... bytes again! (see documentation on b64encode)

json can't handle raw bytes, that's why you get the error.

I have just written some example, with comments, I hope it helps:

from base64 import b64encode
from json import dumps

ENCODING = 'utf-8'
IMAGE_NAME = 'spam.jpg'
JSON_NAME = 'output.json'

# first: reading the binary stuff
# note the 'rb' flag
# result: bytes
with open(IMAGE_NAME, 'rb') as open_file:
    byte_content = open_file.read()

# second: base64 encode read data
# result: bytes (again)
base64_bytes = b64encode(byte_content)

# third: decode these bytes to text
# result: string (in utf-8)
base64_string = base64_bytes.decode(ENCODING)

# optional: doing stuff with the data
# result here: some dict
raw_data = {IMAGE_NAME: base64_string}

# now: encoding the data to json
# result: string
json_data = dumps(raw_data, indent=2)

# finally: writing the json string to disk
# note the 'w' flag, no 'b' needed as we deal with text here
with open(JSON_NAME, 'w') as another_open_file:
    another_open_file.write(json_data)
2 of 3
13

Alternative solution would be encoding stuff on the fly with a custom encoder:

import json
from base64 import b64encode

class Base64Encoder(json.JSONEncoder):
    # pylint: disable=method-hidden
    def default(self, o):
        if isinstance(o, bytes):
            return b64encode(o).decode()
        return json.JSONEncoder.default(self, o)

Having that defined you can do:

m = {'key': b'\x9c\x13\xff\x00'}
json.dumps(m, cls=Base64Encoder)

It will produce:

'{"key": "nBP/AA=="}'
Top answer
1 of 2
3

Image-processing is CPU-expensive. So, the performance first:

ZeroMQ shall allow one to enjoy a Zero-Copy modus operandi, so prevent any adverse operations, that spoil that.

Having used just a generic OpenCV Camera, not the RPi / PiCamera, I always prefer to take individual Camera-frames ( not a sequence ) on the acquisition side under a controlled event-loop.

Camera gets a known, fixed-geometry picture ( in OpenCV a numpy.ndarray 3D-structure [X,Y,[B,G,R]] ), so the fastest and the most straightforward serialisation is using a struct.pack( CONST_FRAME_STRUCT_MASK, aFrame ) on sender-side and struct.unpack( CONST_FRAME_STRUCT_MASK, aMessage ) on the receiver(s)-side(s).

Yes, struct.pack() was so far a fastest way, even when documentation offers other means ( a flexibility comes at an additional cost, which is not justified ):

import numpy

def send_array( socket, A, flags = 0, copy = True, track = False ):
    """send a numpy array with metadata"""
    md = dict( dtype = str( A.dtype ),
               shape =      A.shape,
               )
    pass;  socket.send_json( md, flags | zmq.SNDMORE )
    return socket.send(      A,  flags, copy = copy, track = track )

def recv_array( socket, flags = 0, copy = True, track = False ):
    """recv a numpy array"""
    md = socket.recv_json( flags = flags )
    msg = socket.recv(     flags = flags, copy = copy, track = track )
    buf = buffer( msg )
    pass;  A = numpy.frombuffer( buf, dtype = md['dtype'] )
    return A.reshape(                         md['shape'] )

Any color-conversion and similar source-side transformations may consume +150 ~ 180 [ms], so try to avoid any and all un-necessary color-space or reshape or similar non-core conversions, as these adversely increase the accumulated pipeline latency envelope.

Using struct.pack() also avoids any kind of size-mismatches, so what you load onto a binary payload landing pad, is exactly what you receive on the receiver(s) side(s).

If indeed keen to have also a JSON-related overheads around the message core-data, then rather setup a two-socket paradigm, both having ZMQ_CONFLATE == 1, where the first moves struct-payloads and the second JSON-decorated telemetry.

If RPi permits, the zmq.Context( nIOthreads ) may further increase the data-pumping throughput on both sides with nIOthreads >= 2, and additional JSON_socket.setsockopt( ZMQ_AFFINITY, 1 ); VIDEO_socket.setsockopt( ZMQ_AFFINITY, 0 ) mapping can separate / distribute the workload to ride each on a different, separate IOthread.

2 of 2
1

Check out the code below . I've used Nlohmann json and some minor tweaks,(from various sources) to send images,vetors,strings etc.

Client code

#include <zmq.hpp> 
#include <string>
#include <iostream>
#include <sstream>

#include <nlohmann/json.hpp> 
#include <opencv2/opencv.hpp>
#include "opencv2/imgproc/imgproc_c.h"
#include "opencv2/imgproc/imgproc.hpp"
#include <typeinfo>
using json = nlohmann::json;


class image_test
{
  public:
    void client1(){
      zmq::context_t context (1);
      zmq::socket_t socket (context, ZMQ_REQ);
      socket.connect ("tcp://localhost:5555");

      while (true){
        // create an empty structure (null)
        json j;
        std::string data;
        float f = 3.12;
        cv::Mat mat = cv::imread("cat.jpg",CV_LOAD_IMAGE_COLOR);

        // std::cout<<Imgdata;

        std::vector<uchar> array;
        if (mat.isContinuous()) 
          {
            array.assign(mat.datastart, mat.dataend);
          } 

        else 
          {
            for (int i = 0; i < mat.rows; ++i) 
              {
                  array.insert(array.end(), mat.ptr<uchar>(i), mat.ptr<uchar>(i)+mat.cols);
               }
          }

        std::vector<uint> v = {1,5,9};

        j["Type"] = f;
        j["vec"] = v;
        j["Image"]["rows"] = mat.rows;
        j["Image"]["cols"] = mat.cols;
        j["Image"]["channels"] = mat.channels();
        j["Image"]["data"] = array;

        // add a Boolean that is stored as bool
        j["Parameter"] = "Frequency";

        // add a string that is stored as std::string
        j["Value"] = "5.17e9";


        // explicit conversion to string
        std::string s = j.dump();  


        zmq::message_t request (s.size());
        memcpy (request.data (), (s.c_str()), (s.size()));
        socket.send(request);

        zmq::message_t reply;
        socket.recv (&reply);
        std::string rpl = std::string(static_cast<char*>(reply.data()), reply.size());

        json second = json::parse(rpl);

        std::cout << second["num"] << std::endl;


      }
    }         
};


int main (void)
{

  image_test caller;
  caller.client1();
}

Server code

import zmq
import json
import numpy as np
import matplotlib.pyplot as plt
import cv2

context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")

while True:

    json_str = socket.recv()

    data_print = json.loads(json_str)

    img = np.array(data_print["Image"]["data"])
    img = img.reshape(data_print["Image"]["rows"],data_print["Image"]["cols"], data_print["Image"]["channels"])

    b,g,r = cv2.split(img)
    img = cv2.merge((r,g,b))

    print(img.shape)
    # plt.imshow(img)
    # plt.show()

    Type = data_print['Type']
    Parameter = data_print['Parameter']
    Value = data_print['Value']

    a = {"info": "hello", "num":1}
socket.send(json.dumps(a))

The include package from nlohmann git should be included. Or yoy can directly download the source code and the link from : https://github.com/zsfVishnu/zmq.git. Also, if you're using g++ or any other compiler which doesn't include the include folder from nlohmann, just specify at the CLI ie add -I/path-to-the-include-folder/

๐ŸŒ
Python
docs.python.org โ€บ 3 โ€บ library โ€บ pickle.html
pickle โ€” Python object serialization
Serialization is a more primitive notion than persistence; although pickle reads and writes file objects, it does not handle the issue of naming persistent objects, nor the (even more complicated) issue of concurrent access to persistent objects. The pickle module can transform a complex object into a byte stream and it can transform the byte stream into an object with the same internal structure.