Basically what you do is correct. Looking at redmine docs you linked to, it seems that suffix after the dot in the url denotes type of posted data (.json for JSON, .xml for XML), which agrees with the response you get - Processing by AttachmentsController#upload as XML. I guess maybe there's a bug in docs and to post binary data you should try using http://redmine/uploads url instead of http://redmine/uploads.xml.

Btw, I highly recommend very good and very popular Requests library for http in Python. It's much better than what's in the standard lib (urllib2). It supports authentication as well but I skipped it for brevity here.

import requests
with open('./x.png', 'rb') as f:
    data = f.read()
res = requests.post(url='http://httpbin.org/post',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

# let's check if what we sent is what we intended to send...
import json
import base64

assert base64.b64decode(res.json()['data'][len('data:application/octet-stream;base64,'):]) == data

UPDATE

To find out why this works with Requests but not with urllib2 we have to examine the difference in what's being sent. To see this I'm sending traffic to http proxy (Fiddler) running on port 8888:

Using Requests

import requests

data = 'test data'
res = requests.post(url='http://localhost:8888',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

we see

POST http://localhost:8888/ HTTP/1.1
Host: localhost:8888
Content-Length: 9
Content-Type: application/octet-stream
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/1.0.4 CPython/2.7.3 Windows/Vista

test data

and using urllib2

import urllib2

data = 'test data'    
req = urllib2.Request('http://localhost:8888', data)
req.add_header('Content-Length', '%d' % len(data))
req.add_header('Content-Type', 'application/octet-stream')
res = urllib2.urlopen(req)

we get

POST http://localhost:8888/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 9
Host: localhost:8888
Content-Type: application/octet-stream
Connection: close
User-Agent: Python-urllib/2.7

test data

I don't see any differences which would warrant different behavior you observe. Having said that it's not uncommon for http servers to inspect User-Agent header and vary behavior based on its value. Try to change headers sent by Requests one by one making them the same as those being sent by urllib2 and see when it stops working.

Answer from Piotr Dobrogost on Stack Overflow
🌐
Hustle Play
hustleplay.wordpress.com › 2010 › 01 › 05 › sending-binary-with-json
Sending Binary with JSON | Hustle Play - WordPress.com
January 5, 2010 - In my work, I used Python on the packaging side and Java on the unpacking side. First, you need to open the file. Since the file is binary, you have to open it with an additional "b" flag.…
🌐
Python
docs.python.org › 3 › library › json.html
JSON encoder and decoder — Python 3.14.3 documentation
February 23, 2026 - JSONDecodeError – When the data being deserialized is not a valid JSON document. UnicodeDecodeError – When the data being deserialized does not contain UTF-8, UTF-16 or UTF-32 encoded data. ... Added the optional object_pairs_hook parameter. parse_constant doesn’t get called on ‘null’, ‘true’, ‘false’ anymore. ... All optional parameters are now keyword-only. fp can now be a binary file.
Discussions

Upload file as JSON to Python webserver
You are trying to upload binary files (word, jpg), serialised as JSON, and store them on the server. More on stackoverflow.com
🌐 stackoverflow.com
August 31, 2018
Python - Send file through JSON - Stack Overflow
I'm trying to create a live chat application using ws4py (if there are better socket libraries I would love recommendations) and I want to implement file transfer (specifically .wav files) in my More on stackoverflow.com
🌐 stackoverflow.com
March 20, 2017
python - sending binary data over json - Stack Overflow
I wanted to upload file to s3 using python. I am using requests_aws4 auth library for this import requests from requests_aws4auth import AWS4Auth # data=encode_audio(data) endpoint... More on stackoverflow.com
🌐 stackoverflow.com
April 21, 2017
How to convert a JSON file to Binary type?
You can read a file in binary mode as follows: with open('file.json', 'rb') as f: json_binary = file.read() More on reddit.com
🌐 r/learnpython
7
2
October 30, 2022
Top answer
1 of 4
104

Basically what you do is correct. Looking at redmine docs you linked to, it seems that suffix after the dot in the url denotes type of posted data (.json for JSON, .xml for XML), which agrees with the response you get - Processing by AttachmentsController#upload as XML. I guess maybe there's a bug in docs and to post binary data you should try using http://redmine/uploads url instead of http://redmine/uploads.xml.

Btw, I highly recommend very good and very popular Requests library for http in Python. It's much better than what's in the standard lib (urllib2). It supports authentication as well but I skipped it for brevity here.

import requests
with open('./x.png', 'rb') as f:
    data = f.read()
res = requests.post(url='http://httpbin.org/post',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

# let's check if what we sent is what we intended to send...
import json
import base64

assert base64.b64decode(res.json()['data'][len('data:application/octet-stream;base64,'):]) == data

UPDATE

To find out why this works with Requests but not with urllib2 we have to examine the difference in what's being sent. To see this I'm sending traffic to http proxy (Fiddler) running on port 8888:

Using Requests

import requests

data = 'test data'
res = requests.post(url='http://localhost:8888',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

we see

POST http://localhost:8888/ HTTP/1.1
Host: localhost:8888
Content-Length: 9
Content-Type: application/octet-stream
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/1.0.4 CPython/2.7.3 Windows/Vista

test data

and using urllib2

import urllib2

data = 'test data'    
req = urllib2.Request('http://localhost:8888', data)
req.add_header('Content-Length', '%d' % len(data))
req.add_header('Content-Type', 'application/octet-stream')
res = urllib2.urlopen(req)

we get

POST http://localhost:8888/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 9
Host: localhost:8888
Content-Type: application/octet-stream
Connection: close
User-Agent: Python-urllib/2.7

test data

I don't see any differences which would warrant different behavior you observe. Having said that it's not uncommon for http servers to inspect User-Agent header and vary behavior based on its value. Try to change headers sent by Requests one by one making them the same as those being sent by urllib2 and see when it stops working.

2 of 4
4

This has nothing to do with a malformed upload. The HTTP error clearly specifies 401 unauthorized, and tells you the CSRF token is invalid. Try sending a valid CSRF token with the upload.

More about csrf tokens here:

What is a CSRF token ? What is its importance and how does it work?

🌐
Stack Overflow
stackoverflow.com › questions › 43538675 › sending-binary-data-over-json
python - sending binary data over json - Stack Overflow
April 21, 2017 - I wanted to upload file to s3 using python. I am using requests_aws4 auth library for this · import requests from requests_aws4auth import AWS4Auth # data=encode_audio(data) endpoint = 'http://bucket.s3.amazonaws.com/testing89.mp3' data = //some binary data(mp3) here auth = AWS4Auth('xxxxxx', 'xxxxxx', 'eu-west-2', 's3') response = requests.put(endpoint,data=data, auth=auth, headers={'x-amz-acl': 'public-read'}) print response.text ·
🌐
Python for the Lab
pythonforthelab.com › blog › storing-binary-data-and-serializing
Storing Binary Data and Serializing | Python For The Lab
August 11, 2018 - The limit of JSON is, however, that you have to store data as text files, thus limiting its native capabilities. Fortunately, combining Pickle and base64, you can transform bytes to an ascii string and save it next to easy to read metadata. This article has gone much more in depth regarding how to store data in different formats, but the topic is far from complete. Keep tuned to find more articles regarding how to save data with Python.
Find elsewhere
🌐
GitHub
github.com › fangq › pybj
GitHub - fangq/pybj: Binary JSON (BJData/UBJSON) Support for Python · GitHub
python3 -mbjdata USAGE: bjdata (fromjson|tojson) (INFILE|-) [OUTFILE] EXAMPLES: python3 -mbjdata fromjson input.json output.bjd python3 -mbjdata tojson input.bjd output.json
Author   fangq
🌐
Quora
quora.com › How-do-you-send-binary-data-in-JSON
How to send binary data in JSON - Quora
Description: Store binary in a file/blob store or object storage and put a URL/ID in JSON. Example: { "fileId": "abc123", "url": "https://cdn.example.com/abc123" } Pros: No JSON bloat, efficient for large files, supports streaming, range requests, ...
🌐
FastAPI
fastapi.tiangolo.com › tutorial › request-files
Request Files - FastAPI
You can declare multiple File and Form parameters in a path operation, but you can't also declare Body fields that you expect to receive as JSON, as the request will have the body encoded using multipart/form-data instead of application/json. This is not a limitation of FastAPI, it's part of the HTTP protocol. You can make a file optional by using standard type annotations and setting a default value of None: ... from typing import Annotated from fastapi import FastAPI, File, UploadFile app = FastAPI() @app.post("/files/") async def create_file(file: Annotated[bytes | None, File()] = None): if not file: return {"message": "No file sent"} else: return {"file_size": len(file)} @app.post("/uploadfile/") async def create_upload_file(file: UploadFile | None = None): if not file: return {"message": "No upload file sent"} else: return {"filename": file.filename}
🌐
cyberangles
cyberangles.org › blog › using-python-requests-to-send-file-and-json-in-single-request
How to Use Python Requests to Send File and JSON in a Single POST Request (Troubleshooting Guide) — CyberAngles.org
The key steps are: Serialize JSON data to a string with json.dumps(). Package JSON and files into a files dictionary, with explicit content types. Open files in binary mode ("rb") to avoid corruption.
🌐
PyTutorial
pytutorial.com › handling-binary-data-in-python-json
PyTutorial | Handling Binary Data in Python JSON
November 7, 2024 - # Reading binary file with open('image.png', 'rb') as file: binary_data = file.read() # Encode and save to JSON image_data = { 'filename': 'image.png', 'content': base64.b64encode(binary_data).decode('utf-8') } with open('image_data.json', 'w') as json_file: json.dump(image_data, json_file)
🌐
Stack Abuse
stackabuse.com › how-to-upload-files-with-pythons-requests-library
How to Upload Files with Python's requests Library
September 19, 2021 - The path of the file can be an absolute path or a relative path to where the script is being run. If you're uploading a file in the same directory, you can just use the file's name. The second argument, mode, will take the "read binary" value which is represented by rb.
🌐
GitHub
github.com › psf › requests › issues › 1266
How to upload binary file using PUT? · Issue #1266 · psf/requests
March 26, 2013 - def filegen(fo): yield fo.read(CHUNK_SIZE) def main(): session = requests.Session() session.auth = HTTPKerberosAuth() upload_headers = {'X-Codesigner-security-token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'} with open(pathname, 'rb') as infile: response = session.put(upload_url, headers=upload_headers, data=filegen(infile)) ... Request headers: { 'Accept-Encoding': 'gzip, deflate, compress', 'X-Codesigner-security-token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Accept': '*/*', 'User-Agent': 'python-requests/1.1.0 CPython/2.7.1 Windows/7', 'Transfer-Encoding': 'chunked', 'Authorization': 'Negotiate <bl
Author   dabono
🌐
Stack Overflow
stackoverflow.com › questions › 21645910 › binary-file-upload-python-pycurl-c
Binary file upload (python, PyCurl, C#) - Stack Overflow
WebClient client = new WebClient(); client.UploadFileAsync(uri, "POST", "C:\\Users\\user1\\file\\to\\the\\firmware\\fw-container.efc"); Here the problem is that upload does not succeed (doesn't even start uploading in Python) although the script runs through. The problem should be in the code, because other way the upload could be completed Python code: import pycurl from cStringIO import StringIO import urllib2 import simplejson as json url = 'http://eData/pvi?rName=FirmwareUpload' req = urllib2.Request(url) req.add_header('Content-Type','application/json') c = pycurl.Curl() c.setopt(c.POST,