Try something like this

import requests
data='param1,state=test,param2=1 param3=2.932,param4=3250 1497064544944 '
p = requests.post('http://myServerAddress/write?db=some_data', data.encode())
Answer from Junyong Yao on Stack Overflow
🌐
ReqBin
reqbin.com › req › python › c-d2nzjn3z › curl-post-body
Python | How do I post request body with Curl?
November 29, 2023 - Curl will send data to the server in the same format as the browser when submitting an HTML form. To send binary data in the body of a POST message with Curl, use the --data-binary command-line option.
🌐
SaltyCrane
saltycrane.com › blog › 2012 › 08 › example-posting-binary-data-using-pycurl
Example POSTing binary data using pycurl - SaltyCrane Blog
August 1, 2012 - import StringIO import os.path import pycurl def main(): """ http://curl.haxx.se/libcurl/c/curl_easy_setopt.html http://code.activestate.com/recipes/576422-python-http-post-binary-file-upload-with-pycurl/ http://pycurl.cvs.sourceforge.net/pycurl/pycurl/tests/test_post2.py?view=markup """ method = 4 filename = 'blank-contact-photo.jpg' url = 'http://localhost:8000' c = pycurl.Curl() c.setopt(pycurl.VERBOSE, 1) c.setopt(pycurl.URL, url) fout = StringIO.StringIO() c.setopt(pycurl.WRITEFUNCTION, fout.write) if method == 1: c.setopt(pycurl.HTTPPOST, [ ("file1", (c.FORM_FILE, filename))]) c.setopt(p
Top answer
1 of 4
104

Basically what you do is correct. Looking at redmine docs you linked to, it seems that suffix after the dot in the url denotes type of posted data (.json for JSON, .xml for XML), which agrees with the response you get - Processing by AttachmentsController#upload as XML. I guess maybe there's a bug in docs and to post binary data you should try using http://redmine/uploads url instead of http://redmine/uploads.xml.

Btw, I highly recommend very good and very popular Requests library for http in Python. It's much better than what's in the standard lib (urllib2). It supports authentication as well but I skipped it for brevity here.

import requests
with open('./x.png', 'rb') as f:
    data = f.read()
res = requests.post(url='http://httpbin.org/post',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

# let's check if what we sent is what we intended to send...
import json
import base64

assert base64.b64decode(res.json()['data'][len('data:application/octet-stream;base64,'):]) == data

UPDATE

To find out why this works with Requests but not with urllib2 we have to examine the difference in what's being sent. To see this I'm sending traffic to http proxy (Fiddler) running on port 8888:

Using Requests

import requests

data = 'test data'
res = requests.post(url='http://localhost:8888',
                    data=data,
                    headers={'Content-Type': 'application/octet-stream'})

we see

POST http://localhost:8888/ HTTP/1.1
Host: localhost:8888
Content-Length: 9
Content-Type: application/octet-stream
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/1.0.4 CPython/2.7.3 Windows/Vista

test data

and using urllib2

import urllib2

data = 'test data'    
req = urllib2.Request('http://localhost:8888', data)
req.add_header('Content-Length', '%d' % len(data))
req.add_header('Content-Type', 'application/octet-stream')
res = urllib2.urlopen(req)

we get

POST http://localhost:8888/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 9
Host: localhost:8888
Content-Type: application/octet-stream
Connection: close
User-Agent: Python-urllib/2.7

test data

I don't see any differences which would warrant different behavior you observe. Having said that it's not uncommon for http servers to inspect User-Agent header and vary behavior based on its value. Try to change headers sent by Requests one by one making them the same as those being sent by urllib2 and see when it stops working.

2 of 4
4

This has nothing to do with a malformed upload. The HTTP error clearly specifies 401 unauthorized, and tells you the CSRF token is invalid. Try sending a valid CSRF token with the upload.

More about csrf tokens here:

What is a CSRF token ? What is its importance and how does it work?

Top answer
1 of 1
15

The curl -d switch sends a POST request, but you are using requests.get() instead, sending a GET request (whose body is ignored).

Make it a POST instead, by using request.post():

import requests
import json

url = "http://cdcnepal.com/Modules/HOmeMoviesLists/WebService2.asmx/GetShowsByDate"
headers = {"content-type": "application/json; charset=UTF-8"}
payload = {"portalId":"1","showDate":"26/05/2014","flag":0,"size":9}
r = requests.post(url, headers=headers, data=json.dumps(payload))
print r.text

You also need to:

  1. not use a list for the content-type header, there is no support for paramaters being specified separately.
  2. Encode your JSON data to a JSON string; requests doesn't do this for you. Instead, a dictionary passed to data is encoded as application/x-www-form-urlencoded data instead.

You can compare the curl command with requests more easily using http://httpbin.org/post:

$ curl http://httpbin.org/post \
>  -H 'Content-Type: application/json; charset=UTF-8' \
>  -d '{"portalId":"1","showDate":"26/05/2014","flag":0,"size":9}'

{
  "args": {},
  "data": "{\"portalId\":\"1\",\"showDate\":\"26/05/2014\",\"flag\":0,\"size\":9}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Connection": "close",
    "Content-Length": "58",
    "Content-Type": "application/json; charset=UTF-8",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.30.0",
    "X-Request-Id": "78d7bb7d-e29b-482b-908a-48d2395a050f"
  },
  "json": {
    "flag": 0,
    "portalId": "1",
    "showDate": "26/05/2014",
    "size": 9
  },
  "origin": "84.92.98.170",
  "url": "http://httpbin.org/post"
}

and

>>> import requests
>>> import json
>>> from pprint import pprint
>>> url = 'http://httpbin.org/post'
>>> headers = {"content-type":"application/json; charset=UTF-8"}
>>> payload = {"portalId":"1","showDate":"26/05/2014","flag":0,"size":9}
>>> r = requests.post(url, headers=headers, data=json.dumps(payload))
>>> pprint(r.json())
{u'args': {},
 u'data': u'{"portalId": "1", "flag": 0, "size": 9, "showDate": "26/05/2014"}',
 u'files': {},
 u'form': {},
 u'headers': {u'Accept': u'*/*',
              u'Accept-Encoding': u'gzip, deflate, compress',
              u'Connection': u'close',
              u'Content-Length': u'65',
              u'Content-Type': u'application/json; charset=UTF-8',
              u'Host': u'httpbin.org',
              u'User-Agent': u'python-requests/2.2.1 CPython/2.7.6 Darwin/13.1.0',
              u'X-Request-Id': u'06d6b542-c279-4898-8701-2c0d502aa36e'},
 u'json': {u'flag': 0,
           u'portalId': u'1',
           u'showDate': u'26/05/2014',
           u'size': 9},
 u'origin': u'84.92.98.170',
 u'url': u'http://httpbin.org/post'}

Both cases show the same json dictionary being returned.

If you are using requests version 2.4.2 or newer, you can also leave the JSON encoding to the library; it'll set the correct Content-Type header too, if you pass in the data to send as the json keyword argument:

import requests

url = "http://cdcnepal.com/Modules/HOmeMoviesLists/WebService2.asmx/GetShowsByDate"
payload = {"portalId":"1","showDate":"26/05/2014","flag":0,"size":9}
r = requests.post(url, json=payload)
print r.text
🌐
PyPI
pypi.org › project › curl2pyreqs
curl2pyreqs
JavaScript is disabled in your browser. Please enable JavaScript to proceed · A required part of this site couldn’t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser
Find elsewhere
🌐
CSDN
devpress.csdn.net › python › 63044f14c67703293080ac25.html
Python POST binary data - DevPress官方社区
August 23, 2022 - I am writing some code to interface with redmine and I need to upload some files as part of the process, but I am not sure how to do a POST request from python containing a binary file. ... curl --data-binary "@image.png" -H "Content-Type: application/octet-stream" -X POST -u login:password http://redmine/uploads.xml
🌐
Stack Overflow
stackoverflow.com › questions › 51145193 › how-to-convert-curl-data-binary-to-python-request
how to convert curl --data-binary to python request? - Stack Overflow
July 3, 2018 - Using a bytes instead of a str for data. ... this curl command works with curl 7.58.0 on my linux, but on windows- cmd command returns the same error as in the pycode without $ ... What does "does not work" mean? What happens? And, given that the headers aren't exactly the same, why are they different, and why are you sure that doesn't matter? ... Anyway, if you want to specify the exact binary bytes, you can use a bytes instead of a str for data, which seems to be what you're asking for.
🌐
ActiveState
code.activestate.com › recipes › 576422-python-http-post-binary-file-upload-with-pycurl
Python HTTP POST binary file upload with pycurl « Python recipes « ActiveState Code
August 14, 2008 - There is a way to upload file with ... worked for me for binary data. I didn't liked/tried solution explained at http://fabien.seisen.org/python/urllib2_multipart.html, so I tried with pycurl (http://pycurl.sourceforge.net/ wrapper for http://curl.haxx.se/libcurl/) because ...
🌐
Stack Overflow
stackoverflow.com › questions › 67081047 › unable-to-upload-binary-data-using-python-requests
curl - Unable to upload binary data using python requests - Stack Overflow
curl --header "Content-Type: application/octet-stream" --request PUT --data-binary @content.tar.gz <upload_url> ... import requests data = open("content.tar.gz", "rb").read() response = requests.put( <upload_url>, headers={"Content-Type": ...
🌐
GitHub
github.com › spulec › uncurl
GitHub - spulec/uncurl: A library to convert curl requests to python-requests.
A library to convert curl requests to python-requests. - spulec/uncurl
Starred by 664 users
Forked by 100 users
Languages   Python 98.7% | Makefile 1.3% | Python 98.7% | Makefile 1.3%
🌐
Stack Overflow
stackoverflow.com › questions › 48783237 › what-is-the-equivalent-of-http-request-using-curl-data-binary-in-pyt
What is the equivalent of HTTP request using "curl ... --data-binary ..." in Python's requests package? - Stack Overflow
February 14, 2018 - What is the equivalent of HTTP request using "curl ... --data-binary ..." in Python's requests package? The HTTP request's 'Content-Type' is 'multipart/form-data; boundary=...'.
🌐
GitHub
github.com › psf › requests › issues › 1266
How to upload binary file using PUT? · Issue #1266 · psf/requests
March 26, 2013 - def filegen(fo): yield fo.read(CHUNK_SIZE) def main(): session = requests.Session() session.auth = HTTPKerberosAuth() upload_headers = {'X-Codesigner-security-token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'} with open(pathname, 'rb') as infile: response = session.put(upload_url, headers=upload_headers, data=filegen(infile)) ... Request headers: { 'Accept-Encoding': 'gzip, deflate, compress', 'X-Codesigner-security-token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Accept': '*/*', 'User-Agent': 'python-requests/1.1.0 CPython/2.7.1 Windows/7', 'Transfer-Encoding': 'chunked', 'Authorization': 'Negotiate <bl
Author   dabono
🌐
Stack Overflow
stackoverflow.com › questions › 48257387 › why-is-my-http-post-binary-file-working-with-python-requests-but-not-with-curl
Why is my HTTP POST (binary file) working with Python Requests but not with cURL? - Stack Overflow
January 15, 2018 - I tried with -F instead of -X POST ... --data-binary already, but I am getting 400 no matter what. Using curlify to extract the cURL command from the PreparedRequest object in Python, I get: curl -X POST \ -H 'Accept-Encoding: gzip, deflate' \ -H 'Accept: */*' \ -H 'Connection: keep-alive' \ -H 'Content-Length: 85705' \ -H 'Content-Type: multipart/form-data; boundary=c732fde3afb641d2ba5010efc59497bd' \ -H 'User-Agent: python-requests/2.18.4' \ ...