Basically, if you specify a files parameter (a dictionary), then requests will send a multipart/form-data POST instead of a application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
>>> import requests
>>> response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
>>> response.status_code
200
and httpbin.org lets you know what headers you posted with; in response.json() we have:
>>> from pprint import pprint
>>> pprint(response.json()['headers'])
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'close',
'Content-Length': '141',
'Content-Type': 'multipart/form-data; '
'boundary=c7cbfdd911b4e720f1dd8f479c50bc7f',
'Host': 'httpbin.org',
'User-Agent': 'python-requests/2.21.0'}
And just to be explicit: you should not set the Content-Type header when you use the files parameter, leave this to requests because it needs to specify a (unique) boundary value in the header that matches the value used in the request body.
Better still, you can further control the filename, content type and additional headers for each part by using a tuple instead of a single string or bytes object. The tuple is expected to contain between 2 and 4 elements; the filename, the content, optionally a content type, and an optional dictionary of further headers.
I'd use the tuple form with None as the filename, so that the filename="..." parameter is dropped from the request for those parts:
>>> files = {'foo': 'bar'}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--bb3f05a247b43eede27a124ef8b968c5
Content-Disposition: form-data; name="foo"; filename="foo"
bar
--bb3f05a247b43eede27a124ef8b968c5--
>>> files = {'foo': (None, 'bar')}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--d5ca8c90a869c5ae31f70fa3ddb23c76
Content-Disposition: form-data; name="foo"
bar
--d5ca8c90a869c5ae31f70fa3ddb23c76--
files can also be a list of two-value tuples, if you need ordering and/or multiple fields with the same name:
requests.post(
'http://requestb.in/xucj9exu',
files=(
('foo', (None, 'bar')),
('foo', (None, 'baz')),
('spam', (None, 'eggs')),
)
)
If you specify both files and data, then it depends on the value of data what will be used to create the POST body. If data is a string, only it willl be used; otherwise both data and files are used, with the elements in data listed first.
There is also the excellent requests-toolbelt project, which includes advanced Multipart support. It takes field definitions in the same format as the files parameter, but unlike requests, it defaults to not setting a filename parameter. In addition, it can stream the request from open file objects, where requests will first construct the request body in memory:
from requests_toolbelt.multipart.encoder import MultipartEncoder
mp_encoder = MultipartEncoder(
fields={
'foo': 'bar',
# plain file object, no filename or mime type produces a
# Content-Disposition header with just the part name
'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
r = requests.post(
'http://httpbin.org/post',
data=mp_encoder, # The MultipartEncoder is posted as data, don't use files=...!
# The MultipartEncoder provides the content-type header with the boundary:
headers={'Content-Type': mp_encoder.content_type}
)
Fields follow the same conventions; use a tuple with between 2 and 4 elements to add a filename, part mime-type or extra headers. Unlike the files parameter, no attempt is made to find a default filename value if you don't use a tuple.
Basically, if you specify a files parameter (a dictionary), then requests will send a multipart/form-data POST instead of a application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
>>> import requests
>>> response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
>>> response.status_code
200
and httpbin.org lets you know what headers you posted with; in response.json() we have:
>>> from pprint import pprint
>>> pprint(response.json()['headers'])
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'close',
'Content-Length': '141',
'Content-Type': 'multipart/form-data; '
'boundary=c7cbfdd911b4e720f1dd8f479c50bc7f',
'Host': 'httpbin.org',
'User-Agent': 'python-requests/2.21.0'}
And just to be explicit: you should not set the Content-Type header when you use the files parameter, leave this to requests because it needs to specify a (unique) boundary value in the header that matches the value used in the request body.
Better still, you can further control the filename, content type and additional headers for each part by using a tuple instead of a single string or bytes object. The tuple is expected to contain between 2 and 4 elements; the filename, the content, optionally a content type, and an optional dictionary of further headers.
I'd use the tuple form with None as the filename, so that the filename="..." parameter is dropped from the request for those parts:
>>> files = {'foo': 'bar'}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--bb3f05a247b43eede27a124ef8b968c5
Content-Disposition: form-data; name="foo"; filename="foo"
bar
--bb3f05a247b43eede27a124ef8b968c5--
>>> files = {'foo': (None, 'bar')}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--d5ca8c90a869c5ae31f70fa3ddb23c76
Content-Disposition: form-data; name="foo"
bar
--d5ca8c90a869c5ae31f70fa3ddb23c76--
files can also be a list of two-value tuples, if you need ordering and/or multiple fields with the same name:
requests.post(
'http://requestb.in/xucj9exu',
files=(
('foo', (None, 'bar')),
('foo', (None, 'baz')),
('spam', (None, 'eggs')),
)
)
If you specify both files and data, then it depends on the value of data what will be used to create the POST body. If data is a string, only it willl be used; otherwise both data and files are used, with the elements in data listed first.
There is also the excellent requests-toolbelt project, which includes advanced Multipart support. It takes field definitions in the same format as the files parameter, but unlike requests, it defaults to not setting a filename parameter. In addition, it can stream the request from open file objects, where requests will first construct the request body in memory:
from requests_toolbelt.multipart.encoder import MultipartEncoder
mp_encoder = MultipartEncoder(
fields={
'foo': 'bar',
# plain file object, no filename or mime type produces a
# Content-Disposition header with just the part name
'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
r = requests.post(
'http://httpbin.org/post',
data=mp_encoder, # The MultipartEncoder is posted as data, don't use files=...!
# The MultipartEncoder provides the content-type header with the boundary:
headers={'Content-Type': mp_encoder.content_type}
)
Fields follow the same conventions; use a tuple with between 2 and 4 elements to add a filename, part mime-type or extra headers. Unlike the files parameter, no attempt is made to find a default filename value if you don't use a tuple.
Requests has changed since some of the previous answers were written. Have a look at this Issue on Github for more details and this comment for an example.
In short, the files parameter takes a dictionary with the key being the name of the form field and the value being either a string or a 2, 3 or 4-length tuple, as described in the section POST a Multipart-Encoded File in the Requests quickstart:
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
In the above, the tuple is composed as follows:
(filename, data, content_type, headers)
If the value is just a string, the filename will be the same as the key, as in the following:
>>> files = {'obvius_session_id': '72c2b6f406cdabd578c5fd7598557c52'}
Content-Disposition: form-data; name="obvius_session_id"; filename="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
If the value is a tuple and the first entry is None the filename property will not be included:
>>> files = {'obvius_session_id': (None, '72c2b6f406cdabd578c5fd7598557c52')}
Content-Disposition: form-data; name="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
I am new to python requests and python in general - I have a somewhat intermediate java background.
I am trying to figure out why this post request throws a 400 error when trying to upload a file.
Any hints?
def upload_a_document(self, library_id, folder_id, filepath):
url = "https://xxx" \
"/{library_id}!{folder_id}/documents".format(
server=self.server,
customer_id=self.customer_id,
library_id=library_id,
folder_id=folder_id)
params = \
{
"profile":
{
"doc_profile": {
"access": "full_access",
"comment": f"{filepath}",
"database": f"{library_id}",
"default_security": "public",
"name": f"{os.path.basename(filepath)}",
"size": os.path.getsize(filepath),
"type": "WORDX",
"type_description": "WORD 2010",
"wstype": "document"
},
"warnings_for_required_and_disabled_fields": True
},
"file": {open(f"{filepath}", "rb").read()}
}
payload = ""
# encoded formdata for multipart
encoded_formdata = encode_multipart_formdata(params)
headers = {'X-Auth-Token': self.x_auth_token, 'Content-Type': encoded_formdata[1]}
response = requests.request("POST", url, data=payload, headers=headers, params=params)
myJSON = response.json()
print(json.dumps(myJSON, indent=4))Getting this error -
requests.exceptions.JSONDecodeError: [Errno Expecting value] <html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
Help with multipart-formdata - Python Requests code
requests with multiparts using data and files doesn't have content-type in data part
How to correctly post json as multipart form data with python requests - Stack Overflow
Custom multipart/form-data Content-Type header silently discards file upload
Videos
See this thread How to send JSON as part of multipart POST-request
Do not set the Content-type header yourself, leave that to pyrequests to generate
def send_request():
payload = {"param_1": "value_1", "param_2": "value_2"}
files = {
'json': (None, json.dumps(payload), 'application/json'),
'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream')
}
r = requests.post(url, files=files)
print(r.content)
Don't encode using json.
import requests
info = {
'var1' : 'this',
'var2' : 'that',
}
data = {
'token' : auth_token,
'info' : info,
}
headers = {'Content-type': 'multipart/form-data'}
files = {'document': open('file_name.pdf', 'rb')}
r = requests.post(url, files=files, data=data, headers=headers)
Note that this may not necessarily be what you want, as it will become another form-data section.
You are setting the header yourself, including a boundary. Don't do this; requests generates a boundary for you and sets it in the header, but if you already set the header then the resulting payload and the header will not match. Just drop you headers altogether:
def send_request():
payload = {"param_1": "value_1", "param_2": "value_2"}
files = {
'json': (None, json.dumps(payload), 'application/json'),
'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream')
}
r = requests.post(url, files=files)
print(r.content)
Note that I also gave the file part a filename (the base name of the file path`).
For more information on multi-part POST requests, see the advanced section of the documentation.
In case if someone searches ready to use method to transform python dicts to multipart-form data structures here is a simple gist example to do such transformation:
{"some": ["balls", "toys"], "field": "value", "nested": {"objects": "here"}}
->
{"some[0]": "balls", "some[1]": "toys", "field": "value", "nested[objects]": "here"}
To send some data you may want to use the multipartify method from this gist like this:
import requests # library for making requests
payload = {
"person": {"name": "John", "age": "31"},
"pets": ["Dog", "Parrot"],
"special_mark": 42,
} # Example payload
requests.post("https://example.com/", files=multipartify(payload))
To send same data along with any file (as OP wanted) you may simply add it like this:
converted_data = multipartify(payload)
converted_data["attachment[0]"] = ("file.png", b'binary-file', "image/png")
requests.post("https://example.com/", files=converted_data)
Note, that attachment is a name defined by server endpoint and may vary. Also attachment[0] indicates that it is first file in you request - this is also should be defined by API documentation.