You can use the Session object
import requests
headers = {'User-Agent': 'Mozilla/5.0'}
payload = {'username':'niceusername','password':'123456'}
session = requests.Session()
session.post('https://admin.example.com/login.php',headers=headers,data=payload)
# the session instance holds the cookie. So use it to get/post later.
# e.g. session.get('https://example.com/profile')
Answer from atupal on Stack OverflowYou can use the Session object
import requests
headers = {'User-Agent': 'Mozilla/5.0'}
payload = {'username':'niceusername','password':'123456'}
session = requests.Session()
session.post('https://admin.example.com/login.php',headers=headers,data=payload)
# the session instance holds the cookie. So use it to get/post later.
# e.g. session.get('https://example.com/profile')
Send a POST request with content type = 'form-data':
import requests
files = {
'username': (None, 'myusername'),
'password': (None, 'mypassword'),
}
response = requests.post('https://example.com/abc', files=files)
Python Requests - multipart form data
How to send a "multipart/form-data" with requests in python? - Stack Overflow
Help with multipart-formdata - Python Requests code
Python Requests library - post method with form data help
How do I send form data with Python Requests?
How can I upload a file with Python Requests POST?
What is the difference between data and json in Python Requests POST?
Videos
Hello
So I've been working on this project and currently am using the requests library to pull html data from internal webpages. Because they're proprietary I can't share links, but essentially I have to open with a requests Session object, get the first page which is a search bar. In the search bar I need to pass 2 strings, a command and an account number then hit enter.
Looking at the post method that happens after 'enter' it seems the command and account numbers are passed as strings in the form-data of the headers tab. I copied the key value pair that shows the 2 strings as values and created a dictionary to pass them into the post as an argument. looks like:
with requests.Session() as s:
get_holdings = s.get(url, verify=True, auth=auth)
files = {
'field': 'command%account#'
}
get_holdings2 = s.post(posturl, verify=True, auth=auth, data=files)
soup = BeautifulSoup(get_holdings2.content, 'html5lib')However, the post method does not seem to be passing the form-data to get the information back from the page and I'm just getting the original "get_holdings" page. I may be using the subsequent post incorrectly or not navigating right. I also thought I may need to pass the ENTIRE form-data contents into requests (which is pretty big) . If any one has some advice on navigating through webpages using requests that would be super helpful, a lot of the data I need comes from webpages that are very interactive, lots of clicking through links to get to what you're looking for. This is a pretty new library for me.
I am new to python requests and python in general - I have a somewhat intermediate java background.
I am trying to figure out why this post request throws a 400 error when trying to upload a file.
Any hints?
def upload_a_document(self, library_id, folder_id, filepath):
url = "https://xxx" \
"/{library_id}!{folder_id}/documents".format(
server=self.server,
customer_id=self.customer_id,
library_id=library_id,
folder_id=folder_id)
params = \
{
"profile":
{
"doc_profile": {
"access": "full_access",
"comment": f"{filepath}",
"database": f"{library_id}",
"default_security": "public",
"name": f"{os.path.basename(filepath)}",
"size": os.path.getsize(filepath),
"type": "WORDX",
"type_description": "WORD 2010",
"wstype": "document"
},
"warnings_for_required_and_disabled_fields": True
},
"file": {open(f"{filepath}", "rb").read()}
}
payload = ""
# encoded formdata for multipart
encoded_formdata = encode_multipart_formdata(params)
headers = {'X-Auth-Token': self.x_auth_token, 'Content-Type': encoded_formdata[1]}
response = requests.request("POST", url, data=payload, headers=headers, params=params)
myJSON = response.json()
print(json.dumps(myJSON, indent=4))Getting this error -
requests.exceptions.JSONDecodeError: [Errno Expecting value] <html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
Basically, if you specify a files parameter (a dictionary), then requests will send a multipart/form-data POST instead of a application/x-www-form-urlencoded POST. You are not limited to using actual files in that dictionary, however:
>>> import requests
>>> response = requests.post('http://httpbin.org/post', files=dict(foo='bar'))
>>> response.status_code
200
and httpbin.org lets you know what headers you posted with; in response.json() we have:
>>> from pprint import pprint
>>> pprint(response.json()['headers'])
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'close',
'Content-Length': '141',
'Content-Type': 'multipart/form-data; '
'boundary=c7cbfdd911b4e720f1dd8f479c50bc7f',
'Host': 'httpbin.org',
'User-Agent': 'python-requests/2.21.0'}
And just to be explicit: you should not set the Content-Type header when you use the files parameter, leave this to requests because it needs to specify a (unique) boundary value in the header that matches the value used in the request body.
Better still, you can further control the filename, content type and additional headers for each part by using a tuple instead of a single string or bytes object. The tuple is expected to contain between 2 and 4 elements; the filename, the content, optionally a content type, and an optional dictionary of further headers.
I'd use the tuple form with None as the filename, so that the filename="..." parameter is dropped from the request for those parts:
>>> files = {'foo': 'bar'}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--bb3f05a247b43eede27a124ef8b968c5
Content-Disposition: form-data; name="foo"; filename="foo"
bar
--bb3f05a247b43eede27a124ef8b968c5--
>>> files = {'foo': (None, 'bar')}
>>> print(requests.Request('POST', 'http://httpbin.org/post', files=files).prepare().body.decode('utf8'))
--d5ca8c90a869c5ae31f70fa3ddb23c76
Content-Disposition: form-data; name="foo"
bar
--d5ca8c90a869c5ae31f70fa3ddb23c76--
files can also be a list of two-value tuples, if you need ordering and/or multiple fields with the same name:
requests.post(
'http://requestb.in/xucj9exu',
files=(
('foo', (None, 'bar')),
('foo', (None, 'baz')),
('spam', (None, 'eggs')),
)
)
If you specify both files and data, then it depends on the value of data what will be used to create the POST body. If data is a string, only it willl be used; otherwise both data and files are used, with the elements in data listed first.
There is also the excellent requests-toolbelt project, which includes advanced Multipart support. It takes field definitions in the same format as the files parameter, but unlike requests, it defaults to not setting a filename parameter. In addition, it can stream the request from open file objects, where requests will first construct the request body in memory:
from requests_toolbelt.multipart.encoder import MultipartEncoder
mp_encoder = MultipartEncoder(
fields={
'foo': 'bar',
# plain file object, no filename or mime type produces a
# Content-Disposition header with just the part name
'spam': ('spam.txt', open('spam.txt', 'rb'), 'text/plain'),
}
)
r = requests.post(
'http://httpbin.org/post',
data=mp_encoder, # The MultipartEncoder is posted as data, don't use files=...!
# The MultipartEncoder provides the content-type header with the boundary:
headers={'Content-Type': mp_encoder.content_type}
)
Fields follow the same conventions; use a tuple with between 2 and 4 elements to add a filename, part mime-type or extra headers. Unlike the files parameter, no attempt is made to find a default filename value if you don't use a tuple.
Requests has changed since some of the previous answers were written. Have a look at this Issue on Github for more details and this comment for an example.
In short, the files parameter takes a dictionary with the key being the name of the form field and the value being either a string or a 2, 3 or 4-length tuple, as described in the section POST a Multipart-Encoded File in the Requests quickstart:
>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
In the above, the tuple is composed as follows:
(filename, data, content_type, headers)
If the value is just a string, the filename will be the same as the key, as in the following:
>>> files = {'obvius_session_id': '72c2b6f406cdabd578c5fd7598557c52'}
Content-Disposition: form-data; name="obvius_session_id"; filename="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52
If the value is a tuple and the first entry is None the filename property will not be included:
>>> files = {'obvius_session_id': (None, '72c2b6f406cdabd578c5fd7598557c52')}
Content-Disposition: form-data; name="obvius_session_id"
Content-Type: application/octet-stream
72c2b6f406cdabd578c5fd7598557c52