As far as requests is concerned, there is no difference between a zip file and any other binary blob of data.
Your server is broken here; it is cutting of the connection when you send it a zip file. That is not something requests can do anything about.
You may want to test against http://httpbin.org/ when you run into problems like these; it is a testing service built by the author of the requests library.
Another tip: you don't need to read the whole file object into memory when sending. Just pass the object itself to requests instead:
fileobj = open('/Users/.../test.zip', 'rb')
r = requests.post(url, auth=HTTPDigestAuth('dev', 'dev'), data = {"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})
Demo against httpbin.org:
>>> import requests
>>> fileobj = open('/tmp/test.zip', 'rb')
>>> r = requests.post('http://httpbin.org/post', data={"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})
>>> r
<Response [200]>
>>> r.json()
{u'origin': u'217.32.203.188', u'files': {u'archive': u'data:application/zip;base64,<long base64 body omitted>'}, u'form': {u'mysubmit': u'Go'}, u'url': u'http://httpbin.org/post', u'args': {}, u'headers': {u'Content-Length': u'57008', u'Accept-Encoding': u'gzip, deflate, compress', u'Connection': u'close', u'Accept': u'*/*', u'User-Agent': u'python-requests/1.2.3 CPython/2.7.5 Darwin/12.4.0', u'Host': u'httpbin.org', u'Content-Type': u'multipart/form-data; boundary=9aec1d03a1794177a38b48416dd4c811'}, u'json': None, u'data': u''}
Answer from Martijn Pieters on Stack OverflowI all,
I'm having a headache trying to upload a zipfile to my pydio webserver using this api endpoint
The zipfile is generated by my python script and can be extracted on the source machine.
The source machine is an old Windows XP running the latest release of python 2.7.
When uploading the zip file, the file on the destination server contains the following text at the top :
--9efb552ea5024e6eacabbf15ec02376a
Content-Disposition: form-data; name="archive"; filename="b1ch102.zip"
Content-Type: application/zip
This text is not present on the source file. Of course, the zip file can't be extracted once modified like this.
Here is the code i'm running :
with open(zipname, 'rb') as f:
response = requests.put('https://myserver.com/api/my-files/upload/',
auth=(user, pwd),
files={'archive': (zipname, f, 'application/zip')},
headers={'X-File-Name' : zipname})
I'm trying for 2 days and can't figure out how to make this working.
Any help would be appreciated.
edit: formating
As far as requests is concerned, there is no difference between a zip file and any other binary blob of data.
Your server is broken here; it is cutting of the connection when you send it a zip file. That is not something requests can do anything about.
You may want to test against http://httpbin.org/ when you run into problems like these; it is a testing service built by the author of the requests library.
Another tip: you don't need to read the whole file object into memory when sending. Just pass the object itself to requests instead:
fileobj = open('/Users/.../test.zip', 'rb')
r = requests.post(url, auth=HTTPDigestAuth('dev', 'dev'), data = {"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})
Demo against httpbin.org:
>>> import requests
>>> fileobj = open('/tmp/test.zip', 'rb')
>>> r = requests.post('http://httpbin.org/post', data={"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})
>>> r
<Response [200]>
>>> r.json()
{u'origin': u'217.32.203.188', u'files': {u'archive': u'data:application/zip;base64,<long base64 body omitted>'}, u'form': {u'mysubmit': u'Go'}, u'url': u'http://httpbin.org/post', u'args': {}, u'headers': {u'Content-Length': u'57008', u'Accept-Encoding': u'gzip, deflate, compress', u'Connection': u'close', u'Accept': u'*/*', u'User-Agent': u'python-requests/1.2.3 CPython/2.7.5 Darwin/12.4.0', u'Host': u'httpbin.org', u'Content-Type': u'multipart/form-data; boundary=9aec1d03a1794177a38b48416dd4c811'}, u'json': None, u'data': u''}
If you are facing any errors while uploading the zip: This problem could be caused by 'Content-Type': 'multipart/form-data' setting on the header. The problem can be solved by deleting this setting, as in this example:
header = {'Content-Type': 'multipart/form-data', 'Authorization': 'Bearer {}'.format(bearerToken)}
change it to:
header = {'Authorization': 'Bearer {}'.format(bearerToken)}
Code which worked for me:
header = {"INFA-SESSION-ID":self._v3SessionID}
files = {'package': ("response.zip", open("C:\Users\myUser\Downloads\response.zip", 'rb'),'application/zip')}
response = re.post(url, headers=header, files=files)
PUT (requests.put) specifying the source file location for zip file error 10054
Using request package: Requested zipfiles are not opening and scripts are getting downloaded as HTML
POST zip file to ArcGIS Server using an API (Python) - Geographic Information Systems Stack Exchange
python - Download Returned Zip file from URL - Stack Overflow
may be this will help you.
with open(zipname, 'rb') as f:
uploadbase = requests.put('url',
auth=(base, pwd),
data=f,
headers={'X-File-Name' : zipname,
'Content-Disposition': 'form-data; name="{0}"; filename="{0}"'.format(zipname),
'content-type': 'multipart/form-data'})
the difference between put and post
curl -T ... uses PUT method instead of POST.
As the error message says, you should use
r = requests.put(url, ...)
As far as I can tell, the proper way to do this in Python 2 is:
import requests, zipfile, StringIO
r = requests.get(zip_file_url, stream=True)
z = zipfile.ZipFile(StringIO.StringIO(r.content))
z.extractall()
of course you'd want to check that the GET was successful with r.ok.
For python 3+, sub the StringIO module with the io module and use BytesIO instead of StringIO: Here are release notes that mention this change.
import requests, zipfile, io
r = requests.get(zip_file_url)
z = zipfile.ZipFile(io.BytesIO(r.content))
z.extractall("/path/to/destination_directory")
Most people recommend using requests if it is available, and the requests documentation recommends this for downloading and saving raw data from a url:
import requests
def download_url(url, save_path, chunk_size=128):
r = requests.get(url, stream=True)
with open(save_path, 'wb') as fd:
for chunk in r.iter_content(chunk_size=chunk_size):
fd.write(chunk)
Since the answer asks about downloading and saving the zip file, I haven't gone into details regarding reading the zip file. See one of the many answers below for possibilities.
If for some reason you don't have access to requests, you can use urllib.request instead. It may not be quite as robust as the above.
import urllib.request
def download_url(url, save_path):
with urllib.request.urlopen(url) as dl_file:
with open(save_path, 'wb') as out_file:
out_file.write(dl_file.read())
Finally, if you are using Python 2 still, you can use urllib2.urlopen.
from contextlib import closing
def download_url(url, save_path):
with closing(urllib2.urlopen(url)) as dl_file:
with open(save_path, 'wb') as out_file:
out_file.write(dl_file.read())