I know it's been said already, but I'd highly recommend the requests Python package.
If you've used languages other than python, you're probably thinking urllib and urllib2 are easy to use, not much code, and highly capable, that's how I used to think. But the requests package is so unbelievably useful and short that everyone should be using it.
First, it supports a fully restful API, and is as easy as:
import requests
resp = requests.get('http://www.mywebsite.com/user')
resp = requests.post('http://www.mywebsite.com/user')
resp = requests.put('http://www.mywebsite.com/user/put')
resp = requests.delete('http://www.mywebsite.com/user/delete')
Regardless of whether GET / POST, you never have to encode parameters again, it simply takes a dictionary as an argument and is good to go:
userdata = {"firstname": "John", "lastname": "Doe", "password": "jdoe123"}
resp = requests.post('http://www.mywebsite.com/user', data=userdata)
Plus it even has a built in JSON decoder (again, I know json.loads() isn't a lot more to write, but this sure is convenient):
resp.json()
Or if your response data is just text, use:
resp.text
This is just the tip of the iceberg. This is the list of features from the requests site:
- International Domains and URLs
- Keep-Alive & Connection Pooling
- Sessions with Cookie Persistence
- Browser-style SSL Verification
- Basic/Digest Authentication
- Elegant Key/Value Cookies
- Automatic Decompression
- Unicode Response Bodies
- Multipart File Uploads
- Connection Timeouts
- .netrc support
- List item
- Python 3.10—3.14
- Thread-safe.
I know it's been said already, but I'd highly recommend the requests Python package.
If you've used languages other than python, you're probably thinking urllib and urllib2 are easy to use, not much code, and highly capable, that's how I used to think. But the requests package is so unbelievably useful and short that everyone should be using it.
First, it supports a fully restful API, and is as easy as:
import requests
resp = requests.get('http://www.mywebsite.com/user')
resp = requests.post('http://www.mywebsite.com/user')
resp = requests.put('http://www.mywebsite.com/user/put')
resp = requests.delete('http://www.mywebsite.com/user/delete')
Regardless of whether GET / POST, you never have to encode parameters again, it simply takes a dictionary as an argument and is good to go:
userdata = {"firstname": "John", "lastname": "Doe", "password": "jdoe123"}
resp = requests.post('http://www.mywebsite.com/user', data=userdata)
Plus it even has a built in JSON decoder (again, I know json.loads() isn't a lot more to write, but this sure is convenient):
resp.json()
Or if your response data is just text, use:
resp.text
This is just the tip of the iceberg. This is the list of features from the requests site:
- International Domains and URLs
- Keep-Alive & Connection Pooling
- Sessions with Cookie Persistence
- Browser-style SSL Verification
- Basic/Digest Authentication
- Elegant Key/Value Cookies
- Automatic Decompression
- Unicode Response Bodies
- Multipart File Uploads
- Connection Timeouts
- .netrc support
- List item
- Python 3.10—3.14
- Thread-safe.
In the Python 2 standard library there were two HTTP libraries that existed side-by-side. Despite the similar name, they were unrelated: they had a different design and a different implementation.
urllibwas the original Python HTTP client, added to the standard library in Python 1.2. Earlier documentation forurllibcan be found in Python 1.4.urllib2was a more capable HTTP client, added in Python 1.6, intended as a replacement forurllib:urllib2 - new and improved but incompatible version of urllib (still experimental).
Earlier documentation for
urllib2can be found in Python 2.1.
The Python 3 standard library has a new urllib which is a merged/refactored/rewritten version of the older modules.
urllib3 is a third-party package (i.e., not in CPython's standard library). Despite the name, it is unrelated to the standard library packages, and there is no intention to include it in the standard library in the future.
Finally, requests internally uses urllib3, but it aims for an easier-to-use API.
AskReddit: Why use requests as opposed to urllib3?
It’s a wrapper for many things you want to do with urllib3 and other libraries like ssl.
This is just an abstraction library for ease.
More on reddit.comwhat is the official relation between urllib3 and upstream python?
What is the difference between urllib and python requests library?
Requests vs. urllib: What problem does it solve?
Videos
I am genuinely curious why don't we use urllib3 which requests itself uses? I know it provides easy request response wrappers but I had issues with exception handling in requests quite often. So what do we loose if we use urllib3 as opposed to requests functionality wise ?
It’s a wrapper for many things you want to do with urllib3 and other libraries like ssl.
This is just an abstraction library for ease.
You have to do slightly more work, and in return have much more control. But I get downvoted in this sub for pointing out that urllib is more powerful than requests, for some reason.