You are looking for the Response.reason attribute:
Copy>>> import requests
>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200
>>> r.reason
'OK'
>>> r = requests.get('http://httpbin.org/status/500')
>>> r.reason
'INTERNAL SERVER ERROR'
Answer from Martijn Pieters on Stack OverflowGet HTTP response header using Python requests module - Stack Overflow
How to print out http-response header in Python - Stack Overflow
Using headers with the Python 'Requests' library's get() method - Stack Overflow
How to fetch response headers using Python Requests Library?
How do I rotate headers to avoid detection?
Maintain a list of realistic header sets and randomly select one per request. Vary the User-Agent, Accept-Language, and Referer values. Combine this with proxy rotation for better results.
When should I use browser automation instead of requests?
Use browser automation tools like Playwright or Selenium when you need to load JavaScript, handle clicks, or when you're blocked by TLS fingerprinting. While these tools solve the TLS problem, they are slower and use more computer resources. ScrapFly offers both simple requests and full browser automation.
Why do my headers work in testing but fail in production?
This is usually because you're sending too many requests. In testing, your low number of requests doesn't trigger extra anti-bot checks. In production, more requests trigger TLS fingerprinting. Even with perfect headers, a different TLS fingerprint will get you blocked. ScrapFly solves this by using real browser TLS profiles.
Videos
You are looking for the Response.reason attribute:
Copy>>> import requests
>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200
>>> r.reason
'OK'
>>> r = requests.get('http://httpbin.org/status/500')
>>> r.reason
'INTERNAL SERVER ERROR'
That is an excellent answer BUT please keep in mind that for certain applications, you need to retrieve the response headers. This is often the case in paginated REST apis. Those can be retrieved with:
Copyr.headers
And iterate the keys with:
Copy[x for x in r.headers]
Happy coding! [R]
Update: Based on comment of OP, that only the response headers are needed. Even more easy as written in below documentation of Requests module:
We can view the server's response headers using a Python dictionary:
>>> r.headers
{
'content-encoding': 'gzip',
'transfer-encoding': 'chunked',
'connection': 'close',
'server': 'nginx/1.0.4',
'x-runtime': '148ms',
'etag': '"e1ca502697e5c9317743dc078f67693f"',
'content-type': 'application/json'
}
And especially the documentation notes:
The dictionary is special, though: it's made just for HTTP headers. According to RFC 7230, HTTP Header names are case-insensitive.
So, we can access the headers using any capitalization we want:
and goes on to explain even more cleverness concerning RFC compliance.
The Requests documentation states:
Using Response.iter_content will handle a lot of what you would otherwise have to handle when using Response.raw directly. When streaming a download, the above is the preferred and recommended way to retrieve the content.
It offers as example:
>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
But also offers advice on how to do it in practice by redirecting to a file etc. and using a different method:
Using Response.iter_content will handle a lot of what you would otherwise have to handle when using Response.raw directly
Here's how you get just the response headers using the requests library like you mentioned (implementation in Python3):
import requests
url = "https://www.google.com"
response = requests.head(url)
print(response.headers) # prints the entire header as a dictionary
print(response.headers["Content-Length"]) # prints a specific section of the dictionary
It's important to use .head() instead of .get() otherwise you will retrieve the whole file/page like the rest of the answers mentioned.
If you wish to retrieve a URL that requires authentication you can replace the above response with this:
response = requests.head(url, auth=requests.auth.HTTPBasicAuth(username, password))
According to the API, the headers can all be passed in with requests.get():
import requests
r = requests.get("http://www.example.com/", headers={"Content-Type":"text"})
This answer taught me that you can set headers for an entire session:
s = requests.Session()
s.auth = ('user', 'pass')
s.headers.update({'x-test': 'true'})
# Both 'x-test' and 'x-test2' are sent
s.get('http://httpbin.org/headers', headers={'x-test2': 'true'})
Bonus: Sessions also handle cookies
I am not able to get Response Headers using the Python requests module.
The method ".header" returns requests headers not response headers.
I want to get location, when I inspect the same in browser, it shows under Response Headers.