Brave Search

What exceptions can be raised by Python HTTPX's json() method?

stackoverflow.com › questions › 75773019 › what-exceptions-can-be-raised-by-python-httpxs-json-method

even looking through the code, it is not obvious to me what the possible exceptions might be.

Here's my attempt at looking through it:

def json(self, **kwargs: typing.Any) -> typing.Any:
    if self.charset_encoding is None and self.content and len(self.content) > 3:
        encoding = guess_json_utf(self.content)
        if encoding is not None:
            return jsonlib.loads(self.content.decode(encoding), **kwargs)
    return jsonlib.loads(self.text, **kwargs)

self.content can raise ResponseNotRead if (as it sounds like) the response has not been read yet. However, this is almost certainly due to a logical error in the code rather than any meaningful problem at runtime worth detecting; so there is no good reason to catch this. (It also wouldn't happen with the most straightforward use cases, such as the one shown in the documentation.) Otherwise, self.content will be a bytes, so len will work.
self.charset_encoding will either return None (if there is no corresponding data in the response header) or else eventually use the standard library email.message.Message.get_content_charset to parse a content type from the response header. The latter is not documented to raise any exceptions; so there should not be any exception from accessing this value.
guess_json_utf will necessarily be passed a bytes that is at least 4 bytes long. Nothing in its logic should be able to fail under those conditions. Either None or a string will be returned.
If jsonlib.loads is called using self.content.decode(encoding), then encoding was necessarily not None, and is a valid encoding name returned from guess_json_utf However, it's possible that the self.content (bytes data) is not valid data (i.e., convertible to text using the guessed text encoding). This would cause UnicodeDecodeError.
Otherwise, it's called with self.text. If the underlying _text was set before, it will be a string and gets returned. If there is no content in the response, an empty string is returned. Otherwise, decoding is attempted using a default text encoder with a "replace" error-handling policy. There again shouldn't be anything that could cause the text encoding name to be invalid, so again this can only raise UnicodeDecodeError, and even that shouldn't be possible with the "replace" error-handling policy.
Finally: jsonlib.loads is simply json.loads (i.e., the standard library json). if the code gets this far, json.loads will definitely be given a string to load; all possible issues with the string contents will be reported as JSONDecodeError.

tl;dr: the possible exceptions that make sense to catch are JSONDecodeError (the response is not valid JSON - this includes e.g. an empty response) and UnicodeDecodeError (the response is corrupt in some way, for example it mixes bytes intended to encode text in two different ways, or it's supposed to be UTF-8 but contains bytes that are illegal in that encoding; or it's encoded using a non-UTF scheme, such as Latin-1, in a way that is incompatible with the corresponding guessed UTF scheme, and doesn't advertise the encoding in the header).

Answer from Karl Knechtel on Stack Overflow

Stack Overflow

stackoverflow.com › questions › 75773019 › what-exceptions-can-be-raised-by-python-httpxs-json-method

What exceptions can be raised by Python HTTPX's json() method? - Stack Overflow

Top answer

1 of 1

even looking through the code, it is not obvious to me what the possible exceptions might be.

Here's my attempt at looking through it:

def json(self, **kwargs: typing.Any) -> typing.Any:
    if self.charset_encoding is None and self.content and len(self.content) > 3:
        encoding = guess_json_utf(self.content)
        if encoding is not None:
            return jsonlib.loads(self.content.decode(encoding), **kwargs)
    return jsonlib.loads(self.text, **kwargs)

self.content can raise ResponseNotRead if (as it sounds like) the response has not been read yet. However, this is almost certainly due to a logical error in the code rather than any meaningful problem at runtime worth detecting; so there is no good reason to catch this. (It also wouldn't happen with the most straightforward use cases, such as the one shown in the documentation.) Otherwise, self.content will be a bytes, so len will work.
self.charset_encoding will either return None (if there is no corresponding data in the response header) or else eventually use the standard library email.message.Message.get_content_charset to parse a content type from the response header. The latter is not documented to raise any exceptions; so there should not be any exception from accessing this value.
guess_json_utf will necessarily be passed a bytes that is at least 4 bytes long. Nothing in its logic should be able to fail under those conditions. Either None or a string will be returned.
If jsonlib.loads is called using self.content.decode(encoding), then encoding was necessarily not None, and is a valid encoding name returned from guess_json_utf However, it's possible that the self.content (bytes data) is not valid data (i.e., convertible to text using the guessed text encoding). This would cause UnicodeDecodeError.
Otherwise, it's called with self.text. If the underlying _text was set before, it will be a string and gets returned. If there is no content in the response, an empty string is returned. Otherwise, decoding is attempted using a default text encoder with a "replace" error-handling policy. There again shouldn't be anything that could cause the text encoding name to be invalid, so again this can only raise UnicodeDecodeError, and even that shouldn't be possible with the "replace" error-handling policy.
Finally: jsonlib.loads is simply json.loads (i.e., the standard library json). if the code gets this far, json.loads will definitely be given a string to load; all possible issues with the string contents will be reported as JSONDecodeError.

GitHub

github.com › Colin-b › httpx_auth › issues › 92

JSONDecodeError due to Improper Handling of Nested JSON Strings in JWT Payloads · Issue #92 · Colin-b/httpx_auth

April 17, 2024 - json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 12 (char 11) This error is caused by the way double quotes inside the nested JSON are handled, which corrupts the JSON string during the base64 decoding step.

Author pythrick

Discussions

Custom JSON library

Hi! Is there a way to use an alternative JSON library to decode the request? Like orjson for example? Thanks! More on github.com

github.com

November 26, 2019

Randomly receiving json.decoder.JSONDecodeError error when using requests.get. Completely out of ideas

check the returned content. Decoder is complaining it has to work with None (?) /gets no value. More on reddit.com

r/learnpython

September 10, 2016

python - JSONDecodeError: Expecting value: line 1 column 1 (char 0) - Stack Overflow

I am getting error Expecting value: line 1 column 1 (char 0) when trying to decode JSON. The URL I use for the API call works fine in the browser, but gives this error when done through a curl requ... More on stackoverflow.com

stackoverflow.com

python requests randomly breaks with JSONDecodeError - Stack Overflow

I have been debugging for hours why my code randomly breaks with this error: JSONDecodeError: Expecting value: line 1 column 1 (char 0) This is the code I have: while True: try: More on stackoverflow.com

stackoverflow.com

GitHub

github.com › encode › httpx › discussions › 2695

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) · encode/httpx · Discussion #2695

I have a script import httpx url = f"https://hn.algolia.com/api/v1/users/morenoh149" response = httpx.get(url) data = response.json() I get this error python try_lib.py Traceback (most recent call ...

Author encode

GitHub

github.com › encode › httpx › issues › 717

Custom JSON library · Issue #717 · encode/httpx

November 26, 2019 - Hi! Is there a way to use an alternative JSON library to decode the request? Like orjson for example? Thanks!

Author victoraugustolls

GitHub

github.com › encode › httpx › discussions › 2320

Parameter quoting in requests · encode/httpx · Discussion #2320

params = {"filter[leaf]":{"op":"eq", "value":"true"}} request = httpx.Request("GET", "/my/url", params=params) When I look at the URL, I get this: /my/url?filter[leaf]={'op':+'eq',+'value':+'true'} Which decodes to this: /my/url?filter[leaf]={'op':+'eq',+'value':+'true'} Now the params in the URL are enclosed in single quotes and not in double quotes, which leads to an error in the API server: JSONDecodeError: Expecting property name enclosed in double quotes ·

Author encode

reddit.com › r/learnpython › randomly receiving json.decoder.jsondecodeerror error when using requests.get. completely out of ideas

r/learnpython on Reddit: Randomly receiving json.decoder.JSONDecodeError error when using requests.get. Completely out of ideas

September 10, 2016 -

I really dont know what is going on...sometimes i am able to get 1-3 records, and other times i can get around 100 before the error appears. There is no consistency. This is how i have my request:

token = lms_token
url = endpoint
headers = {
    "Authorization": "bearer " + str(token),
    "Content-Type": "application/json;odata.metadata=minimal",
    "Accept": "application/json"
}
r = requests.get(url, headers=headers).json()

print(r)

Is there something i need to decode or change to prevent that error from happening? Here is the full error i receive:

File "C:\Users\name\Documents\lib\json_init_.py", line 319, in loads return _default_decoder.decode(s) File "C:\Users\name\Documents\lib\json\decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\name\Documents\lib\json\decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Thanks!

Top answer

1 of 3

check the returned content. Decoder is complaining it has to work with None (?) /gets no value.

2 of 3

They mention this explicitly in the Quickstart : In case the JSON decoding fails, r.json() raises an exception. For example, if the response gets a 204 (No Content), or if the response contains invalid JSON, attempting r.json() raises ValueError: No JSON object could be decoded. It should be noted that the success of the call to r.json() does not indicate the success of the response. Some servers may return a JSON object in a failed response (e.g. error details with HTTP 500). Such JSON will be decoded and returned. To check that a request is successful, use r.raise_for_status() or check r.status_code is what you expect. So the proper workflow is to handle HTTP error codes first: try: response = request.get(url, headers=headers) response.raise_for_status() r = response.json() except requests.exceptions.RequestException as e: print(e)

Stack Overflow

stackoverflow.com › questions › 16573332 › jsondecodeerror-expecting-value-line-1-column-1-char-0

python - JSONDecodeError: Expecting value: line 1 column 1 (char 0) - Stack Overflow

Top answer

1 of 16

322

Your code produced an empty response body; you'd want to check for that or catch the exception raised. It is possible the server responded with a 204 No Content response, or a non-200-range status code was returned (404 Not Found, etc.). Check for this.

Note:

There is no need to decode a response from UTF8 to Unicode, the json.loads() method can handle UTF8-encoded data natively.
pycurl has a very archaic API. Unless you have a specific requirement for using it, there are better choices.

Either requests or httpx offer much friendlier APIs, including JSON support.

If you can, replace your call with the following httpx code:

import httpx

response = httpx.get(url)
response.raise_for_status()  # raises exception when not a 2xx response
if response.status_code != 204:
    return response.json()

Of course, this won't protect you from a URL that doesn't comply with HTTP standards; when using arbitrary URLs where this is a possibility, check if the server intended to give you JSON by checking the Content-Type header, and for good measure catch the exception:

if (
    response.status_code != 204 and
    response.headers["content-type"].strip().startswith("application/json")
):
    try:
        return response.json()
    except ValueError:
        # decide how to handle a server that's misbehaving to this extent

2 of 16

261

Be sure to remember to invoke json.loads() on the contents of the file, as opposed to the file path of that JSON:

json_file_path = "/path/to/example.json"

with open(json_file_path, 'r') as j:
     contents = json.loads(j.read())

I think a lot of people are guilty of doing this every once in a while (myself included):

contents = json.load(json_file_path)

Stack Overflow

stackoverflow.com › questions › 52488117 › python-requests-randomly-breaks-with-jsondecodeerror

python requests randomly breaks with JSONDecodeError - Stack Overflow

Top answer

1 of 3

Looking at the documentation for this API it seems the only responses are in JSON format, so receiving HTML is strange. To increase the likelihood of receiving a JSON response, you can set the 'Accept' header to 'application/json'.

I tried querying this API many times with parameters and did not encounter a JSONDecodeError. This error is likely the result of another error on the server side. To handle it, except a json.decoder.JSONDecodeError in addition to the ConnectionError error you currently except and handle this error in the same way as the ConnectionError.

Here is an example with all that in mind:

import requests, json, time, random

def get_submission_records(client, since, try_number=1):
    url = 'http://reymisterio.net/data-dump/api.php/submission?filter[]=form,cs,'+client+'&filter[]=date,cs,'+since
    headers = {'Accept': 'application/json'}
    try:
        response = requests.get(url, headers=headers).json()
    except (requests.exceptions.ConnectionError, json.decoder.JSONDecodeError):
        time.sleep(2**try_number + random.random()*0.01) #exponential backoff
        return get_submission_records(client, since, try_number=try_number+1)
    else:
        return response['submission']['records']

I've also wrapped this logic in a recursive function, rather than using while loop because I think it is semantically clearer. This function also waits before trying again using exponential backoff (waiting twice as long after each failure).

Edit: For Python 2.7, the error from trying to parse bad json is a ValueError, not a JSONDecodeError

import requests, time, random

def get_submission_records(client, since, try_number=1):
    url = 'http://reymisterio.net/data-dump/api.php/submission?filter[]=form,cs,'+client+'&filter[]=date,cs,'+since
    headers = {'Accept': 'application/json'}
    try:
        response = requests.get(url, headers=headers).json()
    except (requests.exceptions.ConnectionError, ValueError):
        time.sleep(2**try_number + random.random()*0.01) #exponential backoff
        return get_submission_records(client, since, try_number=try_number+1)
    else:
        return response['submission']['records']

so just change that except line to include a ValueError instead of json.decoder.JSONDecodeError.

2 of 3

Catching ValueError is universal across python versions from py2 to py3

For py3 only is better to catch from already imported requests library: requests.JSONDecoderError, so you don't need to import json library.

Find elsewhere

Google Bing Mojeek

Techstaunch

techstaunch.com › blogs › json-decoder-jsondecodeerror-expecting-value-line-1-column-1-char-0-how-to-solve

json.decoder.jsondecodeerror: expecting value: line 1 column 1 (char 0) How to Fix, Solve

August 21, 2024 - The JSONDecodeError commonly occurs when trying to decode a JSON object in Python and the input data is not in valid JSON format.

LSEG Developer Community

community.developers.refinitiv.com › home › eikon data apis

JSONDecodeError in Python API - LSEG Developer Community

October 19, 2020 - When I try to retrieve data using the Pyhon API, I get a JSONDecodeError. I am using Python 3.8.5 (64-bit) on Windows 10 and version 1.1.6.post3 of the eikon package. The same code worked fine, i.e. produced the expected results, a couple of months ago. I updated the eikon package after I got ...

reddit.com › r/learnpython › json.decoder.jsondecodeerror: expecting value: line 1 column 1 (char 0) error while i'm trying to make my alexa/echo dot 3 powered by chatgpt

r/learnpython on Reddit: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ERROR while I'm trying to make my Alexa/Echo dot 3 powered by ChatGPT

April 8, 2023 -

Hello.

I'm trying to make my Alexa/Echo dot 3 powered by ChatGPT. I would like to integrate ChatGPT into my Alexa / Echo dot 3. I want to have more engaging and natural conversations with my smart speaker. To achieve the goal I found a Step-By-Step guide. The relevant steps to do are :

1. I need python 3.10.9

3. Inside the folder run the command `pip install -r requirements.txt` on cmd

4. run the server with: `python server.py`

ok. Let's assume that I'm using the jetson nano where I have installed ubuntu 20.04 and python 3.10.9,as you can see :

marietto@marietto-nano:/mnt/fisso/ChatGPT/BingChatGPT$ python --version
Python 3.10.9

marietto@marietto-nano:/mnt/fisso/ChatGPT/BingChatGPT$ python3 --version
Python 3.10.9

Let's also say that I have been able to satisfy all the python pip dependencies requested :

marietto@marietto-nano:/mnt/fisso/ChatGPT/BingChatGPT$ pip install -r requirements.txt

Requirement already satisfied: EdgeGPT==0.0.60 in /usr/local/lib/python3.10/site-packages (from -r requirements.txt (line 1)) (0.0.60)
Requirement already satisfied: Flask==2.2.3 in /usr/local/lib/python3.10/site-packages (from -r requirements.txt (line 2)) (2.2.3)
Requirement already satisfied: rich in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (13.3.3)
Requirement already satisfied: asyncio in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (3.4.3)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (2022.12.7)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (0.23.3)
Requirement already satisfied: prompt-toolkit in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (3.0.38)
Requirement already satisfied: websockets in /usr/local/lib/python3.10/site-packages (from EdgeGPT==0.0.60->-r requirements.txt (line 1)) (11.0.1)
Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.10/site-packages (from Flask==2.2.3->-r requirements.txt (line 2)) (8.1.3)
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.10/site-packages (from Flask==2.2.3->-r requirements.txt (line 2)) (3.1.2)
Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.10/site-packages (from Flask==2.2.3->-r requirements.txt (line 2)) (2.1.2)
Requirement already satisfied: Werkzeug>=2.2.2 in /usr/local/lib/python3.10/site-packages (from Flask==2.2.3->-r requirements.txt (line 2)) (2.2.3)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/site-packages (from rich->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (2.14.0)
Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /usr/local/lib/python3.10/site-packages (from rich->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (2.2.0)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /usr/local/lib/python3.10/site-packages (from httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (1.5.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/site-packages (from httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /usr/local/lib/python3.10/site-packages (from httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (0.16.3)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.10/site-packages (from prompt-toolkit->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (0.2.6)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/site-packages (from Jinja2>=3.0->Flask==2.2.3->-r requirements.txt (line 2)) (2.1.2)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/site-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (0.1.2)
Requirement already satisfied: idna; extra == "idna2008" in /usr/local/lib/python3.10/site-packages (from rfc3986[idna2008]<2,>=1.3->httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (3.4)
Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (0.14.0)
Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->EdgeGPT==0.0.60->-r requirements.txt (line 1)) (3.6.2)

WARNING: You are using pip version 20.0.2; however, version 23.0.1 is available.
You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command. 

[some python package requested to install pip 20.0.2,so I did it and I kept it to this version]

This is what happens when I try to run the server.py :

marietto@marietto-nano:/mnt/fisso/ChatGPT/BingChatGPT$ python server.py

Traceback (most recent call last):
  File "/mnt/fisso/ChatGPT/BingChatGPT/server.py", line 12, in <module>
    cookies = json.load(f)
  File "/usr/local/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I suspect that there is an error inside the code. Can someone confirm this ? The full code of the server.py is the following :

from flask import Flask, request, jsonify
import json
import sys
import asyncio
import random
from EdgeGPT import Chatbot, ConversationStyle
import threading
cookies = None
import time

with open('./cookies.json', 'r') as f:
    cookies = json.load(f)

app = Flask(__name__)

responses_final_list = {}

def generate_code():
    code = ""
    for i in range(2):
        code += str(random.randint(0, 9))
    return str(code)


@app.route('/res/<codes>', methods=['GET'])
def lidar_com_resposta(codes):
    codes = codes.replace("%20","").replace(" ", "")
    print("Password: " + str(codes))
    if codes in responses_final_list:
        return responses_final_list[codes]
    else:
        time.sleep(6)
        if codes in responses_final_list:
            return responses_final_list[codes]
        else:
            return "Invalid code, try again in 1 minute."


async def ask_stream_task(question, codes):
    wrote = 0
    finished = False
    bot = Chatbot(cookies=cookies)
    response_list = []
    while not finished:
        async for final, response in bot.ask_stream(prompt=question, conversation_style=ConversationStyle.creative):
            if not final:
                response_list.append(response[wrote:])
                wrote = len(response)
                sys.stdout.flush()
            else:
                response_list.clear()
                response_list.append(response["item"]["messages"][-1]["text"])
                responses_final_list[str(codes)] = response["item"]["messages"][-1]["text"]
                finished = True
                print("Finished: " + str(codes))

def between_callback(prompt, codes):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)

    loop.run_until_complete(ask_stream_task(prompt, codes))
    loop.close()

@app.route("/api")
async def api():
    prompt = request.args.get("prompt")
    timeout = 6
    codes = generate_code()
    asyncio.create_task(ask_stream_task(prompt, codes))
    _thread = threading.Thread(target=between_callback, args=(prompt, codes))
    _thread.start()
    return codes

if __name__ == "__main__":
    app.run(port=8080)

very thanks.

Top answer

1 of 3

The file cookies.json is empty, which isn't valid JSON.

2 of 3

The error tells you what the problem is: there's a problem reading the file "cookies.json". What're the contents of this file?

Peterbe.com

peterbe.com › plog › jsondecodeerror-in-requests.get.json-python-2-and-3

Cope with JSONDecodeError in requests.get().json() in Python 2 and 3 - Peterbe.com

import json import requests try: from json.decoder import JSONDecodeError except ImportError: JSONDecodeError = ValueError response = requests.get(url) try: print(response.json()) except JSONDecodeError: print("N'est pas JSON")

WebScraping.AI

webscraping.ai › faq › requests › how-do-i-decode-a-json-response-using-requests

How do I decode a JSON response using Requests? | WebScraping.AI

import requests from requests.exceptions import RequestException, HTTPError import json def fetch_json_data(url): try: response = requests.get(url, timeout=10) response.raise_for_status() # Raises HTTPError for bad responses # Check if response contains JSON content_type = response.headers.get('content-type', '') if 'application/json' not in content_type: print(f"Warning: Content-Type is {content_type}, not JSON") return response.json() except HTTPError as e: print(f"HTTP error occurred: {e}") except json.JSONDecodeError as e: print(f"Invalid JSON response: {e}") print(f"Response content: {response.text[:200]}...") except RequestException as e: print(f"Request error: {e}") except Exception as e: print(f"Unexpected error: {e}") return None # Usage data = fetch_json_data('https://api.example.com/data') if data: # Process the JSON data print(data)

Stack Overflow

stackoverflow.com › questions › 8381193 › handle-json-decode-error-when-nothing-returned

python - Handle JSON Decode Error when nothing returned - Stack Overflow

Top answer

1 of 2

240

There is a rule in Python programming called "it is Easier to Ask for Forgiveness than for Permission" (in short: EAFP). It means that you should catch exceptions instead of checking values for validity.

Thus, try the following:

try:
    qByUser = byUsrUrlObj.read()
    qUserData = json.loads(qByUser).decode('utf-8')
    questionSubjs = qUserData["all"]["questions"]
except ValueError:  # includes simplejson.decoder.JSONDecodeError
    print('Decoding JSON has failed')

EDIT: Since simplejson.decoder.JSONDecodeError actually inherits from ValueError (proof here), I simplified the catch statement by just using ValueError.

2 of 2

If you don't mind importing the json module, then the best way to handle it is through json.JSONDecodeError (or json.decoder.JSONDecodeError as they are the same) as using default errors like ValueError could catch also other exceptions not necessarily connected to the json decode one.

from json.decoder import JSONDecodeError


try:
    qByUser = byUsrUrlObj.read()
    qUserData = json.loads(qByUser).decode('utf-8')
    questionSubjs = qUserData["all"]["questions"]
except JSONDecodeError as e:
    # do whatever you want

//EDIT (Oct 2020):

As @Jacob Lee noted in the comment, there could be the basic common TypeError raised when the JSON object is not a str, bytes, or bytearray. Your question is about JSONDecodeError, but still it is worth mentioning here as a note; to handle also this situation, but differentiate between different issues, the following could be used:

from json.decoder import JSONDecodeError


try:
    qByUser = byUsrUrlObj.read()
    qUserData = json.loads(qByUser).decode('utf-8')
    questionSubjs = qUserData["all"]["questions"]
except JSONDecodeError as e:
    # do whatever you want
except TypeError as e:
    # do whatever you want in this case

Stack Overflow

stackoverflow.com › questions › 63677208 › json-decode-error-when-requesting-json-response-from-google

python - JSON Decode Error when requesting JSON response from Google - Stack Overflow

Top answer

1 of 1

response.json() does not convert any server response to a JSON, it simply parses 'stringified' JSONs. So if the server returns a string that is not a JSON, then this will throw a decode error.

Some servers do return JSON objects, in which case your code will work. In the case of https://www.google.com/search?q=london this actually returns HTML code (as you would expect since it's a webpage).

You can test this by printing the response:

print(response.text)

which outputs:

# some very long output that ends with:
...();})();google.drty&&google.drty();</script></body></html>

Notice the </html> tag at the end? So this cannot be parsed into a JSON.

So how do you parse this into a usable HTML? You can use beautiful soup:

import requests
from bs4 import BeautifulSoup

url = "https://www.google.com/search?q=london"

response = requests.get(url, headers={"Accept": "application/json"})

soup = BeautifulSoup(response.text)

print(soup.prettify())

Newtum

apitest.newtum.com › examples › python-handle-json-decode-error

How to Handle `JSONDecodeError` in Python Requests | API Navigator

August 29, 2025 - import requests # This URL returns a normal HTML page, not JSON url = 'https://httpbin.org/html' response = requests.get(url) if response.status_code == 200: try: data = response.json() print("JSON data:", data) except requests.exceptions.JSONDecodeError: print("Failed to decode JSON.") print("Response was not in JSON format.") print("Raw text response (first 100 chars):") print(response.text[:100] + '...') else: print(f"Request failed with status code {response.status_code}") CopyRun

LSEG Developer Community

community.developers.refinitiv.com › eikon data apis

JSONDecodeError in Python API — LSEG Developer Community

October 15, 2020 - When I try to retrieve data using the Pyhon API, I get a JSONDecodeError. I am using Python 3.8.5 (64-bit) on Windows 10 and version 1.1.6.post3 of the eikon package. The same code worked fine, i.e. produced the expected results, a couple of months ago. I updated the eikon package after I got the error message.

Prefect

linen.prefect.io › t › 16641643 › ulva73b9p-i-am-trying-to-start-a-worker-on-a-process-work-po

< Marvin> I am trying to start a worker on a process work po Prefect Community #ask-marvin

March 5, 2024 - JSONDecodeError due to a response that is not valid JSON. From what I found, there are a couple of GitHub issues reported about similar errors.

Stack Overflow

stackoverflow.com › questions › tagged › jsondecodeerror

Newest 'jsondecodeerror' Questions - Stack Overflow

I am trying to parse the content of an API response, which is of the type application/json. However, I am getting the following error: JSONDecodeError: Extra data: line 1 column 2 (char 1) The API ...

Readthedocs

weaviate-python-client.readthedocs.io › en › v4.5.6 › _modules › weaviate › exceptions.html

weaviate.exceptions — Weaviate Python Client 4.5.7.dev0+ga08b451.d20240423 documentation

[docs] class ResponseCannotBeDecodedError(WeaviateBaseError): def __init__(self, location: str, response: Union[httpx.Response, requests.Response]): """Raised when a weaviate response cannot be decoded to json Arguments: `location`: From which code path the exception was raised.