You have a JSON Lines format text file. You need to parse your file line by line:
import json
data = []
with open('file') as f:
for line in f:
data.append(json.loads(line))
Each line contains valid JSON, but as a whole, it is not a valid JSON value as there is no top-level list or object definition.
Note that because the file contains JSON per line, you are saved the headaches of trying to parse it all in one go or to figure out a streaming JSON parser. You can now opt to process each line separately before moving on to the next, saving memory in the process. You probably don't want to append each result to one list and then process everything if your file is really big.
If you have a file containing individual JSON objects with delimiters in-between, use How do I use the 'json' module to read in one JSON object at a time? to parse out individual objects using a buffered method.
Answer from Martijn Pieters on Stack Overflowpython - Loading and parsing a JSON file with multiple JSON objects - Stack Overflow
How do I open or even find a json file?
python - Reading JSON from a file - Stack Overflow
Handling JSON files with ease in Python
Where can I specify the file path when using 'json.dump()' in Python to write JSON data?
How can I write a JSON file in Python?
How do I read a JSON file in Python using Pandas?
Videos
You have a JSON Lines format text file. You need to parse your file line by line:
import json
data = []
with open('file') as f:
for line in f:
data.append(json.loads(line))
Each line contains valid JSON, but as a whole, it is not a valid JSON value as there is no top-level list or object definition.
Note that because the file contains JSON per line, you are saved the headaches of trying to parse it all in one go or to figure out a streaming JSON parser. You can now opt to process each line separately before moving on to the next, saving memory in the process. You probably don't want to append each result to one list and then process everything if your file is really big.
If you have a file containing individual JSON objects with delimiters in-between, use How do I use the 'json' module to read in one JSON object at a time? to parse out individual objects using a buffered method.
In case you are using pandas and you will be interested in loading the json file as a dataframe, you can use:
import pandas as pd
df = pd.read_json('file.json', lines=True)
And to convert it into a json array, you can use:
df.to_json('new_file.json')
I'm making a text based rpg in python and i learnt that the data can be saved to a json file and then can be read from it and used. But how do I access that json file? Or maybe i should do another method to save game? Also is there a way to open a json file in python so that python can read it's values?
The json.load() method (without "s" in "load") can read a file directly:
import json
with open('strings.json') as f:
d = json.load(f)
print(d)
You were using the json.loads() method, which is used for string arguments only.
The error you get with json.loads is a totally different problem. In that case, there is some invalid JSON content in that file. For that, I would recommend running the file through a JSON validator.
There are also solutions for fixing JSON like for example How do I automatically fix an invalid JSON string?.
Here is a copy of code which works fine for me,
import json
with open("test.json") as json_file:
json_data = json.load(json_file)
print(json_data)
with the data
{
"a": [1,3,"asdf",true],
"b": {
"Hello": "world"
}
}
You may want to wrap your json.load line with a try catch, because invalid JSON will cause a stacktrace error message.
I have finished writing the third article in the Data Engineering with Python series. This is about working with JSON data in Python. I have tried to cover every necessary use case. If you have any other suggestions, let me know.
Working with JSON in Python
Data Engineering with Python series