You'd generally write one JSON object to a file; that object can contain your other objects:
json_data = {
'p_id': p_id,
'word_list': word_list,
# ...
}
with open('data.json', 'w') as fp:
json.dump(json_data, fp, sort_keys=True, indent=4)
Now all you have to do is read that one object and address the values by the same keys.
If you must write multiple JSON documents, avoid using newlines so you can read the file line by line, as parsing the file one JSON object at a time is a lot more involved.
Answer from Martijn Pieters on Stack Overflow
Hey, i am new to programming and I am trying to decode thousands of JSON files.
Usually there is one object in each JSON file, but for some reason a lot of my files have multiple JSON objects. Some have up to 5 objects.
{
"testNumber": "test200",
"device": {
"deviceID": 4000008
},
"user": {
"userID": "4121412"
}
}
{
"testNumber": "test201",
"device": {
"deviceID": 4000009
},
"user": {
"userID": "4121232"
}
}
My code gives me the error: json.decoder.JSONDecodeError: Extra data: line 2 column 1
Because of that I am using except ValueError but I would like to get the data out of these JSON files.
import json
import os
test_dir = r'C:\Users\path\path'
for file in os.listdir(test_dir):
if 'testNumber' in file:
try:
data = json.load(open(test_dir + '\\' + file, 'r'))
print("valid")
except ValueError:
print("Decoding JSON has failed")
Since json.loads and json.load don't work: is there any other way open the JSON file so that I can try to split the content in 2 objects?
Hi all!
I'm getting this error when I want to load (decode) multiple JSON objects.
json.decoder.JSONDecodeError: Extra data: line 1 column 3 (char 2)
Done a little digging and found it's due to the JSON module being unable to parse multiple top level objects from a JSON file. I read, if you put the Dictionaries inside a list, you can dump them all and load them back. Perfect!
I wrote this code to test it, sadly it doesn't work because (I think) I'm adding another JSON Object wrapped in an Array outside of the first JSON Array.
import json
dict1 = {}
dict2 = {}
with open('test.json', 'a') as test:
json.dump([dict1,dict2], test) # This works and decodes!
json.dump([dict2],test) # This line breaks the decoder when run with line above!
with open('test.json','r') as test:
x = json.load(test)
print(x) # Should print out contents of file. Is there any workaround (or something I'm missing) that can help me out and will let me load multiple top level Objects from a JSON file?
Thanks!
Example code:
def filewriter(line):
dictionary = {}
dictionary['name'] = line[0][0] #assume this is a a string
dictionary['item_to_buy'] = line[0][1]
dictionary['currency'] = line[0][2]
dictionary['league'] = line[0][3]
with open('logs.json', 'r+') as f:
if len(f.read()) == 0:
f.write(json.dumps(dictionary))
else:
f.write(',\n' + json.dumps(dictionary))
def retrieve():
with open('logs.json') as f:
g = json.load(f)
print(g[1]['name'])So I'm creating dictionaries separated by a comma and a newline, however json format dictates that I need brackets enclosing multiple dictionaries. For example, the dictionaries
{"name": "name", "item_to_buy": "item", "currency": "orb", "league": "league"},
{"name": "name", "item_to_buy": "item", "currency": "orb", "league": "league"}need to be
[{"name": "name", "item_to_buy": "item", "currency": "orb", "league": "league"},
{"name": "name", "item_to_buy": "item", "currency": "orb", "league": "league"}]I've tried to do g = json.load(list(f.read())) to hopefully encapsulate them with a list every time I want to read them but to no avail.
I want to store a dictionary in some file and be able to retrieve the dictionary for another program, and the best way seems to be JSON, but I'm having a little trouble formatting it.
Edit: I'm adding dictionaries in real time, so it's not just 2 or 3 dictionaries, but a whole lot I need to write.
Ok so I'm back again :)
Here's the input json file
[
{
"PlayerID": 589,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Trevor",
"Surname": "Scowcroft",
"Age": 29,
"Rating": 35
},
{
"PlayerID": 859,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Francisco",
"Surname": "Alves",
"Age": 18,
"Rating": 53
},
{
"PlayerID": 610,
"TeamID": 609,
"TeamName": "Bury",
"Forename": "Simon",
"Surname": "Marsden",
"Age": 18,
"Rating": 40
},
{
"PlayerID": 611,
"TeamID": 609,
"TeamName": "Bury",
"Forename": "Robert",
"Surname": "Venus",
"Age": 25,
"Rating": 44
},
{
"PlayerID": 629,
"TeamID": 609,
"TeamName": "Bury",
"Forename": "Carl",
"Surname": "Williams",
"Age": 18,
"Rating": 53
},
{
"PlayerID": 654,
"TeamID": 611,
"TeamName": "Charlton Athletic",
"Forename": "Robbie",
"Surname": "Moffat",
"Age": 24,
"Rating": 40
},
{
"PlayerID": 655,
"TeamID": 611,
"TeamName": "Charlton Athletic",
"Forename": "Anthony",
"Surname": "Rowett",
"Age": 21,
"Rating": 43
}
]And here's the code so far...
import json
jsonFilePath="new.json"
with open(jsonFilePath, encoding='utf-8') as jsonFile:
jsonData=json.load(jsonFile)
for row in jsonData:
filename = "teams/"+str(row.get("TeamID"))+".json"
print(filename)
# struggling with this part...
with open(filename, 'w', encoding='utf-8') as outputFile:
outputFile.write(json.dumps(row, ensure_ascii=False, indent=2))I've already created the teams directory ;)
The above script creates 3 json files (607.json,609.json,611.json) in teams directory (exactly as I want to)
But inner content is not what I want... :(
It only contains the last item
for example 607.json file only contains
{
"PlayerID": 859,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Francisco",
"Surname": "Alves",
"Age": 18,
"Rating": 53
}I want it to contain both (I mean all because real data is large again...)
So I change the mode from w to a and sure enough it now contains both (all) items like this
again taking 607.json for example
{
"PlayerID": 589,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Trevor",
"Surname": "Scowcroft",
"Age": 29,
"Rating": 35
}{
"PlayerID": 859,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Francisco",
"Surname": "Alves",
"Age": 18,
"Rating": 53
}Now there are two issues with this one
First the data is not in correct format....
Secondly if I run the script again then it keeps appending to the file (which is to be expected with a mode) :P
The data should be in this format (again 607.json only)
[
{
"PlayerID": 589,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Trevor",
"Surname": "Scowcroft",
"Age": 29,
"Rating": 35
},
{
"PlayerID": 859,
"TeamID": 607,
"TeamName": "Walsall",
"Forename": "Francisco",
"Surname": "Alves",
"Age": 18,
"Rating": 53
}
]Please point me in the right direction.
Thanks :)
If you want to produce valid JSON file you need to write all values at once, not one value at a time (which will produce ndjson or JSON lines)
So, for a valid JSON
values = [{"first_name": "John", "last_name": "Smith", "food": "corn"},
{"first_name": "Jane", "last_name": "Doe", "food": "soup"}]
import json
with open('some_file.json', 'w') as f:
json.dump(values, f, indent=4)
some_file.json
[
{
"first_name": "John",
"last_name": "Smith",
"food": "corn"
},
{
"first_name": "Jane",
"last_name": "Doe",
"food": "soup"
}
]
if you really need ndjson - you can use ndjson package (need install via pip from PyPi).
import ndjson
with open('some_file2.ndjson', 'w') as f:
ndjson.dump(values, f)
some_file2.ndjson
{"first_name": "John", "last_name": "Smith", "food": "corn"}
{"first_name": "Jane", "last_name": "Doe", "food": "soup"}
Alternative to ndjson package is jsonlines package
You can newline after each
f.write(json.dumps(value, sort_keys=True, indent=0))
like this - f.write('\n')