Why json.dump() and .load() are really needed?
python - How do I write JSON data to a file? - Stack Overflow
python - Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence - Stack Overflow
python - Using json.dumps(string) to dump a string - Stack Overflow
Hi, hope everyone is well.
Just nearing the basics end of PCC book, I'm at saving user's data now. What exactly is the reason, when storing simple data, to use json.dump() or load(), instead of just saving and then reading it from simple text file?
I just can't place it in my head why do I really need it and it always makes it more difficult for me to learn if that's the case.
Thank you all in advance.
data is a Python dictionary. It needs to be encoded as JSON before writing.
Use this for maximum compatibility (Python 2 and 3):
import json
with open('data.json', 'w') as f:
json.dump(data, f)
On a modern system (i.e. Python 3 and UTF-8 support), you can write a nicer file using:
import json
with open('data.json', 'w', encoding='utf-8') as f:
json.dump(data, f, ensure_ascii=False, indent=4)
See json documentation.
To get utf8-encoded file as opposed to ascii-encoded in the accepted answer for Python 2 use:
import io, json
with io.open('data.txt', 'w', encoding='utf-8') as f:
f.write(json.dumps(data, ensure_ascii=False))
The code is simpler in Python 3:
import json
with open('data.txt', 'w') as f:
json.dump(data, f, ensure_ascii=False)
On Windows, the encoding='utf-8' argument to open is still necessary.
To avoid storing an encoded copy of the data in memory (result of dumps) and to output utf8-encoded bytestrings in both Python 2 and 3, use:
import json, codecs
with open('data.txt', 'wb') as f:
json.dump(data, codecs.getwriter('utf-8')(f), ensure_ascii=False)
The codecs.getwriter call is redundant in Python 3 but required for Python 2
Readability and size:
The use of ensure_ascii=False gives better readability and smaller size:
>>> json.dumps({'price': '€10'})
'{"price": "\\u20ac10"}'
>>> json.dumps({'price': '€10'}, ensure_ascii=False)
'{"price": "€10"}'
>>> len(json.dumps({'абвгд': 1}))
37
>>> len(json.dumps({'абвгд': 1}, ensure_ascii=False).encode('utf8'))
17
Further improve readability by adding flags indent=4, sort_keys=True (as suggested by dinos66) to arguments of dump or dumps. This way you'll get a nicely indented sorted structure in the json file at the cost of a slightly larger file size.
Use the ensure_ascii=False switch to json.dumps(), then encode the value to UTF-8 manually:
>>> json_string = json.dumps("ברי צקלה", ensure_ascii=False).encode('utf8')
>>> json_string
b'"\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94"'
>>> print(json_string.decode())
"ברי צקלה"
If you are writing to a file, just use json.dump() and leave it to the file object to encode:
with open('filename', 'w', encoding='utf8') as json_file:
json.dump("ברי צקלה", json_file, ensure_ascii=False)
Caveats for Python 2
For Python 2, there are some more caveats to take into account. If you are writing this to a file, you can use io.open() instead of open() to produce a file object that encodes Unicode values for you as you write, then use json.dump() instead to write to that file:
with io.open('filename', 'w', encoding='utf8') as json_file:
json.dump(u"ברי צקלה", json_file, ensure_ascii=False)
Do note that there is a bug in the json module where the ensure_ascii=False flag can produce a mix of unicode and str objects. The workaround for Python 2 then is:
with io.open('filename', 'w', encoding='utf8') as json_file:
data = json.dumps(u"ברי צקלה", ensure_ascii=False)
# unicode(data) auto-decodes data to unicode if str
json_file.write(unicode(data))
In Python 2, when using byte strings (type str), encoded to UTF-8, make sure to also set the encoding keyword:
>>> d={ 1: "ברי צקלה", 2: u"ברי צקלה" }
>>> d
{1: '\xd7\x91\xd7\xa8\xd7\x99 \xd7\xa6\xd7\xa7\xd7\x9c\xd7\x94', 2: u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'}
>>> s=json.dumps(d, ensure_ascii=False, encoding='utf8')
>>> s
u'{"1": "\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4", "2": "\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4"}'
>>> json.loads(s)['1']
u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'
>>> json.loads(s)['2']
u'\u05d1\u05e8\u05d9 \u05e6\u05e7\u05dc\u05d4'
>>> print json.loads(s)['1']
ברי צקלה
>>> print json.loads(s)['2']
ברי צקלה
To write to a file
import codecs
import json
with codecs.open('your_file.txt', 'w', encoding='utf-8') as f:
json.dump({"message":"xin chào việt nam"}, f, ensure_ascii=False)
To print to stdout
import json
print(json.dumps({"message":"xin chào việt nam"}, ensure_ascii=False))
You don't dump a string but to a string.
d = {"foo": "bar"}
j = json.dumps(d)
print(j)
Output:
{"foo": "bar"}
It makes no sense to use a string as the argument of json.dumps which is expecting a dictionary or a list/tuple as the argument.
Serialize
objto a JSON formatted str using this conversion table.
In [17]: d = {"foo": "bar"}
In [18]: j = json.dumps(d)
In [19]: j
Out[19]: '{"foo": "bar"}'
Now if you need your dictionary back instead of a string
In [37]: l=json.loads(j)
In [38]: l
Out[38]: {u'foo': u'bar'}
or alternatively
In [20]: import ast
In [22]: k=ast.literal_eval(j)
In [23]: k
Out[23]: {'foo': 'bar'}