I notice this question was asked a few years ago but if someone else find this, here are some newer projects trying to address this same problem:
- ObjectPath (for Python and Javascript): http://objectpath.org/
- jsonpath (Python reimplementation of the Javascript equivalent): https://pypi.org/project/jsonpath/
- yaql: https://yaql.readthedocs.io/en/latest/readme.html
- pyjq (Python bindings for jq https://stedolan.github.io/jq/): https://pypi.org/project/pyjq/
- JMESPath: https://github.com/jmespath/jmespath.py
I personally went with pyjq because I use jq all the time for data exploration but ObjectPath seems very attractive and not limited to json.
I notice this question was asked a few years ago but if someone else find this, here are some newer projects trying to address this same problem:
- ObjectPath (for Python and Javascript): http://objectpath.org/
- jsonpath (Python reimplementation of the Javascript equivalent): https://pypi.org/project/jsonpath/
- yaql: https://yaql.readthedocs.io/en/latest/readme.html
- pyjq (Python bindings for jq https://stedolan.github.io/jq/): https://pypi.org/project/pyjq/
- JMESPath: https://github.com/jmespath/jmespath.py
I personally went with pyjq because I use jq all the time for data exploration but ObjectPath seems very attractive and not limited to json.
I thought about this a little bit, and I lean towards something less specific such as a "JSON Query Language" and considered something more generic. I remembered from working with C# a bit that they had a somewhat generic querying system called LINQ for handling these sort of querying issues.
It looks as though Python has something similar called Pynq which supports basic querying such as:
filtered_collection = From(some_collection).where("item.property > 10").select_many()
It even appears to have some basic aggregation functions. While not being specific to JSON, I think it's a least a good starting point for querying.
Python - Query JSON file for occurrences of key and create list of values
Have you tried Python's built-in JSON decoder?
import json
with open('your_file.json', 'r') as f:
data = json.load(f)
# Now you have a Python dict with all of the information
# from the json fileSo let's assume that the JSON file looks like this:
{
'runs': [
{'name': 'run1', 'time': 1000, 'distance': '35'},
{'name': 'run2', 'time': 500, 'distance': '20'}
]}
You could do
for run in data['runs']: print(run['distance'])More on reddit.com
Best method to query and search JSON in Python?
Big Query JSON Validation
Simple GraphQL Client for Python
Thanks for sgqlc.
It would be nice if it provided help with pagination. I found it really cumbersome to do manually especially with more than one level.
More on reddit.comCan JMESPath be used on HTML?
No, for that refer to very similar HTML path languages like CSS Selectors and Xpath Selectors.
What are some alternatives to JMESPath?
Some other popular query language for JSON are JsonPath, JQ and pyquery
What's the difference between [] and [*] in JMESPath?
The [] flattens all results while [*] keeps the structure as it is in the orignal dataset. See this example in Python:
data = {
"employees": [
{
"people": [
{"address": ["123 Main St", "California", "US"]},
{"address": ["456 Sec St", "Nevada", "US"]},
],
},
{
"people": [
{"address": ["789 Main St", "Washington", "US"]},
{"address": ["a12 Sec St", "Alaska", "US"]},
],
},
]
}
jmespath.search("employees[*].people[*].address", data)
[
# fromt he first group:
[['123 Main St', 'California', 'US'], ['456 Sec St', 'Nevada', 'US']],
# from the second group:
[['789 Main St', 'Washington', 'US'], ['a12 Sec St', 'Alaska', 'US']]
]
jmespath.search("employees[].people[].address", data)
[
# all groups merged:
['123 Main St', 'California', 'US'],
['456 Sec St', 'Nevada', 'US'],
['789 Main St', 'Washington', 'US'],
['a12 Sec St', 'Alaska', 'US']
]