As of mypy 0.990, mypy finally supports recursive type annotations, using the natural syntax:
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, List['JSONVal'], Dict[str, 'JSONVal']]
d: JSONVal = {'a': ['b']}
mypy output:
Success: no issues found in 1 source file
Before 0.990, this would produce an error reporting a lack of recursive type support:
$ mypy asdf.py
asdf.py:3: error: Recursive types not fully supported yet, nested types replaced with "Any"
On such versions, Dict[str, Any] would be the way to go.
You can also use mutually recursive type aliases now, so you can do things like
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, 'JSONArray', 'JSONObject']
JSONArray = List[JSONVal]
JSONObject = Dict[str, JSONVal]
d: JSONObject = {'a': ['b']}
Answer from user2357112 on Stack OverflowAs of mypy 0.990, mypy finally supports recursive type annotations, using the natural syntax:
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, List['JSONVal'], Dict[str, 'JSONVal']]
d: JSONVal = {'a': ['b']}
mypy output:
Success: no issues found in 1 source file
Before 0.990, this would produce an error reporting a lack of recursive type support:
$ mypy asdf.py
asdf.py:3: error: Recursive types not fully supported yet, nested types replaced with "Any"
On such versions, Dict[str, Any] would be the way to go.
You can also use mutually recursive type aliases now, so you can do things like
from typing import Union, Dict, List
JSONVal = Union[None, bool, str, float, int, 'JSONArray', 'JSONObject']
JSONArray = List[JSONVal]
JSONObject = Dict[str, JSONVal]
d: JSONObject = {'a': ['b']}
Support for recursive types is now in Mypy.
As of October 2022 the implementation is provisional. You can enable it by adding the enable_recursive_aliases = true flag to pyproject.toml.
Starting from version 0.990 this will be enabled by default. Source.
» pip install python-easy-json
json - using recursive function and structure type in python? - Stack Overflow
recursion - Recursive search in json with python - Stack Overflow
python - After recursive search of my json object how to determine data type for looping data - Stack Overflow
Json Type in Mypy.
Use recursion, and use type() or isinstance() to decide what to do.
def handle(x):
if isinstance(x, dict):
# do dict stuff
for key, val in x.items():
handle(val)
elif isinstance(x, list):
# do list stuff
for val in x:
handle(val)
elif isinstance(x, str):
# do string stuff
print(x)
elif isinstance(x, (int, float)):
# maybe integer, float, etc
print(x)
else:
print("None???")
d = json.loads(json_string)
handle(d)
Above recursive implementation will handle your use case of array in array, dict in array, etc
Use isinstance:
s='this is a string (str)'
if isinstance(s,str):
do something
Also able to do multiple like:
isinstance(s,(str,int))
Or more inefficient way by checking type:
if type(s) is str:
do something
This is able to use multiple like:
type(s) in (str,int)
But in these solutions i recommend to use isinstance
with recursion you can do something like this:
result = []
def finditem(obj, key):
if key in obj: result.append({"timestamp": obj[key], "text": obj['textsList'][0]['text']})
for k, v in obj.items():
if isinstance(v,dict):
item = finditem(v, key)
if item is not None:
result.append({"timestamp": item,"text":v['textsList'][0]['text']})
elif isinstance(v,list):
for i in v:
item = finditem(i, key)
if item is not None:
result.append({"timestamp": item,"text": i['textsList'][0]['text']})
finditem(data, 'timestamp')
print (result)
result:
[{'timestamp': 1234567890, 'text': 'some text here 0'}, {'timestamp': 12345678901234, 'text': 'some other text 1'}, {'timestamp': 12345678901234, 'text': 'some other text 2'}]
EDIT:
result = []
def finditem(obj, key):
if key in obj and obj[key]=='Recursive': result.append({"timestamp": obj["data"]["timestamp"],"text": obj["data"]['textsList'][0]['text']})
for k, v in obj.items():
if isinstance(v,dict):
item = finditem(v, key)
if item == 'Recursive':
d = v['timestamp']
result.append({"timestamp": d,"text": v['textsList'][0]['text']})
elif isinstance(v,list):
for list_item in v:
item = finditem(list_item, key)
if item == 'Recursive':
d = list_item['timestamp']
result.append({"timestamp": d,"text": list_item['textsList'][0]['text']})
finditem(data,'type')
print (result)
result:
[{'timestamp': 12345678901234, 'text': 'some other text 1'}, {'timestamp': 12345678901234, 'text': 'some other text 2'}]
You can use a recursive generator function:
def get_text(d):
if isinstance(d, dict):
if 'timestamp' in d:
yield {'timestamp':d['timestamp'], 'textsList':''.join(i['text'] for i in d['textsList'])}
for b in d.values():
yield from get_text(b)
elif isinstance(d, list):
yield from [i for j in d for i in get_text(j)]
data = {'result': [{'timestamp': 1234567890, 'textsList': [{'text': 'some text here 0'}], 'otherList': [{'type': 'Nothing'}, {'type': 'Recursive', 'data': {'timestamp': 12345678901234, 'textsList': [{'text': 'some other text 1'}], 'otherList': [{'type': 'Nothing'}, {'type': 'Recursive', 'data': {'timestamp': 12345678901234, 'textsList': [{'text': 'some other text 2'}], 'otherList': []}}]}}]}]}
print(list(get_text(data)))
Output:
[{'timestamp': 1234567890, 'textsList': 'some text here 0'},
{'timestamp': 12345678901234, 'textsList': 'some other text 1'},
{'timestamp': 12345678901234, 'textsList': 'some other text 2'}]
Hey there! I have a weird question but Im stuck for days on this.
I have quite a lot of requests done via python using the requests module.
def url_requester(url: str) -> list | ???: response=requests.request(method, payload, url, etc) if method=="get": return response.json()[0] return response.json()
Is there a proper way you type hint this function? It has being killing me, mypy says it is wrong . Thanks!
The JSON might contain a list of objects, which needs to be searched:
Python 2.7 version:
def item_generator(json_input, lookup_key):
if isinstance(json_input, dict):
for k, v in json_input.iteritems():
if k == lookup_key:
yield v
else:
for child_val in item_generator(v, lookup_key):
yield child_val
elif isinstance(json_input, list):
for item in json_input:
for item_val in item_generator(item, lookup_key):
yield item_val
Python 3.x version:
def item_generator(json_input, lookup_key):
if isinstance(json_input, dict):
for k, v in json_input.items():
if k == lookup_key:
yield v
else:
yield from item_generator(v, lookup_key)
elif isinstance(json_input, list):
for item in json_input:
yield from item_generator(item, lookup_key)
def id_generator(dict_var):
for k, v in dict_var.items():
if k == "id":
yield v
elif isinstance(v, dict):
for id_val in id_generator(v):
yield id_val
This will create an iterator which will yield every value on any level under key "id". Example usage (printing all of those values):
for _ in id_generator(some_json_dict):
print(_)
for asg in resources:
This iterates through the keys of resources, rather than the values. Try:
for asg in resources.values():
param_file_handle = json.load(open("sample.json"))
resources = param_file_handle['Resources']
for asg in resources.values():
if asg["Type"] == "AWS::AutoScaling::AutoScalingGroup":
print asg["Properties"]["MaxSize"]
try this.
I am studying trying to self teach python with an online course and I am stuck on a question I would like some help with, the question is:
"George runs a company. He manages James and Jamila, who each have a small team to manage. In James' team are Jill and Jenny. In Jamila's team are Jewel, Jasmine and Jeremy. Create a JSON object in a string variable called company where each item has a name field and a field called manages which contains an array of the people managed by that person. If a person does not manage anybody, they have no field called manages. Then convert the JSON string to a dictionary in a variable called company_dict.
Finally, write a recursive function that accepts the dictionary as an argument and lists each person and their level in the organisation, so that George is at level 1, James and Jamila are at 2, and so on. Call the function and print the result. It should look like this: George is at level 1 James is at level 2 Jill is at level 3 Jenny is at level 3 Jamila is at level 2 Jewel is at level 3 Jasmine is at level 3 Jeremey is at level 3 The function can either return a string, which you print after it has been called, or it can print its results as it runs and return nothing."
It's a little hard to understand exactly what you are after here, but it seems like for some reason you have a bunch of nested json and you want to search it for an id and return a list that represents the path down the json nesting. If so, the quick and easy path is to recurse on the dictionary (that you got from json.load) and collect the keys as you go. When you find an 'id' key that matches the id you are searching for you are done. Here is some code that does that:
def all_keys(search_dict, key_id):
def _all_keys(search_dict, key_id, keys=None):
if not keys:
keys = []
for i in search_dict:
if search_dict[i] == key_id:
return keys + [i]
if isinstance(search_dict[i], dict):
potential_keys = _all_keys(search_dict[i], key_id, keys + [i])
if 'id' in potential_keys:
keys = potential_keys
break
return keys
return _all_keys(search_dict, key_id)[:-1]
The reason for the nested function is to strip off the 'id' key that would otherwise be on the end of the list.
This is really just to give you an idea of what a solution might look like. Beware the python recursion limit!
Based on the assumption that you need the full dictionary path until a key named id has a particular value, here's a recursive solution that iterates the whole dict. Bear in mind that:
- The code is not optimized at all
- For huge json objects it might yield StackOverflow :)
- It will stop at first encountered value found (in theory there shouldn't be more than 1 if the json is semantically correct)
The code:
import json
from types import DictType
SEARCH_KEY_NAME = "id"
FOUND_FLAG = ()
CRONO_FILE = "a.jsn"
def decode_crono(crono_file):
with open(crono_file) as json_file:
return json.load(json_file)
def traverse_dict(dict_obj, value):
for key in dict_obj:
key_obj = dict_obj[key]
if key == SEARCH_KEY_NAME and key_obj == value:
return FOUND_FLAG
elif isinstance(key_obj, DictType):
inner = traverse_dict(key_obj, value)
if inner is not None:
return (key,) + inner
return None
if __name__ == "__main__":
value = "23314"
json_dict = decode_crono(CRONO_FILE)
result = traverse_dict(json_dict, value)
print result