This can happen if you violate a requirement of dict, and change its hash.
When an object is used in a dict, its hash value must not change, and its equality to other objects must not change. Other properties may change, as long as they don't affect how it appears to the dict.
(This does not mean that a hash value is never allowed to change. That's a common misconception. Hash values themselves may change. It's only dict which requires that key hashes be immutable, not __hash__ itself.)
The following code adds an object to a dict, then changes its hash out from under the dict. q[a] = 2 then adds a as a new key in the dict, even though it's already present; since the hash value changed, the dict doesn't find the old value. This reproduces the peculiarity you saw.
class Test(object):
def __init__(self, h):
self.h = h
def __hash__(self):
return self.h
a = Test(1)
q = {}
q[a] = 1
a.h = 2
q[a] = 2
print q
# True:
print len(set(q.keys())) != len(q.keys())
Keys are not unique for a python dictionary! - Stack Overflow
hash - How To Create a Unique Key For A Dictionary In Python - Stack Overflow
Dictionary with non unique keys
python - Unique dictionary values - print keys - Code Review Stack Exchange
This can happen if you violate a requirement of dict, and change its hash.
When an object is used in a dict, its hash value must not change, and its equality to other objects must not change. Other properties may change, as long as they don't affect how it appears to the dict.
(This does not mean that a hash value is never allowed to change. That's a common misconception. Hash values themselves may change. It's only dict which requires that key hashes be immutable, not __hash__ itself.)
The following code adds an object to a dict, then changes its hash out from under the dict. q[a] = 2 then adds a as a new key in the dict, even though it's already present; since the hash value changed, the dict doesn't find the old value. This reproduces the peculiarity you saw.
class Test(object):
def __init__(self, h):
self.h = h
def __hash__(self):
return self.h
a = Test(1)
q = {}
q[a] = 1
a.h = 2
q[a] = 2
print q
# True:
print len(set(q.keys())) != len(q.keys())
The underlying code for dictionaries and sets is substantially the same, so you can usually expect that len(set(d.keys()) == len(d.keys()) is an invariant.
That said, both sets and dicts depend on __eq__ and __hash__ to identify unique values and to organize them for efficient search. So, if those return inconsistent results (or violate the rule that "a==b implies hash(a)==hash(b)", then there is no way to enforce the invariant:
>>> from random import randrange
>>> class A():
def __init__(self, x):
self.x = x
def __eq__(self, other):
return bool(randrange(2))
def __hash__(self):
return randrange(8)
def __repr__(self):
return '|%d|' % self.x
>>> s = [A(i) for i in range(100)]
>>> d = dict.fromkeys(s)
>>> len(d.keys())
29
>>> len(set(d.keys()))
12
I prefer serializing the dict as JSON and hashing that:
import hashlib
import json
a={'name':'Danish', 'age':107}
b={'age':107, 'name':'Danish'}
# Python 2
print hashlib.sha1(json.dumps(a, sort_keys=True)).hexdigest()
print hashlib.sha1(json.dumps(b, sort_keys=True)).hexdigest()
# Python 3
print(hashlib.sha1(json.dumps(a, sort_keys=True).encode()).hexdigest())
print(hashlib.sha1(json.dumps(b, sort_keys=True).encode()).hexdigest())
Returns:
71083588011445f0e65e11c80524640668d3797d
71083588011445f0e65e11c80524640668d3797d
No - you can't rely on particular order of elements when converting dictionary to a string.
You can, however, convert it to sorted list of (key,value) tuples, convert it to a string and compute a hash like this:
a_sorted_list = [(key, a[key]) for key in sorted(a.keys())]
print hashlib.sha1( str(a_sorted_list) ).hexdigest()
It's not fool-proof, as a formating of a list converted to a string or formatting of a tuple can change in some future major python version, sort order depends on locale etc. but I think it can be good enough.
Im not entirely new to python but Im having a problem that I cant quite seem to find the answer to.
What I have is a dictionary that ofcourse have keys and values. Only the keys arent unique ( and shouldnt be ) But what I need is essentially this:
I input a key string. What I need returned is every value that match that key.
Problem is that using regular dictionary it only returns the first value. Not all of them.
Should I be looping through the return to get all the values ? I tried looking but it doesnt seem like dictionary returns a tuple when having multiple values. Is there an easy way to output every match of the key ?
Use set here as they only contain unique items.
>>> lis = [{"abc":"movies"}, {"abc": "sports"}, {"abc": "music"}, {"xyz": "music"}, {"pqr":"music"}, {"pqr":"movies"},{"pqr":"sports"}, {"pqr":"news"}, {"pqr":"sports"}]
>>> s = set( val for dic in lis for val in dic.values())
>>> s
set(['movies', 'news', 'music', 'sports'])
Loop over this set to get the expected output:
for x in s:
print x
print len(s) #print the length of set after the for-loop
...
movies
news
music
sports
4
This line s = set( val for dic in lis for val in dic.values()) is roughly equivalent to:
s = set()
for dic in lis:
for val in dic.values():
s.add(val)
You can exploit the properties of set for this purpose; each element is unique and duplicates will be ignored.
uniqueValues = set(myDict.values())
By using set in this way, duplicate elements returned from .values() will be discarded. You can then print them as follows:
for value in uniqueValues:
print(value)
print(len(uniqueValues))
I think efficient way if dict is too large would be
countMap = {}
for v in a.itervalues():
countMap[v] = countMap.get(v,0) + 1
uni = [ k for k, v in a.iteritems() if countMap[v] == 1]
Note that this actually is a bruteforce:
l = a.values()
b = [x for x in a if l.count(a[x]) == 1]
Yes, you can use the defaultdict:
Sample code:
»»» from collections import defaultdict
»»» mydict = defaultdict(list)
»»» letters = ['a', 'b', 'a', 'c', 'a']
»»» for l in letters:
....: mydict[l].append('1')
....:
»»» mydict
Out[15]: defaultdict(<type 'list'>, {'a': ['1', '1', '1'], 'c': ['1'], 'b': ['1']})
If you need the content to be initialised to something fancier, you can specify your own construction function as the first argument to defaultdict. Passing context-specific arguments to that constructor might be tricky though.
The solution provided by m01 is cool and all but I believe it's worth mentionning that we can do that with a plain dict object..
mydict = dict()
letters = ['a', 'b', 'a', 'c', 'a']
for l in letters:
mydict.setdefault(l, []).append('1')
the result should be the same. You'll have a default dict instead of using a subclass. It really depends on what you're looking for. My guess is that the big problem with my solution is that it will create a new list even if it is not needed.
The defaultdict object has the advantage to create a new object only when something is missing. This solution has the advantage to be a simple dict without nothing special.
Edit
After thinking about it, I found out that using setdefault on a defaultdict will work as expected. But it's not yet good enough to say that a plain old dict should be used instead. There are cases where having a dict is important. To make it short, an invalid key on a dict will raise a KeyError. A defaultdict will return a default value.
As an example, there is the traversal algorithm that stops whenever it catches a KeyError or it traversed a whole path. With a defaultdict, you'd have to raise yourself the KeyError in case of errors.