If you have 100 million numbers like in the question, then this is actually manageable in-memory (it takes about 0.5 GB).
As DSM pointed out, this can be done with the standard modules in an efficient way:
>>> import array
>>> a = array.array('I', xrange(10**8)) # a.itemsize indicates 4 bytes per element => about 0.5 GB
>>> import random
>>> random.shuffle(a)
It is also possible to use the third-party NumPy package, which is the standard Python tool for managing arrays in an efficient way:
>>> import numpy
>>> ids = numpy.arange(100000000, dtype='uint32') # 32 bits is enough for numbers up to about 4 billion
>>> numpy.random.shuffle(ids)
(this is only useful if your program already uses NumPy, as the standard module approach is about as efficient).
Both method take about the same amount of time on my machine (maybe 1 minute for the shuffling), but the 0.5 GB they use is not too big for current computers.
PS: There are too many elements for the shuffling to be really random because there are way too many permutations possible, compared to the period of the random generators used. In other words, there are fewer Python shuffles than the number of possible shuffles!
Answer from Eric O. Lebigot on Stack OverflowIf you have 100 million numbers like in the question, then this is actually manageable in-memory (it takes about 0.5 GB).
As DSM pointed out, this can be done with the standard modules in an efficient way:
>>> import array
>>> a = array.array('I', xrange(10**8)) # a.itemsize indicates 4 bytes per element => about 0.5 GB
>>> import random
>>> random.shuffle(a)
It is also possible to use the third-party NumPy package, which is the standard Python tool for managing arrays in an efficient way:
>>> import numpy
>>> ids = numpy.arange(100000000, dtype='uint32') # 32 bits is enough for numbers up to about 4 billion
>>> numpy.random.shuffle(ids)
(this is only useful if your program already uses NumPy, as the standard module approach is about as efficient).
Both method take about the same amount of time on my machine (maybe 1 minute for the shuffling), but the 0.5 GB they use is not too big for current computers.
PS: There are too many elements for the shuffling to be really random because there are way too many permutations possible, compared to the period of the random generators used. In other words, there are fewer Python shuffles than the number of possible shuffles!
Maybe something like (won't be consecutive, but will be unique):
from uuid import uuid4
def unique_nums(): # Not strictly unique, but *practically* unique
while True:
yield int(uuid4().hex, 16)
# alternative yield uuid4().int
unique_num = unique_nums()
next(unique_num)
next(unique_num) # etc...
Which method is best for generating N unique numbers from a small range
Does random.shuffle produce unique items if the input list already has duplicates
How to handle an OverflowError when using range(very_large_number)
Instead of picking random integers, shuffle a list and pick the first two items:
import random
choices = ['Candy', 'Steak', 'Vegetables']
random.shuffle(choices)
item1, item2 = choices[:2]
Because we shuffled a list of possible choices first, then picked the first two, you can guarantee that item1 and item2 are never equal to one another.
Using random.shuffle() leaves the option open to do something with the remaining choices; you only have 1 here, but in a larger set you can continue to pick items that have so far not been picked:
choices = list(range(100))
random.shuffle(choices)
while choices:
if input('Do you want another random number? (Y/N)' ).lower() == 'n':
break
print(choices.pop())
would give you 100 random numbers without repeating.
If all you need is a random sample of 2, use random.sample() instead:
import random
choices = ['Candy', 'Steak', 'Vegetables']
item1, item2 = random.sample(choices, 2)
You can use the random module in Python to do the heavy lifting for you, specifically random.sample():
>>> import random
>>> random.sample(['candy', 'steak', 'vegetable'], 2)
['vegetable', 'steak']
Every number from 1,2,5,6,9,10... is divisible by 4 with remainder 1 or 2.
>>> ','.join(str(i) for i in xrange(100) if i % 4 in (1,2))
'1,2,5,6,9,10,13,14,...'
>>> ','.join('{},{}'.format(i, i + 1) for i in range(1, 100, 4))
'1,2,5,6,9,10,13,14,17,18,21,22,25,26,29,30,33,34,37,38,41,42,45,46,49,50,53,54,57,58,61,62,65,66,69,70,73,74,77,78,81,82,85,86,89,90,93,94,97,98'
That was a quick and quite dirty solution.
Now, for a solution that is suitable for different kinds of progression problems:
def deltas():
while True:
yield 1
yield 3
def numbers(start, deltas, max):
i = start
while i <= max:
yield i
i += next(deltas)
print(','.join(str(i) for i in numbers(1, deltas(), 100)))
And here are similar ideas implemented using itertools:
from itertools import cycle, takewhile, accumulate, chain
def numbers(start, deltas, max):
deltas = cycle(deltas)
numbers = accumulate(chain([start], deltas))
return takewhile(lambda x: x <= max, numbers)
print(','.join(str(x) for x in numbers(1, [1, 3], 100)))
It seems like you're mixing up your different variables -- f is what you want the length to be, l is just the number 2, and the way you're comparing those two has nothing to do with the actual input entered by the user, which is my_list.
Using variable names that indicate their meaning might make it easier to keep it all straight:
num_count = int(input("Length of string of numbers: "))
num_list = input('Enter numbers in the string, separated by spaces: ').split()
if len(num_list) == num_count:
print(f"there are {len(set(num_list))} different numbers")
else:
print("incorrect string length")
In the above code, num_count is the count of how many (non-unique) numbers you expect them to input, and num_list is the actual list. To figure out if the list is the expected length, compare num_count to len(num_list).
Note that since all you're doing is looking for unique values, converting the strings in num_list to int is not necessary (whether or not you use a set as I've done here).
You will most likely be better off using another function that ultimately has a while loop. This will make sure that when the user is giving the input that if anything is malformed you can then parse it checking and finally making sure to prompt the user again.
For example:
f=int(input("String of numbers: "))
my_list = input('Enter numbers in the string, separated by spaces: ').split()
list_of_integers=[]
l=len(str(list_of_integers))
for i in my_list:
list_of_integers.append((i))
mylist = list(dict.fromkeys(list_of_integers))
for i in range(f):
# XXX Here call your "input-function"
get_user_input(i, l)
def get_user_input(user_len, len):
while True user_len != len:
print('Incorrect Input')
user_len = int(input("String of numbers: "))
return
This is not exactly a working example but with what you have you get the idea that you want to do a while loop until your inputs match.