It all depends on what you plan to do with the array. If all you're doing is creating arrays of simple data types and doing I/O, the array module will do just fine.
If, on the other hand, you want to do any kind of numerical calculations, the array module doesn't provide any help with that. NumPy (and SciPy) give you a wide variety of operations between arrays and special functions that are useful not only for scientific work but for things like advanced image manipulation or in general anything where you need to perform efficient calculations with large amounts of data.
Numpy is also much more flexible, e.g. it supports arrays of any type of Python objects, and is also able to interact "natively" with your own objects if they conform to the array interface.
Answer from dF. on Stack OverflowIt all depends on what you plan to do with the array. If all you're doing is creating arrays of simple data types and doing I/O, the array module will do just fine.
If, on the other hand, you want to do any kind of numerical calculations, the array module doesn't provide any help with that. NumPy (and SciPy) give you a wide variety of operations between arrays and special functions that are useful not only for scientific work but for things like advanced image manipulation or in general anything where you need to perform efficient calculations with large amounts of data.
Numpy is also much more flexible, e.g. it supports arrays of any type of Python objects, and is also able to interact "natively" with your own objects if they conform to the array interface.
Small bootstrapping for the benefit of whoever might find this useful (following the excellent answer by @dF.):
import numpy as np
from array import array
# Fixed size numpy array
def np_fixed(n):
q = np.empty(n)
for i in range(n):
q[i] = i
return q
# Resize with np.resize
def np_class_resize(isize, n):
q = np.empty(isize)
for i in range(n):
if i>=q.shape[0]:
q = np.resize(q, q.shape[0]*2)
q[i] = i
return q
# Resize with the numpy.array method
def np_method_resize(isize, n):
q = np.empty(isize)
for i in range(n):
if i>=q.shape[0]:
q.resize(q.shape[0]*2)
q[i] = i
return q
# Array.array append
def arr(n):
q = array('d')
for i in range(n):
q.append(i)
return q
isize = 1000
n = 10000000
The output gives:
%timeit -r 10 a = np_fixed(n)
%timeit -r 10 a = np_class_resize(isize, n)
%timeit -r 10 a = np_method_resize(isize, n)
%timeit -r 10 a = arr(n)
1 loop, best of 10: 868 ms per loop
1 loop, best of 10: 2.03 s per loop
1 loop, best of 10: 2.02 s per loop
1 loop, best of 10: 1.89 s per loop
It seems that array.array is slightly faster and the 'api' saves you some hassle, but if you need more than just storing doubles then numpy.resize is not a bad choice after all (if used correctly).
As mentioned in the title, preferably a more ELI answer if possible. Thank you!
I'm wondering if there is a use case where lists are better than numpy arrays besides storing multiple types of data in a relatively small/low dimension list (say i just want a quick-and-dirty list to store some constants i want to call on later?
I'm not well versed enough in how processors/memory work to know if I'm missing something else obvious. I know it won't make a huge performance impact anyways if you don't actually have a lot of math or large lists/arrays to work with, but I've been diving deeper into how numpy/pandas function at a basic level and it got me thinking about this.
To understand the differences between numpy and array, I ran a few more quantitative test.
What I have found is that, for my system (Ubuntu 18.04, Python3), array seems to be twice as fast at generating a large array from the range generator compared to numpy (although numpy's dedicated np.arange() seems to be much faster -- actually too fast, and perhaps it is caching something during tests), but twice as slow than using list.
However, quite surprisingly, array objects seems to be larger than the numpy counterparts.
Instead, the list objects are roughly 8-13% larger than array objects (this will vary with the size of the individual items, obviously).
Compared to list, array offers a way to control the size of the number objects.
So, perhaps, the only sensible use case for array is actually when numpy is not available.
For completeness, here is the code that I used for the tests:
import numpy as np
import array
import sys
num = int(1e6)
num_i = 100
x = np.logspace(1, int(np.log10(num)), num_i).astype(int)
%timeit list(range(num))
# 10 loops, best of 3: 32.8 ms per loop
%timeit array.array('l', range(num))
# 10 loops, best of 3: 86.3 ms per loop
%timeit np.array(range(num), dtype=np.int64)
# 10 loops, best of 3: 180 ms per loop
%timeit np.arange(num, dtype=np.int64)
# 1000 loops, best of 3: 809 µs per loop
y_list = np.array([sys.getsizeof(list(range(x_i))) for x_i in x])
y_array = np.array([sys.getsizeof(array.array('l', range(x_i))) for x_i in x])
y_np = np.array([sys.getsizeof(np.array(range(x_i), dtype=np.int64)) for x_i in x])
import matplotlib.pyplot as plt
plt.figure(figsize=(12, 6))
plt.plot(x, y_list, label='list')
plt.plot(x, y_array, label='array')
plt.plot(x, y_np, label='numpy')
plt.legend()
plt.show()

Yes, if you don't want another dependency in your code.
You first need to understand the difference between arrays and lists.
An array is a contiguous block of memory consisting of elements of some type (e.g. integers).
You cannot change the size of an array once it is created.
It therefore follows that each integer element in an array has a fixed size, e.g. 4 bytes.
On the other hand, a list is merely an "array" of addresses (which also have a fixed size).
But then each element holds the address of something else in memory, which is the actual integer that you want to work with. Of course, the size of this integer is irrelevant to the size of the array. Thus you can always create a new (bigger) integer and "replace" the old one without affecting the size of the array, which merely holds the address of an integer.
Of course, this convenience of a list comes at a cost: Performing arithmetic on the integers now requires a memory access to the array, plus a memory access to the integer itself, plus the time it takes to allocate more memory (if needed), plus the time required to delete the old integer (if needed). So yes, it can be slower, so you have to be careful what you're doing with each integer inside an array.
Your first example could be speed up. Python loop and access to individual items in a numpy array are slow. Use vectorized operations instead:
import numpy as np
x = np.arange(1000000).cumsum()
You can put unbounded Python integers to numpy array:
a = np.array([0], dtype=object)
a[0] += 1232234234234324353453453
Arithmetic operations compared to fixed-sized C integers would be slower in this case.
This may be an odd question but I was simply curious and googling only found me articles comparing python lists to numpy arrays. (edit: for clarity, I mean here that I haven't used numpy or the builtin arrays from This module, I am familiar with python lists, how to use them and how they are implemented) I haven't used either of these which is likely part of why I don't understand the differences but I've really never seen anyone talk about the usage of the standard library module for arrays in python, only using numpy arrays to improve efficiency though they both seem to have the same job, implementing C arrays in python.
I guess my basic question is this: What is the difference between standard library arrays and numpy arrays and why do people prefer to use numpy arrays for efficient data handling?