With pandas.DataFrame.to_csv you can write the columns and the index to a file:
import numpy as np
import pandas as pd
A = np.random.randint(0, 10, size=36).reshape(6, 6)
names = [_ for _ in 'abcdef']
df = pd.DataFrame(A, index=names, columns=names)
df.to_csv('df.csv', index=True, header=True, sep=' ')
will give you the following df.csv file:
a b c d e f
a 1 5 5 0 4 4
b 2 7 5 4 0 9
c 6 5 6 9 7 0
d 4 3 7 9 9 3
e 8 1 5 1 9 0
f 2 8 0 0 5 1
Answer from bmu on Stack OverflowWith pandas.DataFrame.to_csv you can write the columns and the index to a file:
import numpy as np
import pandas as pd
A = np.random.randint(0, 10, size=36).reshape(6, 6)
names = [_ for _ in 'abcdef']
df = pd.DataFrame(A, index=names, columns=names)
df.to_csv('df.csv', index=True, header=True, sep=' ')
will give you the following df.csv file:
a b c d e f
a 1 5 5 0 4 4
b 2 7 5 4 0 9
c 6 5 6 9 7 0
d 4 3 7 9 9 3
e 8 1 5 1 9 0
f 2 8 0 0 5 1
Numpy will handle n-dimensional arrays fine, but many of the facilities are limited to 2-dimensional arrays. Not even sure how you want the output file to look.
Many people who would wish for named columns overlook the recarray() capabilities of numpy. Good stuff to know, but that only "names" one dimension.
For two dimensions, Pandas is very cool.
In [275]: DataFrame.from_items([('A', [1, 2, 3]), ('B', [4, 5, 6])],
.....: orient='index', columns=['one', 'two', 'three'])
Out[275]:
one two three
A 1 2 3
B 4 5 6
If output is the only problem you are trying to solve here, I'd probably just stick with a few lines of hand coded magic as it will be less weighty than installing another package for one feature.
Ever since Numpy 1.7.0, three parameters have been added to numpy.savetxt for exactly this purpose: header, footer and comments. So the code to do as you wanted can easily be written as:
import numpy
a = numpy.array([[0.0,1.630000e+01,1.990000e+01,1.840000e+01],
[1.0,1.630000e+01,1.990000e+01,1.840000e+01],
[2.0,1.630000e+01,1.990000e+01,1.840000e+01]])
fmt = ",".join(["%s"] + ["%10.6e"] * (a.shape[1]-1))
numpy.savetxt("temp", a, fmt=fmt, header="SP,1,2,3", comments='')
Note: this answer was written for an older version of numpy, relevant when the question was written. With modern numpy, makhlaghi's answer provides a more elegant solution.
Since numpy.savetxt can also write to file objects, you can open the file youself and write your header before the data:
import numpy
a = numpy.array([[0.0,1.630000e+01,1.990000e+01,1.840000e+01],
[1.0,1.630000e+01,1.990000e+01,1.840000e+01],
[2.0,1.630000e+01,1.990000e+01,1.840000e+01]])
fmt = ",".join(["%s"] + ["%10.6e"] * (a.shape[1]-1))
# numpy.savetxt, at least as of numpy 1.6.2, writes bytes
# to file, which doesn't work with a file open in text mode. To
# work around this deficiency, open the file in binary mode, and
# write out the header as bytes.
with open('final.csv', 'wb') as f:
f.write(b'SP,1,2,3\n')
#f.write(bytes("SP,"+lists+"\n","UTF-8"))
#Used this line for a variable list of numbers
numpy.savetxt(f, a, fmt=fmt, delimiter=",")
Maybe this can help (if you don't want to use numpy):
headers = ['foo', 'bar', 'baz', 'other']
l = len(headers)
arr = [["xxx" for i in range(l)] for j in range(l)]
# adding top row
arr = [headers] + arr
# adding first column
headers_mod = ['Title'] + headers
new_arr = [[headers_mod[i]]+arr[i] for i in range(l+1)]
for i in new_arr:
print(*i)
gives you the output as:
Title foo bar baz other
foo xxx xxx xxx xxx
bar xxx xxx xxx xxx
baz xxx xxx xxx xxx
other xxx xxx xxx xxx
Otherwise, when dealing with array manipulations in python try going with numpy, pandas, as they provide better operations like by giving option for axis, transpose, etc.
numpy is excellent for tables, but for a labeled table like this pandas might be better for your needs.
Solution using numpy:
import numpy as np
# The header that i want to add
headers = ['foo', 'bar', 'baz', 'other']
ll = len(headers)+1
data = [['xxx' for _ in range(ll)] for j in range(ll)]
data = np.array(data, dtype=object)
data[0,0] = 'Title'
data[0,1:] = headers
data[1:,0] = headers
print(data)
prints
[['Title' 'foo' 'bar' 'baz' 'other']
['foo' 'xxx' 'xxx' 'xxx' 'xxx']
['bar' 'xxx' 'xxx' 'xxx' 'xxx']
['baz' 'xxx' 'xxx' 'xxx' 'xxx']
['other' 'xxx' 'xxx' 'xxx' 'xxx']]
Setting dtype to object allows your array to mix strings and other data types you might want to use. If your data is just strings then you can use 'UN' as the dtype, where N is the longest string you plan to use. (Numpy, when making an all string array automatically picks your longest string as the maximum length for the strings, which is fine unless your strings are all shorter than the headers you plan to add.)
Alternate version of the above code:
import numpy as np
# The header that i want to add
headers = ['foo', 'bar', 'baz', 'other']
# Add Title to headers to simply later assignment
headers = ['Title'] + headers
ll = len(headers)
data = [['xxx' for _ in range(ll)] for j in range(ll)]
data = np.array(data)
data[0,:] = headers
data[:,0] = headers
print(data)
pandas, on the other hand, is explicitly designed to handle headers
import numpy as np, pandas as pd
# The header that i want to add
headers = ['foo', 'bar', 'baz', 'other']
ll = len(headers) + 1
data = [['xxx' for _ in range(ll)] for j in range(ll)]
data = np.array(data)
data = pd.DataFrame(data[1:,1:], columns=headers, index=headers)
data.columns.name = 'Title'
data.loc['foo','bar'] = 'yes'
print(data)
print('')
print(data['bar'])
print('')
print(data.loc['foo',:])
prints
Title foo bar baz other
foo xxx yes xxx xxx
bar xxx xxx xxx xxx
baz xxx xxx xxx xxx
other xxx xxx xxx xxx
foo yes
bar xxx
baz xxx
other xxx
Name: bar, dtype: object
Title
foo xxx
bar yes
baz xxx
other xxx
Name: foo, dtype: object
You can achieve this with pandas.
import pandas as pd
matrix = [...] # your ndarray
matrix = pd.DataFrame(data=matrix, columns=["summary", "age", "label"])
You can fiddle the dtype:
>>> a = np.arange(12).reshape(4, 3)
>>>
>>> dt = a.dtype
>>>
>>> ahead = a.view(np.dtype([('summary', dt), ('age', dt), ('label', dt)]))
>>>
>>> ahead
array([[(0, 1, 2)],
[(3, 4, 5)],
[(6, 7, 8)],
[(9, 10, 11)]],
dtype=[('summary', '<i8'), ('age', '<i8'), ('label', '<i8')])
>>> ahead['summary']
array([[0],
[3],
[6],
[9]])
But be warned that those composite dtype arrays are not very useful as far as I can tell:
>>> ahead @ ahead.T
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid data type for einsum
to give just one example.
1.) I would recommend storing column and row header information in a separate data structure. Numpy matrices can store mixed data types (in this case strings and floats), I try to avoid it. Mixing data types is messy and seems inefficient to me. If you want to, you can make your own class with your matrix data and header information in it. It seems like a cleaner solution to me.
2.) No, summaryMeansArray is set-up to have 11 rows and 6 columns. The first dimension of a matrix is the number of rows. You can get the transpose of summaryMeansArray with summaryMeansArray.T. When you are taking the mean of summary3dArray on the 0th axis, the next axis becomes the rows and the one after that the columns.
Edit: As per request, you can create a python list from a numpy array with the method tolist(). For instance,
newMeansArray = summaryMeansArray.tolist()
Then you can insert the column headers using
newMeansArray.insert(0,headers)
Inserting the row headers can be done with:
newMeansArray[i].insert(0,rowheader)
for each row i. Of course, if you've already inserted the column headers, then the counting for i starts with 1 rather than 0.
I agree with Justin Peel's answer, regarding question #1 (row/header labels).
I created my own class that allows me to decorate a matrix with extra data necessary to my task at hand (for example: row and column labels, a descriptive text for each row, or numerical properties of a row that are external to or independent of the matrix values).
My first solution that I used for almost 2 years was to have an object for each matrix row, where I would store each row's matrix values in a dictionary, with the dictionary key (ID) providing the second piece of information for that pair's matrix value. This was quite useful, especially for non-square matrices, and matrix manipulations and output were isolated cleanly.
However, I ran into a problem with this design: scalability. When using square, symmetric matrices, I needed 91 MB of memory for a 1000x1000 matrix, 327 MB of memory for a 2000x2000 matrix, and 1900 MB of memory for a 5000x5000 matrix. For my recent project that works on the order of 20000x20000 matrix entries, I will quickly and disastrously use up all of my workstation's 8GB of RAM and more.
My second solution was to have a single dictionary of (ID1,ID2)-->value mappings. Compared to my first solution, a 1000x1000 matrix required only 20 MB of memory. This solution also fails miserably in the scalability department, but in a different way, because the time to create and store C(1000+1,2)=500500 mappings was over 3 minutes, compared to 0.88 seconds when using my first design.
My third and current solution was to create a mapping between the numpy matrix row/column index and a matrix row/column label. Using numpy directly with a 5000x5000 matrix required 202 MB of memory on my system, a 10000x1000 matrix required 774 MB, and a 20000x2000 matrix required 3000 MB. A mapping of 20000 IDs to row/column indexes required 5 MB of memory on my system, which is negligible compared to the value matrix itself.
If one is processing only small matrices less than 100x100 elements, then my first solution will be quick and the implemented data structure will be easy to manipulate and extend. However, if you are thinking of large-scale processing, then I recommend the third solution.
To concatenate your header with the numpy array you'll need to have the array in binary form. Assuming data is an array of ints:
import struct
raw_data = packedHdr + struct.pack('i' * data.size, *data)
If your data is of a different kind, you need to specify it in st.pack. You can then unpack the resulting raw_data in the desired form.
Why not put headerVersion and formatEnum inside a numpy array, then try to concat?