If need test substrings use Series.str.contains:
conditions = [
(df['column'].str.contains('one')),
(df['column'].str.contains('two')),
(df['column'].str.contains('three')),
(df['column'].str.contains('other'))]
If need exact match use Series.eq or ==:
conditions = [
(df['column'].eq('one')),
(df['column'].eq('two')),
(df['column'].eq('three')),
(df['column'].eq('other'))]
conditions = [
(df['column'] == 'one'),
(df['column'] == 'two'),
(df['column'] == 'three'),
(df['column'] == 'other')]
print (new_df)
column Int
0 one 1
1 ones 1
2 other 0
3 two 2
4 twos 2
5 others 0
6 three 3
7 threes 3
Create a mapping of each country -> area, then use explode and map to apply the mapping and finally, use groupby and apply to rebuild the list:
geo = {'Europe': Europe, 'North_America': North_America, 'Asia': Asia}
mapping = {country: area for area, countries in geo.items() for country in countries}
df['geo'] = df['location'].str.split(', ').explode().map(mapping) \
.groupby(level=0).apply(', '.join)
Output:
>>> df
location geo
0 London, Paris Europe, Europe
1 Hong Kong Asia
2 London, New York Europe, North_America
3 Singapore, Toroto Asia, North_America
4 Boston North_America
By using NumPy library together with python for loops we can get the results. At first we combine lists of country's cities together and then create another list named continents which length is the same as the created list of cities:
import numpy as np
import pandas as pd
continents = ["Europe"] * len(Europe) + ["North_America"] * len(North_America) + ["Asia"] * len(Asia)
countries = Europe + North_America + Asia
locations = data['location']
Then for each city, even for each in the combinations, we find its index in the created country list. Then we create a list for number of commas in each of that combinations for using to create the desired output with commas:
corsp = []
comma_nums = []
for i in locations:
for j, k in enumerate(i.split(', ')):
corsp.append(np.where(np.array(countries) == k)[0][0])
comma_nums.append(j)
continents list will be reordered and modified by created index list. Then its arguments combined in list format as the combination style which where in locations, and then the lists convert to strings as they are needed for the output:
reordered_continents = [continents[i] for i in corsp]
mod_continents = []
iter = 0
f = 1
for i in comma_nums:
mod_continents.append(reordered_continents[iter:i + f])
iter = i + f
f = iter + 1
for i, j in enumerate(mod_continents):
if len(j) > 1:
for k in j:
mod_continents[i] = ', '.join(j)
else:
mod_continents[i] = ''.join(j)
df['geo'] = mod_continents
Why not simple comprehension lists:
[u[:2] for u in HHMM]
#Out[39]: ['00', '01', '02', '03', '04', '05', '06', '07', '08', '09']
[u[-2:] for u in HHMM]
#Out[40]: ['00', '01', '02', '03', '04', '05', '06', '07', '08', '09']
Example:
for i in range(len(HHMM)):
hour = HHMM[i][0:2]
minute = HHMM[i][2:]
print('Time: {}:{}'.format(hour, minute))
Output:
Time: 00:00
Time: 00:01
Time: 00:02
Time: 00:03
Time: 00:04
Time: 00:05
Time: 00:06
Time: 00:07
Time: 00:08
Time: 00:09
EDIT:
HHMM = ['0000', '0001', '0002', '0003', '0004', '0005', '0006', '0007', '0008', '0009']
hours = []
minutes = []
for i in range(len(HHMM)):
hours.append(HHMM[i][0:2])
minutes.append(HHMM[i][2:])
print('Hours: {}'.format(list(hours)))
print('Minutes: {}'.format(list(minutes)))
We can use np.core.defchararray.find to find the position of foo string in each element of bar, which would return -1 if not found. Thus, it could be used to detect whether foo is present in each element or not by checking for -1 on the output from find. Finally, we would use np.flatnonzero to get the indices of matches. So, we would have an implementation, like so -
np.flatnonzero(np.core.defchararray.find(bar,foo)!=-1)
Sample run -
In [91]: bar
Out[91]:
array(['aaa', 'aab', 'aca'],
dtype='|S3')
In [92]: foo
Out[92]: 'aa'
In [93]: np.flatnonzero(np.core.defchararray.find(bar,foo)!=-1)
Out[93]: array([0, 1])
In [94]: bar[2] = 'jaa'
In [95]: np.flatnonzero(np.core.defchararray.find(bar,foo)!=-1)
Out[95]: array([0, 1, 2])
Look at some examples of using in:
In [19]: bar = np.array(["aaa", "aab", "aca"])
In [20]: 'aa' in bar
Out[20]: False
In [21]: 'aaa' in bar
Out[21]: True
In [22]: 'aab' in bar
Out[22]: True
In [23]: 'aab' in list(bar)
It looks like in when used with an array works as though the array was a list. ndarray does have a __contains__ method, so in works, but it is probably simple.
But in any case, note that in alist does not check for substrings. The strings __contains__ does the substring test, but I don't know any builtin class that propagates the test down to the component strings.
As Divakar shows there is a collection of numpy functions that applies string methods to individual elements of an array.
In [42]: np.char.find(bar, 'aa')
Out[42]: array([ 0, 0, -1])
Docstring:
This module contains a set of functions for vectorized string operations and methods. The preferred alias fordefchararrayisnumpy.char.
For operations like this I think the np.char speeds are about same as with:
In [49]: np.frompyfunc(lambda x: x.find('aa'), 1, 1)(bar)
Out[49]: array([0, 0, -1], dtype=object)
In [50]: np.frompyfunc(lambda x: 'aa' in x, 1, 1)(bar)
Out[50]: array([True, True, False], dtype=object)
Further tests suggest that the ndarray __contains__ operates on the flat version of the array - that is, shape doesn't affect its behavior.
You have put the NumPy tag on your Question, so i'll assume you want NumPy syntax, which the answer before mine doesn't use.
If in fact you wish to use NumPy, then you likely don't want the strings in your array, otherwise you will also have to represent your floats as strings.
What you are looking for is the NumPy syntax to access elements of a 2D array by row (and exclude the first column).
That syntax is:
M[row_index,1:] # selects all but 1st col from row given by 'row_index'
W/r/t the second scenario in your Question--selecting non-adjacent columns:
M[row_index,[0,2]] # selects 1st & 3rd cols from row given by 'row_index'
The small complication in your Question is just that you want to use a string for row_index, so it's necessary to remove the strings (so you can create a 2D NumPy array of floats), replace them with numerical row indices and then create a look-up table to map the the strings with the numerical row indices:
>>> import numpy as NP
>>> # create a look-up table so you can remove the strings from your python nested list,
>>> # which will allow you to represent your data as a 2D NumPy array with dtype=float
>>> keys
['foo', 'bar', 'noo', 'tar', 'boo']
>>> values # 1D index array comprised of one float value for each unique string in 'keys'
array([0., 1., 2., 3., 4.])
>>> LuT = dict(zip(keys, values))
>>> # add an index to data by inserting 'values' array as first column of the data matrix
>>> A = NP.hstack((vals, A))
>>> A
NP.array([ [ 0., .567, .611],
[ 1., .469, .479],
[ 2., .22, .269],
[ 3., .48, .508],
[ 4., .324, .324] ])
>>> # so now to look up an item, by 'key':
>>> # write a small function to perform the look-ups:
>>> def select_row(key):
return A[LuT[key],1:]
>>> select_row('foo')
array([ 0.567, 0.611])
>>> select_row('noo')
array([ 0.22 , 0.269])
The second scenario in your Question: what if the index column changes?
>>> # e.g., move index to column 1 (as in your Q)
>>> A = NP.roll(A, 1, axis=1)
>>> A
array([[ 0.611, 1. , 0.567],
[ 0.479, 2. , 0.469],
[ 0.269, 3. , 0.22 ],
[ 0.508, 4. , 0.48 ],
[ 0.324, 5. , 0.324]])
>>> # the original function is changed slightly, to select non-adjacent columns:
>>> def select_row2(key):
return A[LuT[key],[0,2]]
>>> select_row2('foo')
array([ 0.611, 0.567])
First, the vector of first elements is
dv = dd[:,0]
(python is 0-indexed)
Second, to walk the array (and store in a dict, for example) you write:
dc = {}
ind = 0 # this corresponds to the column with the names
for row in dd:
dc[row[ind]] = row[1:]
If you want to achieve this by using numpy only, then try this:
import numpy as np
a = np.loadtxt('input.txt', dtype=str, delimiter=',')
msk = a[:, 1] == 'cb'
b = a[msk, :]
np.savetxt('out.txt', b, delimiter=',', fmt='%s', header='col1,col2,col3')
Edit
Since the second column of b is now redundant, you can avoid writing it by:
np.savetxt('out.txt', b[:, 0:3:2], delimiter=',', fmt='%s', header='col1,col3')
As is obvious, a and b are string arrays. If, however, you need to do some further processing of the 1st and last columns using their native types, then use the following variables:
cb_col1 = b[:, 0].astype(int)
cb_col3 = b[:, 2].astype(float)
Your sample, corrected for the extra column in line 5:
In [4]: txt = """col1, col2, col3
...: -3,cb,7.9356
...: -3,mo,13.0402
...: -2,cb,12.9229
...: -2,cb,13.2063
...: -1,va,12.7535"""
Using the full power of genfromtxt I can load it as a structured array:
In [5]: data = np.genfromtxt(txt.splitlines(), delimiter=',', dtype=None, names=True, encoding=None)
In [6]: data
Out[6]:
array([(-3, 'cb', 7.9356), (-3, 'mo', 13.0402), (-2, 'cb', 12.9229),
(-2, 'cb', 13.2063), (-1, 'va', 12.7535)],
dtype=[('col1', '<i8'), ('col2', '<U2'), ('col3', '<f8')])
The 'col2' field is already string dtype, so can be tested against a string like 'cb':
In [7]: data['col2']
Out[7]: array(['cb', 'mo', 'cb', 'cb', 'va'], dtype='<U2')
In [8]: idx=data['col2']=='cb'
In [9]: idx
Out[9]: array([ True, False, True, True, False])
we can easily select the records with just 'cb' from data.
In [10]: data[idx]
Out[10]:
array([(-3, 'cb', 7.9356), (-2, 'cb', 12.9229), (-2, 'cb', 13.2063)],
dtype=[('col1', '<i8'), ('col2', '<U2'), ('col3', '<f8')])
But if all you want to do is select lines with 'cb' and write them back out, it might easiest to use plain Python file read/write.
Use readlines to fetch all lines:
In [11]: txt1 = txt.splitlines()
In [12]: txt1
Out[12]:
['col1, col2, col3',
'-3,cb,7.9356',
'-3,mo,13.0402',
'-2,cb,12.9229',
'-2,cb,13.2063',
'-1,va,12.7535']
and use a simple string find to identify lines that contain 'cb':
In [13]: txt1[1].find('cb')
Out[13]: 3
In [14]: txt1[0].find('cb')
Out[14]: -1
In [15]: [row for row in txt1 if row.find('cb')!=-1]
Out[15]: ['-3,cb,7.9356', '-2,cb,12.9229', '-2,cb,13.2063']
finally write these lines back out to a file.
print(a[arr_index])
not array_index!!
a = np.array(['apple', 'orange', 'apple', 'banana'])
arr_index = np.where(a == 'apple')
print(arr_index)
print(a[arr_index])
I believe an easier way is to just do :
A = np.array(['apple', 'orange', 'apple', 'banana'])
arr_index = np.where(A == 'apple')
print(arr_index)
And you get:
(array([0, 2]),)