You're iterating through the elements within the DataFrame, in which case I'm assuming it's type str (or being coerced to str when you replace). str.replace doesn't have an argument for inplace=....
You should be doing this instead:
dataset['ver'] = dataset['ver'].str.replace('.', '')
Answer from r.ook on Stack OverflowYou're iterating through the elements within the DataFrame, in which case I'm assuming it's type str (or being coerced to str when you replace). str.replace doesn't have an argument for inplace=....
You should be doing this instead:
dataset['ver'] = dataset['ver'].str.replace('.', '')
Sander van den Oord in the comments is quite correct to point out:
dataset['ver'].replace("[.]","", inplace=True, regex=True)
This is the way we do operations on a column in Pandas because in general, Pandas tries to optimize over for loops. The Pandas developers consider for loops the among least desirable pattern for row-wise operations in Python (see here.)
Python Pandas Commands .replace() Not Working despite inplace = True - Stack Overflow
python - How to apply pandas.DataFrame.replace on selected columns with inplace = True? - Stack Overflow
'in-place' string modifications in Python - Stack Overflow
python - Why pandas DataFrame replace method does not work (inplace=True argument is used) - Stack Overflow
I've run some tests here and I think I've found the answer.
My var self.name has a <class 'PyQt4.QtCore.QString'> type, since I'm getting it from a QtWidget.
Someone, please, correct me if I'm wrong but from the tests I run and from the docs (http://doc.qt.io/qt-5/qstring.html#replace) it seems that the replace method in <class 'PyQt4.QtCore.QString'> happens inplace. Whereas for the Python str, it does not since python strings are immutable.
So, in short:
- Python
str:inplace=False(Python strings are immutable) PyQt4.QtCore.QString:inplace=True
Anyway, hope this can be helpful
When you do self.name you are actually replacing in place on the objects name string.
However when you do str(self.name) you are replacing on the new object created by str which is not self.name. Hence self.name remains unchanged.
When you select the columns for replacement with df[col_list], a slice (a copy) of your dataframe is created. The copy is updated, but never written back into the original dataframe.
You should either replace one column at a time or use nested dictionary mapping:
df.replace(to_replace={'col1' : {99 : 0}, 'col2' : {99 : 0}},
inplace=True)
The nested dictionary for to_replace can be generated automatically:
d = {col : {99:0} for col in col_list}
You can use replace with loc. Here is a slightly modified version of your sample df:
d = {'col1':[99,99,9],'col2':[99,5,6],'col3':[7,None,99]}
df = pd.DataFrame(data=d)
col_list = ['col1','col2']
df.loc[:, col_list] = df.loc[:, col_list].replace(99,0)
You get
col1 col2 col3
0 0 0 7.0
1 0 5 NaN
2 9 6 99.0
Here is a nice explanation for similar issue.
Don't use a string, use something mutable like bytearray:
#!/usr/bin/python
s = bytearray("my dog has fleas")
for n in xrange(len(s)):
s[n] = chr(s[n]).upper()
print s
Results in:
MY DOG HAS FLEAS
Edit:
Since this is a bytearray, you aren't (necessarily) working with characters. You're working with bytes. So this works too:
s = bytearray("\x81\x82\x83")
for n in xrange(len(s)):
s[n] = s[n] + 1
print repr(s)
gives:
bytearray(b'\x82\x83\x84')
If you want to modify characters in a Unicode string, you'd maybe want to work with memoryview, though that doesn't support Unicode directly.
The Python analog of your C:
for(int i = 0; i < strlen(s); i++)
{
s[i] = F(s[i]);
}
would be:
s = "".join(F(c) for c in s)
which is also very expressive. It says exactly what is happening, but in a functional style rather than a procedural style.
df['funding_total_usd'].replace({'-',0})Does not work.
When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:
df.an_operation(inplace=True)
When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:
df = df.an_operation(inplace=False)
In pandas, is inplace = True considered harmful, or not?
TLDR; Yes, yes it is.
inplace, contrary to what the name implies, often does not prevent copies from being created, and (almost) never offers any performance benefitsinplacedoes not work with method chaininginplacecan lead toSettingWithCopyWarningif used on a DataFrame column, and may prevent the operation from going though, leading to hard-to-debug errors in code
The pain points above are common pitfalls for beginners, so removing this option will simplify the API.
I don't advise setting this parameter as it serves little purpose. See this GitHub issue which proposes the inplace argument be deprecated api-wide.
It is a common misconception that using inplace=True will lead to more efficient or optimized code. In reality, there are absolutely no performance benefits to using inplace=True. Both the in-place and out-of-place versions create a copy of the data anyway, with the in-place version automatically assigning the copy back.
inplace=True is a common pitfall for beginners. For example, it can trigger the SettingWithCopyWarning:
df = pd.DataFrame({'a': [3, 2, 1], 'b': ['x', 'y', 'z']})
df2 = df[df['a'] > 1]
df2['b'].replace({'x': 'abc'}, inplace=True)
# SettingWithCopyWarning:
# A value is trying to be set on a copy of a slice from a DataFrame
Calling a function on a DataFrame column with inplace=True may or may not work. This is especially true when chained indexing is involved.
As if the problems described above aren't enough, inplace=True also hinders method chaining. Contrast the working of
result = df.some_function1().reset_index().some_function2()
As opposed to
temp = df.some_function1()
temp.reset_index(inplace=True)
result = temp.some_function2()
The former lends itself to better code organization and readability.
Another supporting claim is that the API for set_axis was recently changed such that inplace default value was switched from True to False. See GH27600. Great job devs!
Use fileinput.FileInput, with inplace=True. printed line will be used as a replacement string for each line.
myfile = fileinput.FileInput("inputRegex.txt", inplace=True)
for line in myfile:
line = re.sub(r"[+-]? *(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?",
"foundValue",
line.rstrip())
print(line)
UPDATE
re.sub can accept a function as replacement. It will be called with match object and the return value of the function is used as a replacement string.
The following is slightly modified version to use captured groups (to use in replacement function).
line = re.sub(r"([+-]? *)(\d+(?:\.\d*)?|\.\d+)([eE][+-]?\d+)?",
lambda m: m.group(1) + re.sub('(\..{4}).*', r'\1', m.group(2)) + (m.group(3) or ''),
line.rstrip())
import fileinput
import re
myfile = open("inputRegex.txt", "r")
def changePrecision(matchObj):
return str(round(float(matchObj.group(0).replace(" ","")),4))
for line in myfile:
newLine = re.sub(r"[+-]? *(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?", changePrecision, line)
print newLine
I hope this is what you are looking for
Generally, you can do things to dataframes two ways in pandas:
df.<do_thing>(<args>, inplace=True)
or
df = df.<do_thing>(<args>)
My intuition is that the second way is much worse, because python essentially applies the transformation to a whole copy of df in memory, before then overwriting df. Whereas using 'inplace' sounds like it would instead do the thing in parts to the existing object in memory.
Is my intuition for these two syntaxes correct? If not, how can you truly modify a large dataframe inplace in memory without requiring double its size in memory to apply the operation?