Yes, you can use a regular expression for this:
import re
output = re.sub(r'\d+', '', '123hello 456world')
print output # 'hello world'
Answer from Martin Konecny on Stack OverflowYes, you can use a regular expression for this:
import re
output = re.sub(r'\d+', '', '123hello 456world')
print output # 'hello world'
str.translate should be efficient.
In [7]: 'hello467'.translate(None, '0123456789')
Out[7]: 'hello'
To compare str.translate against re.sub:
In [13]: %%timeit r=re.compile(r'\d')
output = r.sub('', my_str)
....:
100000 loops, best of 3: 5.46 µs per loop
In [16]: %%timeit pass
output = my_str.translate(None, '0123456789')
....:
1000000 loops, best of 3: 713 ns per loop
Try something like below:
import string
import re
import nltk
from nltk.tokenize import TweetTokenizer
tweet = "first think another Disney movie, might good, it's kids movie. watch it, can't help enjoy it. ages love movie. first saw movie 10 8 years later still love it! Danny Glover superb could play"
def clean_text(text):
# remove numbers
text_nonum = re.sub(r'\d+', '', text)
# remove punctuations and convert characters to lower case
text_nopunct = "".join([char.lower() for char in text_nonum if char not in string.punctuation])
# substitute multiple whitespace with single whitespace
# Also, removes leading and trailing whitespaces
text_no_doublespace = re.sub('\s+', ' ', text_nopunct).strip()
return text_no_doublespace
cleaned_tweet = clean_text(tweet)
tt = TweetTokenizer()
print(tt.tokenize(cleaned_tweet))
output:
['first', 'think', 'another', 'disney', 'movie', 'might', 'good', 'its', 'kids', 'movie', 'watch', 'it', 'cant', 'help', 'enjoy', 'it', 'ages', 'love', 'movie', 'first', 'saw', 'movie', 'years', 'later', 'still', 'love', 'it', 'danny', 'glover', 'superb', 'could', 'play']
# Function for removing Punctuation from Text and It gives total no.of punctuation removed also
# Input: Function takes Existing fie name and New file name as string i.e 'existingFileName.txt' and 'newFileName.txt'
# Return: It returns two things Punctuation Free File opened in read mode and a punctuation count variable.
from nltk.tokenize import word_tokenize
import string
def removePunctuation(tokenizeSampleText, newFileName):
stringPun = list(string.punctuation)
with open(tokenizeSampleText, "r") as existingFile:
tokenize_existingFile = word_tokenize(existingFile.read())
count_pun = 0
words = []
for word in tokenize_existingFile:
if word in stringPun:
count_pun += 1
else:
words.append(word)
with open(newFileName, "w+") as puncRemovedFile:
puncRemovedFile.write(" ".join(words))
with open(newFileName, "r") as file:
yield file.read(), count_pun
punRemoved, punCount = removePunctuation(
"Macbeth.txt", "Macbeth-punctuationRemoved.txt"
)
print(f"Total Punctuation : {punCount}")
print(punRemoved)
Would this work for your situation?
>>> s = '12abcd405'
>>> result = ''.join([i for i in s if not i.isdigit()])
>>> result
'abcd'
This makes use of a list comprehension, and what is happening here is similar to this structure:
no_digits = []
# Iterate through the string, adding non-numbers to the no_digits list
for i in s:
if not i.isdigit():
no_digits.append(i)
# Now join all elements of the list with '',
# which puts all of the characters together.
result = ''.join(no_digits)
As @AshwiniChaudhary and @KirkStrauser point out, you actually do not need to use the brackets in the one-liner, making the piece inside the parentheses a generator expression (more efficient than a list comprehension). Even if this doesn't fit the requirements for your assignment, it is something you should read about eventually :) :
>>> s = '12abcd405'
>>> result = ''.join(i for i in s if not i.isdigit())
>>> result
'abcd'
And, just to throw it in the mix, is the oft-forgotten str.translate which will work a lot faster than looping/regular expressions:
For Python 2:
from string import digits
s = 'abc123def456ghi789zero0'
res = s.translate(None, digits)
# 'abcdefghizero'
For Python 3:
from string import digits
s = 'abc123def456ghi789zero0'
remove_digits = str.maketrans('', '', digits)
res = s.translate(remove_digits)
# 'abcdefghizero'
Change
for word in word_list:
word = punctuation.sub("", word)
to
word_list = [punctuation.sub("", word) for word in word_list]
Assignment to word in the for-loop above, simply changes the value referenced by this temporary variable. It does not alter word_list.
You're not updating your word list. Try
for i, word in enumerate(word_list):
word_list[i] = punctuation.sub("", word)
Remember that although word starts off as a reference to the string object in the word_list, assignment rebinds the name word to the new string object returned by the sub function. It doesn't change the originally referenced object.