Hiya;

Part of a project I'm doing involves reading 200k+ line csv files and using keywords to filter those lines into different types of messages which will then be examined further

I was reading up on how to do this, and the regular expression re module appeared to be the way forward. however I did some testing comparing the re.match function to the is this string in that string function, and the former was several times slower. If Regex is so much slower than normal string matching, why/when should you use it/ or did I use regex wrong and its actually much faster?

results:

regex: 275.0342330016417

normal string matching: 44.675843185239046

test code:

import random
import re
from timeit import timeit

alp_str="abcdefghijklmnopqrstuvwxyz"


result=[]
counter=200000
while counter > 0:
    example=""
    length=50
    while length > 0:
        example+=(alp_str[random.randrange(25)])
        length-=1
    result.append(example)
    counter-=1

text='search'

def re_find(strings, text):
    num=0
    for string in strings:
        num+=1
        if re.match(text, string):
            pass

def best_find(strings, text):
    num=0
    for string in strings:
        num+=1
        if text in string:
            pass



print (timeit("re_find(result, text)","from __main__ import re_find;from __main__ import result;from __main__ import text",number=1000))  
print (timeit("best_find(result, text)","from __main__ import best_find;from __main__ import result;from __main__ import text",number=1000))