Csv is already built into python. Don't use pandas unless you have a specific reason to. Especially if you're new to python. Also, you should use Google before posting questions. "Python csv" obviously informs you of the built in csv library and lots of examples https://docs.python.org/3/library/csv.html More on reddit.com

In theory, you could. If you only have lines without \n, " and , (or whatever separator you have), it shouldn't be that difficult. But there is already the csv module, which is rather simple, so you might want to use that. More on reddit.com

Sure. csv just stands for "comma separated value", so you'll be using things like .split to split a string on commas, .join for constructing the line to write back to the file, and things like basic file reading and writing. >>> line = 'data1,data2,data2' >>> line.split(',') ['data1', 'data2', 'data3'] >>> data = ['data1', 'data2', 'data3'] >>> ','.join(data) 'data1,data2,data3' You'll have to be careful about values that themselves contain commas or newlines (for example addresses) so check to be sure whether you have to worry about those in your project data or not. More on reddit.com

tl;dr: Your data file likely includes non-UTF-8 data, whether because it's a different encoding or due to corrupted/dirty records. You are likely hitting an error in your processing code after the CSV step, which you haven't shown us. That error is probably due to your (possibly implicitly) attempting to decode some data as UTF-8. You should fix this by (a) making sure you are decoding with the right encoding (e.g. UTF-8 vs. Latin-1 vs. Windows-1252, etc.) and (b) using try: ... except: ... to catch and properly deal with malformed records in your processing code. The solutions you have tried so far attempt to move the unicode decoding to before the csv processing, but the csv module requires raw bytes, not decoded characters, so don't help. You should continue to explicitly pass raw bytes to the csv module (using open("myfile.csv", "rb")), then immediately and explicitly attempt to decode the (raw, undecoded byte) strings in the data list, prior to applying your business logic to the data list. You will likely encounter these same errors you've been seeing here in this explicit decoding step, so you should expect and deal with them here. I'm posting based on details from below, but as a top level response because I'm addressing your general question. I don't think Python 3 will help you; the restriction on the CSV module still applies there. Further, one reason you're not seeing the error disappear when you mangle the data with io.open(..., errors="replace", ...) is that you're not actually attempting to process the mangled data; you're mangling the data, then throwing away the result, then reopening the raw file with the csv module: import io # you're not saving the file object returned by io.open io.open(csv, encoding="ascii", errors="replace") # you're now creating a new file object with codecs.open(...) data = [row for row in csv.reader(codecs.open (filename, 'U', encoding="ascii"))] Since you didn't actually use the "sanitized" data, you're of course still seeing the error. Now, I happen to think that raw destruction of accented/non-ASCII characters is a terrible approach; I wouldn't use io.open(..., errors="replace",...) unless I absolutely had to. However, that's still not the solution; it's just a debugging of the buggy solution u/willm gave you. Here's the problem: The csv module wants raw byte string inputs, not Unicode strings. Using io.open(..., encoding="...", ...) gives you a file reader that returns Unicode strings, which is precisely what you don't want. Actually, this is also a problem with codecs.open(..., encoding=...); that lets you open a file and automatically decode the inputs to Unicode strings using the encoding you specify. Again, since csv needs raw bytes strings, this is not what you want. Incidentally, csv shouldn't want raw byte strings; it must operate on the contents of the files, which must be decoded in some fashion for this to work (for instance, what if the comma character were not encoded using the same byte value in some obscure text encoding; then the csv module wouldn't be able to see the delimiters!). This is a limitation of the module, and is why there is a warning in the docs. So your codecs.open(...) approach was a good idea, but csv isn't ready for it yet. So, back to why you have a problem in the first place. Let's start with your initial attempt: data = [row for row in csv.reader(open (os.path.expanduser("~/Downloads/" + filename),'U'))] And break down the list comprehension: data = [] file_path = os.path.expanduser("~/Downloads/" + filename) file_object = open(file_path, "U") reader = csv.reader(file_object) for row in reader: data.append(row) Ok, much clearer. One thing to note is that's it's conceivable that you'll one day have a unicode filename, which could cause trouble on my line 2 when you construct the file path. Since you're processing some but not all rows of the file, though, that's not what's happening here. Next, you're opening the file in text mode using the "U" flag to open(). Don't do this; open the file as binary, and let the csv module figure out the line endings: ... file_object = open(file_path, "rb") ... What will happen now is that csv will deal with the raw bytes of your file, and return lists filled with raw byte strings, not friendly Unicode strings of characters. Now, this is still the same as what "U" was doing, just a bit more explicit. i.e. Your initial program behaved exactly the same way, returning byte not Unicode strings. What does this mean? My guess is that the Unicode error was not thrown on the data = ... line, but later on in your program, as you attempt to utilize the returned byte strings. Can you show us more of your processing of the data object? My suspicion is that somewhere you're doing something equivalent to unicode("some data field with broken utf-8 \xe8", "utf8"), and it's throwing a decode error when it hits invalid UTF-8 data. If this is the case, you need to guard against malformed UTF-8 data by catching exceptions, and figuring out how to process damaged data records. Alternatively, your file may not be encoded in UTF-8! One hint that this may be the case is that the offending byte, 0xe8, is 'a' circumflex ("â") when encoded in ISO-8859-1 ("Latin-1"). In this case, you'd switch to unicode("some data field with latin-1 \xe8", "latin1"), and everything would be fine. Note that if your file is Latin-1 encoded, there will be no decode errors; every byte is valid Latin-1, so mangled data will just show inappropriate characters, but Python will not be able to auto-detect that it's mangled and throw an exception. This may seem convenient, but it's a bad thing, overall – you'd usually rather have an exception tell you the data's bad, rather than use bad data silently. Here are a few tests to run that get rid of the CSV processing complexity: # silent if file is good UTF-8, exception if it's not UTF-8 or is broken with open("myfile.csv", "rb") as f: unicode(f.read(), "utf-8") # prints whole file, assuming your OS can handle UTF-8 data printing # (you're good on Mac/Linux). Look through the data manually; accented # characters should be correct. If instead you see multiple junk chars # in place of an accented char, your file is probably not Latin-1 with open("myfile.csv", "rb") as f: print unicode(f.read(), "latin-1") Finally, note that the codecs.open(...) solution you tried moved the Unicode decode issue, because it attempted to convert the presumably broken UTF-8 data up front, before you did your CSV processing, whereas your initial program attempted to decode the data on a per-record basis, after the CSV processing. More on reddit.com