As Niklas answered, the lineterminator argument lets you choose your line endings. Rather than hard coding it to '\n', make it platform independent by using your platform's line separator: os.linesep. Also, make sure to specify newline='' for Python 3 (see this comment).

import csv
import os

with open('output.csv', 'w', newline='') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

Old solution (Python 2 only)

In Python 2, use 'wb' mode (see the docs).

import csv
import os

with open('output.csv', 'wb') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

For others who find this post, don't miss the 'wb' if you're still using Python 2. (In Python 3, this problem is handled by Python). You won't notice a problem if you're missing it on some platforms like GNU/Linux, but it is important to open the file in binary mode on platforms where that matters, like Windows. Otherwise, the csv file can end up with line endings like \r\r\n. If you use the 'wb' and os.linesep, your line endings should be correct on all platforms.

I came across this issue on windows for Python 3. I tried changing newline parameter while opening file and it worked properly with newline=''.

Add newline='' to open() method as follows:

with open('stocks2.csv','w', newline='') as f:
    f_csv = csv.DictWriter(f, headers)
    f_csv.writeheader()
    f_csv.writerows(rows)

It will work as charm.

Hope it helps.

While @john-machin gives a good answer, it's not always the best approach. For example, it doesn't work on Python 3 unless you encode all of your inputs to the CSV writer. Also, it doesn't address the issue if the script wants to use sys.stdout as the stream.

I suggest instead setting the 'lineterminator' attribute when creating the writer:

import csv
import sys

doc = csv.writer(sys.stdout, lineterminator='\n')
doc.writerow('abc')
doc.writerow(range(3))

That example will work on Python 2 and Python 3 and won't produce the unwanted newline characters. Note, however, that it may produce undesirable newlines (omitting the LF character on Unix operating systems).

In most cases, however, I believe that behavior is preferable and more natural than treating all CSV as a binary format. I provide this answer as an alternative for your consideration.

In Python 2, you could write to CSV files with the 'wb' option on the file and avoid this.

In Python 3, it's a little different - here's the documentation, take a look at the footnote.

Since you're opening the csv file as a file, you should replace line 25 with this:

with open('UB04_nudge.csv', 'w', newline='') as csvfile:

Basically, since Windows uses \r\n line endings, file() is already planning to write a newline ending out after each line. CSV does this as well - so you're getting the duplicate newlines after each row. By setting newline='', you're telling file() to not terminate new lines - which works since csv() will terminate the lines on its own.

You should open the source file with newline='' instead so that the original newline endings can be preserved for csv.reader to handle newlines properly:

datafile = open('source.csv', 'r', newline='')

Excerpt from the documentation:

If newline='' is not specified, newlines embedded inside quoted fields will not be interpreted correctly, and on platforms that use \r\n linendings on write an extra \r will be added. It should always be safe to specify newline='', since the csv module does its own (universal) newline handling.

I assume that you want to keep the newlines in the strings for some reason after you have loaded the csv files from disk. Also that this is done again in Python. My solution will require Python 3, although the principle could be applied to Python 2.

The main trick

This is to replace the \n characters before writing with a weird character that otherwise wouldn't be included, then to swap that weird character back for \n after reading the file back from disk.

For my weird character, I will use the Icelandic thorn: Þ, but you can choose anything that should otherwise not appear in your text variables. Its name, as defined in the standardised Unicode specification is: LATIN SMALL LETTER THORN. You can use it in Python 3 a couple of ways:

    weird_literal = 'þ'
    weird_name = '\N{LATIN SMALL LETTER THORN}'
    weird_char = '\xfe'  # hex representation
    weird_literal == weird_name == weird_char  # True

That \N is pretty cool (and works in python 3.6 inside formatted strings too)... it basically allows you to pass the Name of a character, as per Unicode's specification.

An alternative character that may serve as a good standard is '\u2063' (INVISIBLE SEPARATOR).

Replacing \n

Now we use this weird character to replace '\n'. Here are the two ways that pop into my mind for achieving this:

1. using a list comprehension on your list of lists: data:

    new_data = [[sample[0].replace('\n', weird_char) + weird_char, sample[1]]
                 for sample in data]
   

2. putting the data into a dataframe, and using replace on the whole text column in one go

    df1 = pd.DataFrame(data, columns=['text', 'category'])
    df1.text = df.text.str.replace('\n', weird_char)
   

The resulting dataframe looks like this, with newlines replaced:

               text              category
0         some text in one line      1   
1  text withþnew line character      0   
2    another newþline character      1   

Writing the results to disk

Now we write either of those identical dataframes to disk. I set index=False as you said you don't want row numbers to be in the CSV:

FILE = '~/path/to/test_file.csv'
df.to_csv(FILE, index=False)

What does it look like on disk?

text,category

some text in one line,1

text withþnew line character,0

another newþline character,1

Getting the original data back from disk

Read the data back from file:

new_df = pd.read_csv(FILE)

And we can replace the Þ characters back to \n:

new_df.text = new_df.text.str.replace(weird_char, '\n')

And the final DataFrame:

new_df
               text               category
0          some text in one line      1   
1  text with\nnew line character      0   
2    another new\nline character      1   

If you want things back into your list of lists, then you can do this:

original_lists = [[text, category] for index, text, category in old_df_again.itertuples()]

Which looks like this:

[['some text in one line', 1],
 ['text with\nnew line character', 0],
 ['another new\nline character', 1]]

CSV uses RFC 4180. So the comment https://stackoverflow.com/a/42666642/9179591 is incorrect. Most use cases are going that the EOL of csv is CR/LF.

In Unix(Ubuntu in my case), you can look it, when you open your file in cat -A /<someYourFilePath>