There are some oddities with the code above, notably that "w" option will overwrite the csv file. from my test this would actually append to an already existing file.

import csv
with open(r'loop.csv','a') as f1: # need "a" and not w to append to a file, if not will overwrite
    writer=csv.writer(f1, delimiter='\t',lineterminator='\n',)
    # two options here, either:
    for i in range(0,10):
       row = [i]
       writer.writerow(row)
    #OR
    writer.writerows([i for i in range(10)]) #note that range(0,10) and range(10) are the same thing
Answer from Flying Turtle on Stack Overflow
🌐
Esri Community
community.esri.com › t5 › python-questions › add-rows-to-csv-in-python-loop › td-p › 488625
Add rows to csv in python loop? - Esri Community
December 11, 2021 - You can use something like DictWriter to create a csv file. However, I usually do something a bit more basic, usually using tab delimited files: # before your loop, open a new file fw = open("outputfile.txt", "w") # can be .csv for item in some_loop: # do things ...
Discussions

Appending output of a for loop for Python to a csv file - Stack Overflow
I have a folder with .txt files in it. My code will find the line count and character count in each of these files and save the output for each file in a single csv file in a different directory. T... More on stackoverflow.com
🌐 stackoverflow.com
May 24, 2017
python - Append data to CSV using a nested loop - Stack Overflow
I am trying to append data from the list json_responsecontaining Twitter data to a CSV file using the function append_to_csv. I understand the structure of the json_response. It contains data on us... More on stackoverflow.com
🌐 stackoverflow.com
python 3.x - append entries in csv in a for loop by using dataframe.to_csv - Stack Overflow
I am using the below code in a for loop, the problem is whenever a new loop starts, a new csv is formed that means deleting the previous entries. I want that the entries get appended. I can do it u... More on stackoverflow.com
🌐 stackoverflow.com
March 11, 2018
python 3.x - Appending data to Pandas DataFrame with for loop - Stack Overflow
I have a list of four URL's that I'm attempting to loop over and read into Pandas as a DataFrame. The problem is that it will only read in one of the .csv files. The following is my code: #Import More on stackoverflow.com
🌐 stackoverflow.com
November 9, 2018
🌐
YouTube
youtube.com › codetube
python append to csv in loop - YouTube
Download this code from https://codegive.com Title: Python Tutorial - Appending to CSV in a LoopIntroduction:In this tutorial, we will explore how to append ...
Published   December 13, 2023
Views   14
🌐
GeeksforGeeks
geeksforgeeks.org › python › how-to-append-a-new-row-to-an-existing-csv-file
How to append a new row to an existing csv file? - GeeksforGeeks
July 23, 2025 - Explanation: to_csv() appends the row to event.csv using mode='a' to avoid overwriting, index=False to skip the index column and header=False to prevent repeating column headers. DictWriter from Python’s built-in csv module writes rows using ...
🌐
Stack Overflow
stackoverflow.com › questions › 39905678 › appending-output-of-a-for-loop-for-python-to-a-csv-file
Appending output of a for loop for Python to a csv file - Stack Overflow
May 24, 2017 - import glob import os import csv os.chdir('c:/Users/dasa17/Desktop/sample/Upload') for file in glob.glob("*.txt"): chars = lines = 0 with open(file,'r')as f: for line in f: lines+=1 chars += len(line) a=file b=lines c=chars print(a,b,c) d=open('c:/Users/dasa17/Desktop/sample/Output/LineCount.cs‌​v', 'w') writer = csv.writer(d,lineterminator='\n') for a in os.listdir('c:/Users/dasa17/Desktop/sample/Upload'): writer.writerow((a,b,c)) d.close()
Top answer
1 of 2
1

There are two each_dict objects in json_response. They have 5 and 13 tweets, respectively (each_dict['data']). In addition, there are 5 and 13 elements in each_dict['includes']['users'], respectively.

You got 194 elements because in the first iteration of for each_dict in json_response: you save data 5x5=25 times (loop 2 is executed 5 times for every tweet in loop 1). While in the second iteration you save data 13x13=169 times (loop 2 is executed 13 times for every tweet in loop 1).

You should append data to your csv outside loop 2. That is,

for each_dict in json_response:

    for tweet in each_dict['data']:
        # ...
        
        for dic in each_dict['includes']['users']:
            # ...
        
        res = [author_id, created_at, tweet_id, text, bio, image_url]
        csvWriter.writerow(res)

In addition, I recommend using a pandas dataframe to store the info you need and save to csv. It makes the code more readable and you do not have to worry about opening a buffer. See my recommendation below, including renaming:

import pandas as pd

df = pd.DataFrame()

for each_dict in json_response:
    
    for tweet in each_dict['data']:
        row = {}
        row["author_id"] = tweet['author_id']
        row["created_at"] = dateutil.parser.parse(tweet['created_at'])
        row["tweet_id"] = tweet['id']
        row["text"] = tweet['text']
        
        for user in each_dict['includes']['users']:
            if user["id"] == row["author_id"]:
                row["bio"] = user['description']#.encode('utf-16','surrogatepass').decode('utf-16') # uncomment this if you get UnicodeError
        
        for media in each_dict['includes']['media']:
            row['image_url'] = media.get('url', ' ')

        df = df.append(row, ignore_index=True)  
        # Note, since the dataframe is initially empty with no columns, appending a dictionary (i.e, row) will automatically generate the header based on the dictionary's keys.  

df.to_csv('path/to/file.csv')

Output

               tweet_id            author_id                created_at   ...
0   1375057688355336195           2877379617  2021-03-25T12:11:14.000Z   ...
1   1374085719472361474  1265018154444562440  2021-03-22T19:48:59.000Z   ...
...
17  1360693490880032770            926909484  2021-02-13T20:53:03.000Z   ...
2 of 2
1

Looks like the else branch of if 'description' in dic: is never executed. If your code is indented correctly, then also the csvWriter.writerow part is never executed because of this.

That yields that no contents are written to your file.


A comment on code style:

  • use with open(file) as file_variable: instead of manually using open and close. That can save you some trouble, e.g. the trouble you would get when the else branch would indeed be executed and the file would be closed multiple times :)
Find elsewhere
🌐
Reddit
reddit.com › r/learnpython › python, how to repeatedly append to csv file created by pandas for storing weather data
r/learnpython on Reddit: Python, how to repeatedly append to CSV file created by pandas for storing weather data
March 28, 2017 -

Hello. This is a follow up question regarding the project I originally made involving this post: https://www.reddit.com/r/learnpython/comments/61hb2k/looking_to_scrape_historical_weather_data_new_to/

My current issue right now is that I can't store more than a couple of years data in a single list in Python, because it runs out of memory. When I try to append continuously onto one dataframe (instead of a list), I receive this warning and my final CSV file is completely blank:

'<' not supported between instances of 'str' and 'int', sort order is undefined for incomparable objects, result = result.union(other)

Another idea I had was to create one CSV file at the beginning through pandas, and then append that CSV file as my code keeps looping, but I'm unsure of how to do that. Any suggestions would be greatly appreciated!

This is the link to my code: https://pastebin.com/A31JNWp9

🌐
freeCodeCamp
forum.freecodecamp.org › python
Edit/Append data to csv file - Python - The freeCodeCamp Forum
May 11, 2021 - hello all, I am having this problem that I am not sure how to solve. is this a good place to post it? the csv file as is: col1,col2,col3,col4,col5 value1,value2,value3,value4,value5 ,,value6,value7,value8,value9 ,,value10,value11,value12,value13` the expected csv file after the function runs: Note: the function will be inside a loop, so the parameters will be passed to it each time the loop gets executed, therefore param 1 and 2 are written in the file as the loop executes for the first...
🌐
Adam the Automator
adamtheautomator.com › read-csv-in-python
How to Read CSV in Python, Write and Append Too
August 18, 2022 - If you need to append row(s) to a CSV file, replace the write mode (w) with append mode (a) and skip writing the column names as a row (writer.writerow(column_name)). You’ll see below that Python creates a new CSV file (demo_csv1.csv), with ...
🌐
Librarycarpentry
librarycarpentry.github.io › lc-python-intro › looping-data-sets.html
Python Intro for Libraries: Looping Over Data Sets
December 18, 2025 - Once we’ve saved the year variable from each file name, we can assign it to every row in a column for each CSV by assigning data['year'] = year inside of the loop. To collect the data from each CSV we’ll use a list “accumulator” (as we covered in the last episode) and append each DataFrame to an empty list.
🌐
Stack Overflow
stackoverflow.com › questions › 51147457 › python-csv-writerow-append-with-a-for-loop
Python CSV Writerow append with a for loop? - Stack Overflow
Normally, one could just .writerow['Teams', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'R', 'H', 'E'] to make that row, however sometimes the games go to extra innings, so that div class with the inning numbers/RHE changes dynamically, so I want the scraper to recognize that and adjust the row accordingly. ... from bs4 import BeautifulSoup import requests import csv with open('BoxScoreURLS.csv', newline='') as f_urls, open('IndividualBoxScoresOutput.csv', 'w', newline='') as f_output: csv_urls = csv.reader(f_urls) csv_output = csv.writer(f_output) #csv_output.writerow(['Teams', 'Box Scores']) for line in csv_urls: page = requests.get(line[0]).text soup = BeautifulSoup(page, 'html.parser') topline = soup.findAll('div', {'class' :'LineScoreCard__lineScoreColumnElement--1byQk LineScoreCard__header--3ZO_N'}) for t in range(len(topline)): csv_output.writerow(['Teams', topline[t].text])
🌐
DevQA
devqa.io › python-read-write-csv-file
Python Read Write CSV File
September 2, 2020 - from pandas import DataFrame import ... to csv file is 'w'. If we want to append data to an existing CSV file we have to use the append mode, e.g....
🌐
Stack Overflow
stackoverflow.com › questions › 61656190 › how-to-create-a-loop-that-appends-new-rows-to-csv-each-time-it-loops
python - How to create a loop that appends new rows to CSV each time it loops? - Stack Overflow
I've so far created a script that successfully loops through the list of API tokens but just overwrites the CSV file each time. Here is my script currently: CopyapiToken = ["n0000000000001", "N0000000002"] for x in apiToken: baseUrl = "https://group.qualtrics.com/API/v3/surveys" headers = { "x-api-token": x, } response = requests.get(baseUrl, headers=headers) surveys = response.text surveys2 = json.loads(response.text) surveys3 = surveys2["result"]["elements"] df = pd.DataFrame(surveys3) df.to_csv('survey_list.csv', index=False)
🌐
YouTube
youtube.com › watch
Append to CSV File (3/3) - Python for Beginners - YouTube
In this video we will cover how to add new data to a CSV.Basic Steps:Open the CSV FileAppend to the new file.Use proper csv format.Create random list data.En...
Published   October 2, 2023
🌐
Delft Stack
delftstack.com › home › howto › python pandas › pandas to csv append
How to Append Data to CSV using Pandas | Delft Stack
March 11, 2025 - In this example, we define a list of dictionaries called data_to_append, where each dictionary represents a new row of data. We then loop through each item in the list, converting it to a DataFrame and appending it to the existing CSV file.
🌐
Northwestern
hades.mech.northwestern.edu › index.php › Writing_a_CSV_File
Writing a CSV File - Northwestern Mechatronics Wiki
import numpy as np # Generate random 3x4 matrix of floats y, 3x1 vector of ints d y = np.random.rand(3, 4) d = np.random.randint(-100, 100, 3) # Open a file for output # Overwrite f = open("output.csv", "w") # Append #f = open("output.csv", "a") # For loop running 3 times to print each csv row for i in range(len(d)): output = " .6f, .6f, .6f, .6f, %d\n" % (y[i,0], y[i,1], y[i,2], y[i,3], d[i]) f.write(output) # close file f.close()