Do you want to do something like this?
import csv
with open('large.csv','w') as f1:
writer=csv.writer(f1, delimiter='\t',lineterminator='\n',)
for i in range(1000000):
row = [i + j*0.2 for j in range(i+1)]
writer.writerow(row)
or also with the row/column headers:
import csv
with open('large.csv','w') as f1:
writer=csv.writer(f1, delimiter='\t',lineterminator='\n',)
writer.writerow([''] + range(1000000))
for i in range(1000000):
row = [i] + [i + j*0.2 for j in range(i+1)]
writer.writerow(row)
The latter returns the following file (I have replaced 1000000 with 10):
0 1 2 3 4 5 6 7 8 9
0 0.0
1 1.0 1.2
2 2.0 2.2 2.4
3 3.0 3.2 3.4 3.6
4 4.0 4.2 4.4 4.6 4.8
5 5.0 5.2 5.4 5.6 5.8 6.0
6 6.0 6.2 6.4 6.6 6.8 7.0 7.2
7 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4
8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6
9 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8
Answer from eumiro on Stack Overflowimport csv
with open("sample.csv", "w") as csvFile:
fieldnames = ['Item Name']
writer = csv.DictWriter(csvFile, fieldnames=fieldnames)
writer.writeheader()
for item in list:
info = inspect.getmembers(item)
name = info[0]
writer.writerow({'Item Name': name })
Put the with before your loop so that you can write the values to the file from within your for loop:
import csv
with open('sample.csv', 'w') as csvFile:
fieldnames = ['Item Name']
writer = csv.DictWriter(csvFile, fieldnames=fieldnames)
writer.writeheader()
for item in list:
# get the data you want from item
info = inspect.getmembers(item)
name = info[0]
# write the data to the file
writer.writerow({'Item Name': name })
NOTE (since I don't have sufficient reputation to comment yet): csvFile.close() found in Rohan's answer is not necessary since that is handled by the with statement. Also, per PEP8, he should use 2 spaces rather than 4 for indentations.
string - Write output from for loop to a csv in python - Stack Overflow
python - Writing to csv with for loops - Stack Overflow
Writing lines to a csv file in a for loop in python - Stack Overflow
csv writer in a loop - Python - Stack Overflow
I have been able to get up to this point of reading my csv and printing out the strings I want. I can't figure out how to get the 'DictWriter' section of my script to write to a new column called "description"
My goal is to loop through each column/row and write a description.
I can get print() to work but nothing I have tried will get the writer to work. I am at a total loss
**Note**
The print() functions shown in the code below are written for readability of this post
import csv
import pandas as pd
f_name = '/Users/ashtoncarroll/Documents/github_repositories/Ashkiebear/projects/csv_desc_builder/csv_files/files/Shopify Listing Descriptions appended.csv'
with open(f_name, 'r') as csv_file:
csv_reader = csv.DictReader(csv_file, delimiter=',')
csv_reader.__next__()
for row in csv_reader:
brand = row['brand']
part_type = row['part_type']
model = row['model']
part_number = row['part number']
quantity = row['quantity']
print(f'''
New {part_type}!
Fits {brand} Models:
{model}
{part_number}
Includes:
{quantity}x {part_type}')
'''
with open('writer_file.csv', 'w') as wf:
fieldnames = ['brand', 'part_type', 'model', 'part number', 'quantity', 'description']
csv_writer = csv.DictWriter(csv_file, filenames=fieldnames)
for row['description'] in csv_reader:
csv_writer.writerow(f'''
New {part_type}!
Fits {brand} Models:
{model}
{part_number}
Includes:
{quantity}x {part_type}')
'''import csv
import pandas as pd
with open('Remarks_Drug.csv', newline='', encoding ='utf-8') as myFile:
reader = csv.reader(myFile)
mydrug = []
for row in reader:
product = row[0].lower()
#print('K---'+ product)
filename = row[1]
product_patterns = ', '.join([i.split("+")[0].strip() for i in product.split(",")])
mydrug.append([product_patterns, filename])
# print(mydrug)
df = pd.DataFrame(mydrug, columns=['product_patterns', 'filename'])
print(df)
df.to_csv('drug_output100.csv', sep=',', index=False)
This utilizes pandas library. If you're to deal with large csv files using pandas will be handy and efficient in terms of performance and memory. This is just an alternative solution for the above.
I hope this is the right way for you, if is not, tell me and we check.
import csv
with open('Remarks_Drug.csv') as myFile:
reader = csv.reader(myFile)
products_list = list()
filenames_list = list()
for row in reader:
products_list.append(row[0].lower().split("+")[0].strip())
filenames_list.append(row[1])
for index, product in enumerate(products_list):
with open ('drug_output100.csv', 'a') as csvfile:
fieldnames = ['product_patterns', 'filename']
print(fieldnames)
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
print(writer)
writer.writerow({'product_patterns':product, 'filename':filenames_list[index]})
- Open the Remarks_Drug.csv file and create two list where store the row value elaborated as you prefer.
- Iterate on the product list and enumerate it so you have an index to use on the filename list.
- Open the output file and append to it the result.
You can also use pandas to elaborate csv files, faster and in a smart way.
Here the pandas solution:
import pandas as pd
def select_real_product(string_to_elaborate):
return string_to_elaborate.split('+')[0].strip()
df = pd.read_csv("Remarks_Drug.csv", delimiter=',', names=("product", "filename"))
df['product'] = df['product'].apply(select_real_product)
df.to_csv("drug_output100.csv", sep=',', na_rep='empty',index_label=False, index=False)
open your csv file once, and write one line per iteration, and no need for so many variables:
with open('test.csv', 'w') as csvFile:
writer=csv.writer(csvFile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL, lineterminator='\n')
for i in list_people:
info = inspect.getmembers(i)
writer.writerow(info[1:3])
or completely replace the for loop by writerows using a generator comprehension, that will be even faster:
writer.writerows(inspect.getmembers(i)[1:3] for i in list_people)
Append each pair of name and age as a list, and then use the writerows method of the csv.writer:
for i in list_people:
info = inspect.getmembers(i)
row_lines.append(info[1:3]) # slice [1:3] returns a list of items 1 and 2
with open('test.csv', 'w') as csvFile:
writer = csv.writer(csvFile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL, lineterminator='\n')
writer.writerows(row_lines)
Every time you open the file in w mode, it will overwrite everything that was there. You should open the file one time, then loop over calls to writerow like:
with open(str(out_dir)+str(nme[0])+'.csv','w') as f1:
writer=csv.writer(f1, delimiter=',')#lineterminator='\n',
for i in np.arange(0,9):
row = data[i]
writer.writerow(row)
instead of reopening the file each iteration through the for loop
Just to finish off the question above.
I solved my problem (not very elegantly) by opening/writing all the csv files I needed with the w attribute. Then used the a attribute to append each csv file within a second for loop.
Thanks for the answers
Cheers
Although for this task you don't need it, I would take advantage of standard library modules when you can, like csv. Try something like this,
import os
import csv
csvfile = open('outputFileName.csv', 'wb')
writer = csv.writer(csvfile)
for filename in os.listdir('/'): # or C:\\ if on Windows
writer.writerow([filename, len(filename)])
csvfile.close()
I'd probably change this:
for filename in os.listdir (image_path):
print filename
print len(filename)
To something like
lines = list()
for filename in os.listdir(image_path):
lines.append("%s, %d" % (filename, len(filename)))
My version creates a python list, then on each iteration of your for loop, appends an entry to it.
After you're done, you could print the lines with something like:
for line in lines:
print(line)
Alternatively, you could initially create a list of tuples in the first loop, then format the output in the second loop. This approach might look like:
lines = list()
# Populate list
for filename in os.listdir(image_path):
lines.append((filename, len(filename))
# Print list
for line in lines:
print("%s, %d" % (line[0], line[1]))
# Or more simply
for line in lines:
print("%s, %d" % line)
Lastly, you don't really need to explicitly store the filename length, you could just calculate it and display it on the fly. In fact, you don't even really need to create a list and use two loops.
Your code could be as simple as
import sys, os
image_path = "C:\\"
for filename in os.listdir(image_path):
print("%s, %d" % (filename, len(filename))
You would create the csv row as a loop (or using list comprehension) I will show the explicit loop for ease of reading and you can change it to a single list comprehension line yourself.
row = []
for n in deats:
row.append(n)
Now you have row ready to write to the .csv file using csv.Writer()
Hei, try like this:
import csv
csv_output = csv.writer(open("output.csv", "wb")) # output.csv is the output file name!
csv_output.writerow(["Col1","Col2","Col3","Col4"]) # Setting first row with all column titles
temp = []
deats = soup.find_all('p')
for n in deats:
temp.append(str(n.text))
csv_output.writerow(temp)
Leaving the with block closes with file. Therefore, the new_file function just opens and immediately closes a file.
You could do somthing like the following:
import csv
rowCounter = 0
fileCounter = 0
List_A = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_B = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_C = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
# create file handle
csvfile = open('file-' + str(fileCounter) + '.csv', 'w')
rowWriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_NONE)
for word1 in List_A:
for word2 in List_B:
for word3 in List_C:
sentence = word1 + word2 + word3
rowWriter.writerow ([sentence])
rowCounter += 1
if rowCounter == 100:
# close current filehandle
csvfile.close()
fileCounter += 1
# open new file
csvfile = open('file-' + str(fileCounter) + '.csv', 'w')
rowWriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_NONE)
rowCounter = 0
# close file
csvfile.close()
or with defining a function:
import csv
rowCounter = 0
fileCounter = 0
List_A = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_B = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_C = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
def new_writer( csvfile, counter ):
if csvfile:
csvfile.close()
# open new file
csvfile = open('file-' + str(counter) + '.csv', 'w')
rowWriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_NONE)
counter += 1
return rowWriter,csvfile,counter
rowWriter, csvFile, fileCounter = new_writer( None, fileCounter )
for word1 in List_A:
for word2 in List_B:
for word3 in List_C:
sentence = word1 + word2 + word3
rowWriter.writerow ([sentence])
rowCounter += 1
if rowCounter == 100:
# close current file and open a new one
rowWriter, csvfile, counter = new_writer( csvfile, fileCounter )
rowCounter = 0
# close file
csvFile.close()
Thanks @desiato!
I accepted your answer, but ended up using lines 23-29 of your code and ended up with this (it works great!):
import csv
rowCounter = 0
fileCounter = 0
List_A = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_B = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
List_C = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
with open('file-' + str(fileCounter) + '.csv', 'w') as csvfile:
rowWriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_NONE)
for word1 in List_A:
for word2 in List_B:
for word3 in List_C:
sentence = word1 + word2 + word3
rowWriter.writerow ([sentence])
rowCounter += 1
if rowCounter == 100:
csvfile.close()
fileCounter += 1
csvfile = open('file-' + str(fileCounter) + '.csv', 'w')
rowWriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_NONE)
rowCounter = 0
else:
continue
Put the
with open('results.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(['company', 'followers', 'result'])
outside (above) the loop, and call writer.writerow inside the loop. The loop must be inside the with block.
You are opening the file every time inside the second loop. The file always has the same name and overwrites the previous file. Move the
with open('results.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
outside the outer loop and just write the rows in the inner loop.
You are encountering the error because it is re-writing a csv for every iteration in your loop. You should move the with open() statement outside of your loop block.
Try opening the file only once and then doing the loop:
with open("output.csv",'wb') as f:
writer = csv.writer(f, dialect='excel')
for item in list_A:
writer.writerow([value1, value2, value3])