You don't need to use urllib.URLopener(), just use urllib.urlretrieve() instead as the following:
import urllib
urllib.urlretrieve("https://www.nseindia.com/products/content/historical/EQUITIES/2017/JUN?cm23JUN2017bhav.csv.zip8", "file.zip8")
You can also use wget:
import wget
wget.download("https://www.nseindia.com/products/content/historical/EQUITIES/2017/JUN?cm23JUN2017bhav.csv.zip8", "file.zip8")
And about automating it to download everyday, you can either use windows scheduler, or use a loop as the following:
import urllib
import time
while True:
urllib.urlretrieve('file', 'file')
time.sleep(86400) # 86400 seconds = 24 hours.
Medium
momohemmanuel370.medium.com › automating-file-downloads-with-python-7640fd7080ce
Automating File Downloads with Python | by Emmanuel Momoh | Medium
October 9, 2023 - You can install the required library using the following command: ... Import Playwright and Define the Function: Begin by importing the necessary modules and defining the download_pdfs function.
How can I use python to pull a file off of a website? [need help]
Take a look at requests and BeautifulSoup. You can iterate through your sequence of URLs and use requests.get(url) on each in turn. If the request works, you can use BeautifulSoup to find an anchor tag with the "download" text (or some other unique characteristic). You can use requests.get(download_link, stream=True) to start the download of each file in chunks (using file_response.iter_content(chunk_size=8192)) where file_response is what you assigned the result of the requests to. More on reddit.com
13
0
March 14, 2025How to Automate the File Download from This Site
Oh there's a captcha that appears on first load ... I don't think this will be easy to fix. First idea: Selenium, screen shot the captcha, feed it to OpenAI and get the result, and type it in. Super fickle solution, because you'll have to keep checking if there are followups, and they may migrate to something more complicated like rotating images. After that, Selenium all the way Why would a government agency put a captcha on this? It's public data and who cares if it's being scraped .. that's actually a good thing b/c people are using your data ... Can you look through their website and see if there's an API you can use instead? More on reddit.com
6
2
November 12, 2023Automate download in web using python - Stack Overflow
I have a script that will login to a site, and then it will click a link and then it needs to export the data to excel file. The script already have a login to the site It is lacking on the downl... More on stackoverflow.com
web scraping - How can I automate downloading files from a website using different inputs using Python? - Stack Overflow
I need to download a number of data from the website https://www.renewables.ninja/ and I want to automate the process using Python if possible. I want to select cities (say Berlin, New York, Seoul)... More on stackoverflow.com
Medium
medium.com › @armaansinghbhau8 › automate-file-downloads-from-urls-with-python-a-simple-guide-9a98cde10095
Automate File Downloads from URLs with Python: A Simple Guide | by Armaansinghbhau | Medium
December 4, 2024 - Downloading files from the internet is a common task, whether it’s for backing up data, retrieving reports, or pulling in datasets for analysis. While manually downloading files one by one can be time-consuming, you can streamline the process by automating it with Python. In this article, we’ll show you how to use Python to automatically download files from URLs.
Real Python
realpython.com › python-download-file-from-url
How to Download Files From URLs With Python – Real Python
January 25, 2025 - Because your URL is quite long, you rely on Python’s implicit concatenation by splitting the string literal over multiple lines inside parentheses. The Python interpreter will automatically join the separate strings on different lines into a single string. You also define the location where you wish to save the file.
Reddit
reddit.com › r/learnpython › how can i use python to pull a file off of a website? [need help]
r/learnpython on Reddit: How can I use python to pull a file off of a website? [need help]
March 14, 2025 -
I have a spreadsheet of direct links to a website that I want to download files from. Each link points to a separate page on the website with the download button to the file. How could I use python to automate this scraping process? Any help is appreciated.
Top answer 1 of 5
5
Take a look at requests and BeautifulSoup. You can iterate through your sequence of URLs and use requests.get(url) on each in turn. If the request works, you can use BeautifulSoup to find an anchor tag with the "download" text (or some other unique characteristic). You can use requests.get(download_link, stream=True) to start the download of each file in chunks (using file_response.iter_content(chunk_size=8192)) where file_response is what you assigned the result of the requests to.
2 of 5
2
Read the website source code or use a webbrowser inspect tool to find out what clicking the button does. Then do that with python.
Medium
aengel.medium.com › downloading-files-from-a-website-using-python-fdc29c520deb
Downloading Files from a Website using Python | by Auryn Engel | Medium
May 8, 2020 - I just wanted to practice a little bit with Python. So don’t be stupid and download too much I mean in the end you have to read the books too 😀 · And yes, I intentionally used images for the code, so you cannot simply copy and paste the code to crawl the site. You should learn to do it and here you can read the code. So I hope next time, you will automate your task to save you time 😀
Medium
medium.com › @moungpeter › how-to-automate-downloading-files-using-python-selenium-and-headless-chrome-9014f0cdd196
How to automate downloading files using Python, Selenium, and Headless Chrome
December 10, 2021 - Verify your file been downloaded in the path you set it to and congratulations you’ve automated file downloading!
Quora
quora.com › How-do-I-automate-the-process-of-downloading-newly-added-files-everyday-from-a-college-website-using-python
How to automate the process of downloading newly added files everyday from a college website using python - Quora
Answer: Create a web scraper and run it. Use selenium webdriver. Just go through their documention and you'll get the idea. However there are other tools like http://scrapy.org You can also visit the website: AUTOMATE THE BORING STUFF WITH PYTHON * I use Selenium on day to day basis like logging...
Medium
medium.com › @jakhalsudhir › how-to-automate-file-download-with-python-a-step-by-step-guide-bb9664a560f0
How to Automate File Download with Python: A Step-by-Step Guide | by Sudhir Kumar | Medium
May 7, 2023 - By using Python’s requests library, web scrapers can download files from a website and extract the relevant information. Machine learning: The code could be used to download training data for machine learning models. By automating the download process, machine learning engineers can ensure that the data is downloaded correctly and that the training process is not interrupted.
Reddit
reddit.com › r/webscraping › how to automate the file download from this site
r/webscraping on Reddit: How to Automate the File Download from This Site
November 12, 2023 -
This is the official page to view/download data for National Air Quality Index - India.
Here, we choose the date and hour, and a clickable link directly downloads a spreadsheet with data for that time-stamp.
Need ideas on how to automate that file download process in Python, so that we can download daywise and hourly data by specifying the date/hour range in the code.
Thanks in advance!
Top answer 1 of 3
3
Oh there's a captcha that appears on first load ... I don't think this will be easy to fix. First idea: Selenium, screen shot the captcha, feed it to OpenAI and get the result, and type it in. Super fickle solution, because you'll have to keep checking if there are followups, and they may migrate to something more complicated like rotating images. After that, Selenium all the way Why would a government agency put a captcha on this? It's public data and who cares if it's being scraped .. that's actually a good thing b/c people are using your data ... Can you look through their website and see if there's an API you can use instead?
2 of 3
3
Get json data Intercept the response of API, get the json data by decrypting. Refer to "Download the excel file. url in response_cache: part of sample json data {"title": "Diwator Nagar, Koppal - KSPCB", "nOfCom": 100, "down": "false", "downmessage": "", "date": "Monday, 13 Nov 2023 08:00 AM", "temp": "", "aqi": {"param": "OZONE", "value": 80, "remark": "Satisfactory", "color": "#009933"}, "metrics": [{"name": "PM2.5", "avg": 53, "avgDesc": "Over the last 24 hours", "min": 39, "max": 70, "pollutantName": "PM2.5"}, {"name": "PM10", "avg": 77, "avgDesc": "Over the last 24 hours", "min": 55, "max": 99, "pollutantName": "PM10"}, {"name": "NO2", "avg": 5, "avgDesc": "Over the last 24 hours", "min": 1, "max": 9, "pollutantName": "NO2"}, {"name": "NH3", "avg": 5, "avgDesc": "Over the last 24 hours", "min": 5, "max": 6, "pollutantName": "NH3"}, {"name": "SO2", "avg": 22, "avgDesc": "Over the last 24 hours", "min": 21, "max": 24, "pollutantName": "SO2"}, {"name": "CO", "avg": 52, "avgDesc": "Over the last 8 hours", "min": 36, "max": 64, "pollutantName": "CO"}, {"name": "OZONE", "avg": 80, "avgDesc": "Over the last 8 hours", "min": 73, "max": 131, "pollutantName": "OZONE"}] ...
BrowserStack
browserstack.com › home › guide › how to download a file using selenium and python
How to download file using Selenium & Python | BrowserStack
May 30, 2025 - Step-by-step tutorial on how to download a file from a website using Selenium and Python. Also, learn how to download files to a specific folder using Selenium.
Medium
shashivarma.medium.com › automating-data-downloads-and-conversions-with-python-a-step-by-step-guide-e92122f0bee7
Automating Data Downloads and Conversions with Python: A Step-by-Step Guide | by Shashi Varma | Medium
January 10, 2025 - Find the right files: Automatically scrape the website for files matching a specific year and type. Download the files: Save them into organized directories for future reference. Convert to CSV: Transform .xlsx files into .csv format for seamless data analysis. This project was not only an opportunity to solve my problem but also a chance to practice my skills in web scraping, file management, and automation. To tackle the problem, I decided to create a Python script that automates the entire workflow.
ScrapingBee
scrapingbee.com › blog › python-wget
Python wget: Automate file downloads with 3 simple commands | ScrapingBee
January 19, 2026 - In practice, most automation scripts only need a handful of patterns, and you can cover 90% of real-world cases with three simple commands: Download a file using subprocess.run(["wget", url]) Save with a custom filename using the -O flag or ...
Stack Overflow
stackoverflow.com › questions › 65127129 › how-can-i-automate-downloading-files-from-a-website-using-different-inputs-using
web scraping - How can I automate downloading files from a website using different inputs using Python? - Stack Overflow
I need to download a number of data from the website https://www.renewables.ninja/ and I want to automate the process using Python if possible. I want to select cities (say Berlin, New York, Seoul) as well as parameters for solar PV and wind based on the inputs from a Python file, and run it (which takes approximately 5 seconds in the website) and download the csv files.enter image description here
Quora
quora.com › How-can-I-auto-download-files-every-5-minutes-from-the-web-via-Python
How to auto download files every 5 minutes from the web via Python - Quora
Answer: First thing you need to do is figure out how to download a file. Here's a sample. [code]>>> import requests >>> >>> url = "http://download.thinkbroadband.com/10MB.zip" >>> r = requests.get(url) >>> print len(r.content) 10485760 [/code]Next, you need to figure out how to put that in loop...
SQLPad
sqlpad.io › tutorial › python-download-file-from-url
Python download file from url | SQLPad
April 29, 2024 - Here, we're simply specifying the URL of the file we wish to download, sending a GET request to that URL, and then writing the content of the response to a file on our local filesystem.