How to schedule a script to run every day for Python - Stack Overflow
Schedule Python script
Schedule Python script
Is there a way to schedule my python script to run everyday?
Videos
Sorry about the newbie post, but I have a python script that I was to run daily and it's not very feasible to turn my computer on every time I need to run it. I know about cron but it seems to require the computer to be turned on. I also tried cloud services such as aws and gcloud, but they are beyond my expertise and I tried for like several days to no avail. If anyone could lend a helping hand that would be great!
Open a text editor, such as Notepad.
Type the following code:
Save the file as "DailyScript.py."
In the Windows Task Scheduler, create a new task.
In the "General" tab, enter the name of the task and select how often you want it to run.
In the "Triggers" tab, select "New."
In the "Begin the task" drop-down, select "On a schedule."
In the "Settings" section, select "Daily" and enter the time you want the task to run.
In the "Actions" tab, select "New."
In the "Action" drop-down, select "Start a program."
In the "Program/script" field, enter the full path to the Python interpreter, such as "C:\Python27\python.exe."
In the "Add arguments (optional)" field, enter the full path to the "DailyScript.py" file.
Click "OK" to save the task.
pip install schedule
import schedule
call_me=lambda:None
schedule.every().day.do(call_me)
Cloud functions aren't really suited to batch jobs that may be longer running than 10 minutes. I'd suggest running your job using a Compute Engine VM and scheduling it with a combination of Cloud functions / Cloud scheduler.
Here's a rough outline:
- Set up a containerized Compute Engine VM.
- Create a Cloud function to start the VM on a pub-sub trigger.
import googleapiclient.discovery
def start_job(event, context):
"""Triggered from a message on a Cloud Pub/Sub topic.
Args:
event (dict): Event payload.
context (google.cloud.functions.Context): Metadata for the event.
"""
compute = googleapiclient.discovery.build('compute', 'v1')
compute.instances().insert(
project='project_id',
zone='us-east1-b',
body=vm_config).execute()
- Create a Cloud Scheduler to trigger the pub-sub according to your schedule.
This lets you avoid the cost of an always-on VM. See this blog post for more detail.
Could this work?
import schedule
import time
def run_daily():
do something
do something else
schedule.every().day.at("08:20:30").do(run_daily) # HH MM SS
while True:
schedule.run_pending()
time.sleep(1)
You can use Google Cloud Scheduler which is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations.
The first 3 jobs per month are free. Next are $0.10 per job/month.
Yes, you will need App Engine and the Cron service to schedule those scripts. You have some more or less straight-forward options to run those scripts on GCP:
- Deploy each one as different functions, and do something as mentioned in here: Basically deploy a simple function in App Engine that sends an HTTP request to your(s) function(s).
- Deploy each one as different handlers in App Engine, and schedule each calls.
- Deploy each one as different services and then schedule the calls, bearing in mind that, if you want to schedule a call to a different service, you must specify to which one, in your
cron.yamlfile, using thetargetkeyword and the name of the service.
Hey all!
Might be that this is a noob question, but I only now come into contact with Google Cloud platform and it's not my comfort zone, I'm more just the data guy. However, getting this setup is absolutely essential. I need to find a way to get my maintenance scripts running in fixed intervals 24/7 without anyone manually running them.
I am not that unfamiliar with Unix-based operating system, so I got a virtual machine on Cloud Compute with Debian, thinking it should be as easy as doing the scheduling in Python and letting the script run forever. But it seems that after some time it just stops. I really don't need anything too fancy, just to have the regular execution of the script.
Any advice is appreciated! Thanks!
I spent quite a bit of time also looking to launch a simple Python program at 01:00. For some reason, I couldn't get cron to launch it and APScheduler seemed rather complex for something that should be simple. Schedule (https://pypi.python.org/pypi/schedule) seemed about right.
You will have to install their Python library:
pip install schedule
This is modified from their sample program:
import schedule
import time
def job(t):
print "I'm working...", t
return
schedule.every().day.at("01:00").do(job,'It is 01:00')
while True:
schedule.run_pending()
time.sleep(60) # wait one minute
You will need to put your own function in place of job and run it with nohup, e.g.:
nohup python2.7 MyScheduledProgram.py &
Don't forget to start it again if you reboot.
You can do that like this:
from datetime import datetime
from threading import Timer
x=datetime.today()
y=x.replace(day=x.day+1, hour=1, minute=0, second=0, microsecond=0)
delta_t=y-x
secs=delta_t.seconds+1
def hello_world():
print "hello world"
#...
t = Timer(secs, hello_world)
t.start()
This will execute a function (eg. hello_world) in the next day at 1a.m.
EDIT:
As suggested by @PaulMag, more generally, in order to detect if the day of the month must be reset due to the reaching of the end of the month, the definition of y in this context shall be the following:
y = x.replace(day=x.day, hour=1, minute=0, second=0, microsecond=0) + timedelta(days=1)
With this fix, it is also needed to add timedelta to the imports. The other code lines maintain the same. The full solution, using also the total_seconds() function, is therefore:
from datetime import datetime, timedelta
from threading import Timer
x=datetime.today()
y = x.replace(day=x.day, hour=1, minute=0, second=0, microsecond=0) + timedelta(days=1)
delta_t=y-x
secs=delta_t.total_seconds()
def hello_world():
print "hello world"
#...
t = Timer(secs, hello_world)
t.start()