Cron

musicplace.co.nr · 03-08-2010, 06:06 PM

Hi,

I did a search and haven't been able to find an answer.

Can I set up a cron job to run every 5mins, but only between midnight and 2am every day?

I think the answer is no.

I need to run a script once a day, but it takes more than 30 seconds to run and dies before it finishes. The script fetches a number of rss feeds, does some processing on them and inserts some info into a database.

I've changed the script to process only a few feeds each time it runs, but it means I need to run the script about 23 times to process all the feeds.

I need all of the info as early in the day as possible. I could run the script/cronjob every hour, but that means that some of the feeds won't be processed until later in the day, making them useless.

If I run the cronjob every 5 mins then all the feeds will be processed nice and early in the day, but I'll be inundated with 288 confirmation emails a day, because the cronjob will keep running even though there aren't any feeds for it to process.
????: NamePros.com http://www.namepros.com/web-hosting-discussion/643803-cron.html

Any thoughts or questions?

Cheers,
Richard

tiawood · 03-09-2010, 06:03 AM

Around how many RSS feeds is it trying to process?
Also, what language is it written in?

**enlytend** · 03-09-2010, 10:19 AM

You can provide multiple values separated by commas:

This example will run every 5 minutes from midnight up to (but not including) 2 AM (and will not send you email)

0,5,10,15,20,25,30,35,40,45,50,55 0,1 * * * command > /dev/null 2>&1

musicplace.co.nr · 03-09-2010, 01:19 PM

Originally Posted by tiawood

Around how many RSS feeds is it trying to process?
Also, what language is it written in?

About 600 feeds in total (I've broken it down into 23 feed chunks). PHP.

Originally Posted by enlytend

You can provide multiple values separated by commas:

This example will run every 5 minutes from midnight up to (but not including) 2 AM (and will not send you email)

0,5,10,15,20,25,30,35,40,45,50,55 0,1 * * * command > /dev/null 2>&1

Perfect... thanks!

sdsinc · 03-09-2010, 01:27 PM

30 seconds must be the timeout for your scripts (PHP ?).
Have a look at the set_time_limit directive, if execution time can be overridden on your host.

WebIntellects - Rob · 03-10-2010, 11:58 PM

I think you need another cron to be involved - just to check if there are any new feeds. If there are - start the main script.

musicplace.co.nr · 03-11-2010, 12:20 AM

Thanks for the response, Rob, but the feeds are updated constantly so they are always new.

I used enlytend's tip to make cron run like I wanted and it works perfect. I log everthing to a file, and when the last cron runs it sends me an email just so I can check everything was ok.

03-08-2010, 06:06 PM	THREAD STARTER #1 (permalink)
musicplace.co.nr Senior Member Join Date: Oct 2007 Location: Sydney, Australia Posts: 1,022 Follow @music_whiz	Cron Hi, I did a search and haven't been able to find an answer. Can I set up a cron job to run every 5mins, but only between midnight and 2am every day? I think the answer is no. I need to run a script once a day, but it takes more than 30 seconds to run and dies before it finishes. The script fetches a number of rss feeds, does some processing on them and inserts some info into a database. I've changed the script to process only a few feeds each time it runs, but it means I need to run the script about 23 times to process all the feeds. I need all of the info as early in the day as possible. I could run the script/cronjob every hour, but that means that some of the feeds won't be processed until later in the day, making them useless. If I run the cronjob every 5 mins then all the feeds will be processed nice and early in the day, but I'll be inundated with 288 confirmation emails a day, because the cronjob will keep running even though there aren't any feeds for it to process. ????: NamePros.com http://www.namepros.com/web-hosting-discussion/643803-cron.html Any thoughts or questions? Cheers, Richard __________________ juegosjuegos Your Hotel in Austria Matratzen Music Database - iTunes Music Chart Archive and Analysis - iTunes App Chart Archive and Analysis - iTunes Book Charts

03-09-2010, 06:03 AM	#2 (permalink)
tiawood NamePros Regular Join Date: Oct 2008 Location: Florida Posts: 450 Follow @NameHoney	Around how many RSS feeds is it trying to process? Also, what language is it written in? __________________ ██ 6 Website HotSpots You May Have Missed \| Custom Web Development & Graphic Deals ██

03-09-2010, 10:19 AM	#3 (permalink)
enlytend Forum Moderator Join Date: Aug 2006 Location: USA Posts: 2,152 Follow @enlytend	You can provide multiple values separated by commas: This example will run every 5 minutes from midnight up to (but not including) 2 AM (and will not send you email) 0,5,10,15,20,25,30,35,40,45,50,55 0,1 * * * command > /dev/null 2>&1 __________________ Enlytend Solutions - Internet marketing and web development Was my advice helpful? Please consider a small donation to the National Canine Cancer Foundation - a tax deductible 501(c)(3)that directly funds cancer research Last edited by enlytend; 03-09-2010 at 10:28 AM. Reason: oops - added an extra $ sign!

03-09-2010, 01:27 PM	#5 (permalink)
sdsinc Domains my Dominion Join Date: Aug 2005 Location: Web 1.0 Posts: 9,963	30 seconds must be the timeout for your scripts (PHP ?). Have a look at the set_time_limit directive, if execution time can be overridden on your host. __________________ NameNewsletter.com - free lists of available domain names ZoneFiles.net (beta) - ccTLD and gTLD droplists

03-10-2010, 11:58 PM	#6 (permalink)
WebIntellects - Rob NamePros Regular Join Date: Nov 2009 Posts: 656	I think you need another cron to be involved - just to check if there are any new feeds. If there are - start the main script. __________________ Use coupon "Rob" and get 10% discount for any payment! WebIntellects - Fully Managed Dedicated and since 1999. WebhostGIANT - Low cost hosting from $2.79!

03-11-2010, 12:20 AM	THREAD STARTER #7 (permalink)
musicplace.co.nr Senior Member Join Date: Oct 2007 Location: Sydney, Australia Posts: 1,022 Follow @music_whiz	Thanks for the response, Rob, but the feeds are updated constantly so they are always new. I used enlytend's tip to make cron run like I wanted and it works perfect. I log everthing to a file, and when the last cron runs it sends me an email just so I can check everything was ok. __________________ juegosjuegos Your Hotel in Austria Matratzen Music Database - iTunes Music Chart Archive and Analysis - iTunes App Chart Archive and Analysis - iTunes Book Charts

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)