Databases
StackOverflow2013
Postsdbo
POTen thousand cron jobs or loop ten thousand times?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POTen thousand cron jobs or loop ten thousand times?
    primarykey
    data
    text
    <p>I have a web app that sends customized emails to my users throughout the day, at times they've selected. So at any given time of the day, I might need to do nothing, or send up to ten thousand emails. </p> <p>Previously I solved this with a single cron job, firing every minute. It would call a script that would find the next thousand emails that needed to be sent. The script would mark them all as "in progress" and then loop over them, trying to send them. The loop is heavy - calling a remote API to send the email, updating the record to "sent", logging the action. The problem I've found is that once in a while, the loop would break, fail, the server would blink, and hundreds of emails would get stuck marked as "in progress".</p> <p>I've rewritten the loop now to get a single email that needs to be sent, mark it as "in progress", send it, mark it as "sent", log it, and then get the next one. I run it for a minute, as many loops as it wants. But this is really cpu intensive, and seems very inefficient, with a lot of db calls. </p> <p>I'm thinking now that each daily email should have it's own cron job. Then I end up with ten thousand or more jobs calling a specific url like <a href="http://mydomain.com/email/123/send" rel="nofollow">http://mydomain.com/email/123/send</a>. My concern here is that I will need to update the cron jobs for things like timezones and user preference updates ("send me my email at 6PM instead of 10AM"). So I'm programmatically re-writing my crontab, and possibly often. </p> <p>How would you solve this problem? As an aside, do you know of any web services designed to solve this problem?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCorey
    1. USCorey
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POTen thousand cron jobs or loop ten thousand times?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POTen thousand cron jobs or loop ten thousand times?
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload