Databases
StackOverflow2013
Postsdbo
POEmail parsing and processing architechture
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POEmail parsing and processing architechture
    text
    copied!<p>ok im doing a heavy process on processing each email. lets say im making an AI for a system at he will auto-reply the email that he receive, but im still dont know where to start.</p> <p>heres what im thinking of</p> <p><strong>architecture 1</strong> <img src="https://i.stack.imgur.com/HgTen.png" alt="enter image description here"></p> <p>problems :</p> <ol> <li><p>lets say we have 1000 emails / sec how does a mail server, exim or sendmail, davecot etc exactly work?</p></li> <li><p>can the parseandsavetomysql.py process 1000 emails in a sec though piping? how does that work too? btw currently its working fine, but i need to know about this.</p></li> <li><p>is my logic correct about a worker? or a queuing system? i have tried to see resque and friends but i still just dont get it how can we lock a session lets say in this this problem "hey im processing this file dont work on email1.rawemail work on other" how can we do that the correct or simpler way?</p></li> </ol> <p><strong>architecture 2</strong></p> <p><img src="https://i.stack.imgur.com/2nrbt.png" alt="enter image description here"></p> <p>problems?</p> <ol> <li>as written</li> <li>how can a pop/stmp server receive 1000 emails/sec?</li> <li>we can get email via imap and pop? becouse we are just processing is pop3 the right way to chose on performance? <em>there is a imap_open on php that im currently using</em></li> </ol> <p>addon</p> <ol> <li>is there a good link or blog post that solve the same problem as me?</li> <li>please give me links of projects,app or 3rd parties that solve my problem?</li> <li>if there is anything in mind, please do write them down.</li> </ol> <p>thanks for helping out, Adam Ramadhan</p> <p>edited <strong>my current architecture</strong></p> <p><img src="https://i.stack.imgur.com/202RG.png" alt="enter image description here"></p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload