StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>In Erlang it is very cheap to create processes (orders of magnitudes compared to other systems).</p> <p>Therefore I recommend to create a new ProcessFileServer each time a new file to process is appearing. When it is done with just terminate the process with exit reason <code>normal</code>.</p> <p>I would suggest the following structure:</p> <pre><code> top_supervisor | +-----------------------+-------------------------+ | | directory_supervisor processing_supervisor | simple_one_for_one +----------+-----...-----+ | | | | starts children transient | | | | dir_watcher_1 dir_watcher_2 dir_watcher_n +-------------+------+---...----+ | | | proc_file_1 proc_file_2 proc_file_n </code></pre> <p>When a <code>dir_watcher</code> notices a new file appeared. It calls the <code>processing_supervisor</code>s <code>supervisor:start_child\2</code> function, with the extra parameter of the file pathe e.g.</p> <p>The <code>processing_supervisor</code> should start its children with <code>transient</code> restart policy.</p> <p>So if one of the <code>proc_file</code> servers is crashing it will be restarted, but when they terminate with exit reason <code>normal</code> they are not restarted. So you just exit <code>normal</code> when done and crash when whatever else happens.</p> <p>If you don't overdo it, cyclic polling for files is Ok. If the system becomes loaded because of this polling you can investigate in kernel notification systems (e.g. FreeBSD KQUEUE or the higher level services building upon it on MacOSX) to send you a message when a file appears in a directory. These services however have a complexity because it is necessary for them to throw up their hands if too many events happen (otherwise they wouldn't be a performance improvement but the opposite). So you will have to have a robust polling solution as a fallback anyway.</p> <p>So don't do premature optimization and start with polling, adding improvements (which would be isolated in the <code>dir_watcher</code> servers) when it gets necessary.</p> <hr> <p>Regarding the comment what behaviour to use as <code>dir_watcher</code> process since it doesn't use much of <code>gen_servers</code> functionality:</p> <ul> <li><p>There is no problem with only using part of <code>gen_servers</code> posibilities, in fact it is very common not to use all of it. In your case you only set up a timer in <code>init</code> and use <code>handle_info</code> to do your work. The rest of the <code>gen_server</code> is just the unchanged template.</p></li> <li><p>If you later want changing parameters like poll frequency it is easy to add into this.</p></li> <li><p><code>gen_fsm</code> is much less used since it only fits a quite limited model and is not very flexible. I use it only when it really fits 100% to the requirement (which it does almost never).</p></li> <li><p>In a case where you just want a simple plain Erlang server you can use the spawn functions in <a href="http://www.erlang.org/doc/man/proc_lib.html" rel="nofollow"><code>proc_lib</code></a> to get just the minimal functionality to run under a supervisor.</p></li> <li><p>A interesting way to write more natural Erlang code and still have the OTP advantages is <a href="https://github.com/uwiger/plain_fsm" rel="nofollow"><code>plain_fsm</code></a>, here you have the advantages of selective receive and flexible message handling needed especially when handling protocols paired with the nice features of OTP.</p></li> </ul> <p>Having said all this: if I would write a <code>dir_watcher</code> I'd just use a <code>gen_server</code> and use only what I need. The unused functionality doesn't really cost you anything and everybody understands what it does.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload