StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POErlang OTP application design
text
Body
copied!<p>I am struggling a little coming to grips with the OTP development model as I convert some code into an OTP app. </p> <p>I am essentially making a web crawler and I just don't quite know where to put the code that does the actual work.</p> <p>I have a supervisor which starts my worker:</p> <pre><code>-behaviour(supervisor). -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}). init(_Args) -> Children = [ ?CHILD(crawler, worker) ], RestartStrategy = {one_for_one, 0, 1}, {ok, {RestartStrategy, Children}}. </code></pre> <p>In this design, the Crawler Worker is then responsible for doing the actual work:</p> <pre><code>-behaviour(gen_server). start_link() -> gen_server:start_link(?MODULE, [], []). init([]) -> inets:start(), httpc:set_options([{verbose_mode,true}]), % gen_server:cast(?MODULE, crawl), % ok = do_crawl(), {ok, #state{}}. do_crawl() -> % crawl! ok. handle_cast(crawl}, State) -> ok = do_crawl(), {noreply, State}; </code></pre> <p><strong>do_crawl</strong> spawns a fairly large number of processes and requests that handle the work of crawling via http. </p> <p>Question, ultimately is: where should the actual crawl happen? As can be seen above I have been experimenting with different ways of triggering the actual work, but still missing some concept essential for grokering the way things fit together. </p> <p><em>Note:</em> some of the OTP plumbing is left out for brevity - the plumbing is all there and the system all hangs together </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload