StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POUsing Celery with SQLAlchemy and Pyramid
text
Body
copied!<p>I'm creating the web app using Pyramid-1.2.1 with SQLAlchemy as database backend.<br> Now I need to do some periodic tasks along with this app and I want to use Celery as a task processor and SQLAlchemy as the message queue and the result backend. </p> <p>1) I've installed Celery-2.1.4 but I can not figure out how to make it work. Here is my <code>celeryconfig.py</code> file:</p> <pre><code># List of modules to import when celery starts. CELERY_IMPORTS = ("my_app.tasks", ) ## Result store settings. CELERY_RESULT_BACKEND = "database" CELERY_RESULT_DBURI = "sqlite:///MyDataBase.db" CELERY_RESULT_SERIALIZER = "json" ## Broker settings. BROKER_TRANSPORT = "sqlakombu.transport.Transport" BROKER_HOST = "sqlite:///MyDataBase.db" </code></pre> <p>The <code>my_app.tasks</code> contain a simple addition task from the celery examples.<br> Now when I run</p> <pre><code>$ celeryd -l info </code></pre> <p>I see the following:</p> <pre><code>[2011-11-11 20:22:50,750: WARNING/MainProcess] celery@OrangeMac.local v2.1.4 is starting. [2011-11-11 20:22:50,765: WARNING/MainProcess] Configuration -> . broker -> sqlakombu.transport.Transport://guest@sqlite:///MyDataBase.db/ . queues -> . celery -> exchange:celery (direct) binding:celery . concurrency -> 4 . loader -> celery.loaders.default.Loader . logfile -> [stderr]@INFO . events -> OFF . beat -> OFF . tasks -> . chatrooms.task.add [2011-11-11 20:22:50,787: INFO/PoolWorker-1] child process calling self.run() [2011-11-11 20:22:50,789: INFO/PoolWorker-2] child process calling self.run() [2011-11-11 20:22:50,791: INFO/PoolWorker-3] child process calling self.run() [2011-11-11 20:22:50,796: INFO/PoolWorker-4] child process calling self.run() [2011-11-11 20:22:50,802: WARNING/MainProcess] celery@OrangeMac.local has started. [2011-11-11 20:22:50,804: WARNING/MainProcess] Traceback (most recent call last): [2011-11-11 20:22:50,805: WARNING/MainProcess] File "/Users/shashkin/python_v_env/bin/celeryd", line 8, in <module> [2011-11-11 20:22:50,805: WARNING/MainProcess] load_entry_point('celery==2.1.4', 'console_scripts', 'celeryd')() [2011-11-11 20:22:50,805: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/celeryd.py", line 166, in main [2011-11-11 20:22:50,805: WARNING/MainProcess] worker.execute_from_commandline() [2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/base.py", line 40, in execute_from_commandline [2011-11-11 20:22:50,806: WARNING/MainProcess] return self.run(*args, **vars(options)) [2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/celeryd.py", line 85, in run [2011-11-11 20:22:50,806: WARNING/MainProcess] return Worker(**kwargs).run() [2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/apps/worker.py", line 121, in run [2011-11-11 20:22:50,807: WARNING/MainProcess] self.run_worker() [2011-11-11 20:22:50,807: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/apps/worker.py", line 219, in run_worker [2011-11-11 20:22:50,807: WARNING/MainProcess] worker.start() [2011-11-11 20:22:50,807: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/__init__.py", line 217, in start [2011-11-11 20:22:50,808: WARNING/MainProcess] component.start() [2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 238, in start [2011-11-11 20:22:50,808: WARNING/MainProcess] self.reset_connection() [2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 416, in reset_connection [2011-11-11 20:22:50,808: WARNING/MainProcess] self.connection = self._open_connection() [2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 480, in _open_connection [2011-11-11 20:22:50,809: WARNING/MainProcess] max_retries=conf.BROKER_CONNECTION_MAX_RETRIES) [2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/utils/__init__.py", line 276, in retry_over_time [2011-11-11 20:22:50,809: WARNING/MainProcess] retval = fun(*args, **kwargs) [2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 472, in _establish_connection [2011-11-11 20:22:50,809: WARNING/MainProcess] conn.connect() # evaluate connection [2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 170, in connect [2011-11-11 20:22:50,810: WARNING/MainProcess] return self.connection [2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 135, in connection [2011-11-11 20:22:50,810: WARNING/MainProcess] self._connection = self._establish_connection() [2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 148, in _establish_connection [2011-11-11 20:22:50,810: WARNING/MainProcess] return self.create_backend().establish_connection() [2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 161, in create_backend [2011-11-11 20:22:50,810: WARNING/MainProcess] return backend_cls(connection=self) [2011-11-11 20:22:50,811: WARNING/MainProcess] TypeError [2011-11-11 20:22:50,811: WARNING/MainProcess] : [2011-11-11 20:22:50,811: WARNING/MainProcess] __init__() takes exactly 2 arguments (1 given) [2011-11-11 20:22:50,811: INFO/MainProcess] process shutting down </code></pre> <p>What am I missing? </p> <p>2) Later I want to start Celery inside my app. What should I do after celery-pylons package installation? Should I put celery's settings in app's <code>.ini</code> file? Could someone provide me any example, because those I found in the net did not help me much. </p> <p>Thanks.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload