没有合适的资源？快使用搜索试试~ 我知道了~

首页Celery 4.1.0官方文档：入门与特性概述

Celery 4.1.0官方文档：入门与特性概述

celery

需积分: 10 3 下载量 48 浏览量更新于2024-07-18 收藏 2.63MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

试读

714页

Celery 4.1.0 是一个功能强大且灵活的分布式任务队列和工作流管理系统，专注于实时处理和任务调度。它设计用于处理大量消息，同时提供必要的工具来维护这样一个系统。作为一款开源软件，Celery基于 BSD 许可证，这意味着它的源代码是公开可用并鼓励社区参与。对于初学者而言，Celery 4.1.0文档提供了详细的入门指南，包括“First Steps with Celery”教程，这是一条清晰的路径，帮助新用户快速了解如何设置、配置和使用Celery。教程覆盖了从安装、配置到创建和管理任务的基本步骤，确保用户能够轻松上手。此外，文档还包含了常见问题解答（FAQ），这是解决初次使用过程中遇到问题的重要资源。FAQ通常涵盖了各种常见的问题和解决方案，可以帮助用户避免许多初期陷阱，并加速学习过程。整个文档结构清晰，分为两个主要章节：第一章“Getting Started”是入门章节，适合新手入门，而第二章则可能涵盖了更深入的内容，如版权信息、Celery User Manual 的版权声明以及所有者AskSolem对材料使用的权利限制。Python模块索引和参考文献列表也提供了进一步的技术支持和学术支持，使得用户在遇到特定功能或技术细节时能迅速找到所需的信息。 Celery 4.1.0文档旨在为用户提供全面的指导，无论是从零开始的用户还是经验丰富的开发人员，都能从中获取到有价值的资源。通过这个版本，开发者可以构建高效、可靠的分布式任务处理系统，同时享受到活跃的社区支持和持续的更新维护。

资源详情

资源推荐

Celery Documentation, Release 4.1.0

With git

Please the Contributing section.

Brokers

Release 4.1

Date Jul 24, 2017

Celery supports several message transport alternatives.

Broker Instructions

Using RabbitMQ

• Installation & Conﬁguration

• Installing the RabbitMQ Server

– Setting up RabbitMQ

– Installing RabbitMQ on macOS

Conﬁguring the system host name

Starting/Stopping the RabbitMQ server

Installation & Conﬁguration

RabbitMQ is the default broker so it doesn’t require any additional dependencies or initial conﬁguration, other than

the URL location of the broker instance you want to use:

broker_url = 'amqp://myuser:mypassword@localhost:5672/myvhost'

For a description of broker URLs and a full list of the various broker conﬁguration options available to Celery, see

Broker Settings, and see below for setting up the username, password and vhost.

Installing the RabbitMQ Server

See Installing RabbitMQ over at RabbitMQ’s website. For macOS see Installing RabbitMQ on macOS.

Note: If you’re getting nodedown errors after installing and using rabbitmqctl then this blog post can help you

identify the source of the problem:

http://www.somic.org/2009/02/19/on-rabbitmqctl-and-badrpcnodedown/

12 Chapter 2. Contents

Celery Documentation, Release 4.1.0

Setting up RabbitMQ

To use Celery we need to create a RabbitMQ user, a virtual host and allow that user access to that virtual host:

$ sudo rabbitmqctl add_user myuser mypassword

$ sudo rabbitmqctl add_vhost myvhost

$ sudo rabbitmqctl set_user_tags myuser mytag

$ sudo rabbitmqctl set_permissions -p myvhost myuser ".

" ".

Substitute in appropriate values for myuser, mypassword and myvhost above.

See the RabbitMQ Admin Guide for more information about access control.

Installing RabbitMQ on macOS

The easiest way to install RabbitMQ on macOS is using Homebrew the new and shiny package management system

for macOS.

First, install Homebrew using the one-line command provided by the Homebrew documentation:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Finally, we can install RabbitMQ using brew:

$ brew install rabbitmq

After you’ve installed RabbitMQ with brew you need to add the following to your path to be able to start and stop

the broker: add it to the start-up ﬁle for your shell (e.g., .bash_profile or .profile).

PATH=$PATH:/usr/local/sbin

Conﬁguring the system host name

If you’re using a DHCP server that’s giving you a random host name, you need to permanently conﬁgure the host

name. This is because RabbitMQ uses the host name to communicate with nodes.

Use the scutil command to permanently set your host name:

$ sudo scutil --set HostName myhost.local

Then add that host name to /etc/hosts so it’s possible to resolve it back into an IP address:

127.0.0.1 localhost myhost myhost.local

If you start the rabbitmq-server, your rabbit node should now be rabbit@myhost, as veriﬁed by rabbitmqctl:

$ sudo rabbitmqctl status

Status of node rabbit@myhost ...

[{running_applications,[{rabbit,"RabbitMQ","1.7.1"},

{mnesia,"MNESIA CXC 138 12","4.4.12"},

{os_mon,"CPO CXC 138 46","2.2.4"},

2.2. Getting Started 13

Celery Documentation, Release 4.1.0

{sasl,"SASL CXC 138 11","2.1.8"},

{stdlib,"ERTS CXC 138 10","1.16.4"},

{kernel,"ERTS CXC 138 10","2.13.4"}]},

{nodes,[rabbit@myhost]},

{running_nodes,[rabbit@myhost]}]

...done.

This is especially important if your DHCP server gives you a host name starting with an IP address, (e.g.,

23.10.112.31.comcast.net). In this case RabbitMQ will try to use rabbit@23: an illegal host name.

Starting/Stopping the RabbitMQ server

To start the server:

$ sudo rabbitmq-server

you can also run it in the background by adding the -detached option (note: only one dash):

$ sudo rabbitmq-server -detached

Never use kill (kill(1)) to stop the RabbitMQ server, but rather use the rabbitmqctl command:

$ sudo rabbitmqctl stop

When the server is running, you can continue reading Setting up RabbitMQ.

Using Redis

Installation

For the Redis support you have to install additional dependencies. You can install both Celery and these dependencies

in one go using the celery[redis] bundle:

$ pip install -U "celery[redis]"

Conﬁguration

Conﬁguration is easy, just conﬁgure the location of your Redis database:

app.conf.broker_url = 'redis://localhost:6379/0'

Where the URL is in the format of:

redis://:password@hostname:port/db_number

all ﬁelds after the scheme are optional, and will default to localhost on port 6379, using database 0.

If a Unix socket connection should be used, the URL needs to be in the format:

redis+socket:///path/to/redis.sock

Specifying a different database number when using a Unix socket is possible by adding the virtual_host param-

eter to the URL:

14 Chapter 2. Contents

Celery Documentation, Release 4.1.0

redis+socket:///path/to/redis.sock?virtual_host=db_number

Visibility Timeout

The visibility timeout deﬁnes the number of seconds to wait for the worker to acknowledge the task before the message

is redelivered to another worker. Be sure to see Caveats below.

This option is set via the broker_transport_options setting:

app.conf.broker_transport_options = {'visibility_timeout': 3600} # 1 hour.

The default visibility timeout for Redis is 1 hour.

Results

If you also want to store the state and return values of tasks in Redis, you should conﬁgure these settings:

app.conf.result_backend = 'redis://localhost:6379/0'

For a complete list of options supported by the Redis result backend, see Redis backend settings

Caveats

Fanout preﬁx

Broadcast messages will be seen by all virtual hosts by default.

You have to set a transport option to preﬁx the messages so that they will only be received by the active virtual host:

app.conf.broker_transport_options = {'fanout_prefix': True}

Note that you won’t be able to communicate with workers running older versions or workers that doesn’t have this

setting enabled.

This setting will be the default in the future, so better to migrate sooner rather than later.

Fanout patterns

Workers will receive all task related events by default.

To avoid this you must set the fanout_patterns fanout option so that the workers may only subscribe to worker

related events:

app.conf.broker_transport_options = {'fanout_patterns': True}

Note that this change is backward incompatible so all workers in the cluster must have this option enabled, or else they

won’t be able to communicate.

This option will be enabled by default in the future.

2.2. Getting Started 15

Celery Documentation, Release 4.1.0

Visibility timeout

If a task isn’t acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.

This causes problems with ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout; in fact

if that happens it will be executed again, and again in a loop.

So you have to increase the visibility timeout to match the time of the longest ETA you’re planning to use.

Note that Celery will redeliver messages at worker shutdown, so having a long visibility timeout will only delay the

redelivery of ‘lost’ tasks in the event of a power failure or forcefully terminated workers.

Periodic tasks won’t be affected by the visibility timeout, as this is a concept separate from ETA/countdown.

You can increase this timeout by conﬁguring a transport option with the same name:

app.conf.broker_transport_options = {'visibility_timeout': 43200}

The value must be an int describing the number of seconds.

Key eviction

Redis may evict keys from the database in some situations

If you experience an error like:

InconsistencyError: Probably the key ('_kombu.binding.celery') has been

removed from the Redis database.

then you may want to conﬁgure the redis-server to not evict keys by setting the timeout parameter to 0 in the

redis conﬁguration ﬁle.

Using Amazon SQS

Installation

For the Amazon SQS support you have to install the boto library using pip:

$ pip install -U boto

Conﬁguration

You have to specify SQS in the broker URL:

broker_url = 'sqs://ABCDEFGHIJKLMNOPQRST:ZYXK7NiynGlTogH8Nj+P9nlE73sq3@'

where the URL format is:

sqs://aws_access_key_id:aws_secret_access_key@

you must remember to include the “@” at the end.

The login credentials can also be set using the environment variables AWS_ACCESS_KEY_ID and

AWS_SECRET_ACCESS_KEY, in that case the broker URL may only be sqs://.

16 Chapter 2. Contents

剩余713页未读，继续阅读

粉丝: 0
资源: 2

上传资源快速赚钱

我的内容管理展开

我的资源快来上传第一个资源

我的收益

登录查看自己的收益

我的积分登录查看自己的积分

我的C币登录后查看C币余额

我的收藏

我的下载

下载帮助

Celery 4.1.0官方文档：入门与特性概述

Python Django2.0集成Celery4.1教程

详解django+django-celery+celery的整合实战

celery-v4.3.0中文.pdf

celery和flask-celery

运行celery提示bash: celery: command not found

django的celery使用

ImportError: cannot import name 'Celery' from 'celery' (C:\Users\刘凯\Envs\flask-env\lib\site-packages\celery\__init__.py)

django-celery使用

python异步请求celery

django中celery和redis

: celery commandnot found

celery如何实现定时执行

liunx怎么运行celery

django服务中如何启动celery

python celery 和 scrpay结合

django celery 定时任务demo

from celery.utils.timeutils import timedelta_seconds ModuleNotFoundError: No module named 'celery.utils.timeutils'

django 中如何使用celery

python Celery使用方法

python celery 定时任务demo

最新资源

ImportError: cannot import name 'Celery' from 'celery' (C:\Users\刘凯\Envs\flask-env\lib\site-packages\celery\init.py)