1. 问题现场
1.1 项目目录结构
项目背景：我的项目对异步没有需求，但是有很多工作依赖定时任务。
- main.py
import os
from celery import Celery
app = Celery('devops')
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'devops.settings')
app.config_from_object('logic_celery.config')
app.autodiscover_tasks(packages=['logic_celery.sms'])
- config.py
from datetime import timedelta
broker_url = 'redis://127.0.0.1:6379/0'
result_backend = 'redis://127.0.0.1:6379/1'
beat_schedule = {
'send-msg-per-5s': {
'task': 'tasks.send_sms',
'schedule': timedelta(seconds=5),
'args': (2911, ),
}
}
注意上面两个脚本里暗藏着两个坑。下面会讲到。
- tasks.py
from logic_celery.main import app
import time
import logging
log = logging.getLogger("django")
@app.task # name表示设置任务的名称，如果不填写，则默认使用函数名做为任务名
def send_sms(num):
"""发送短信"""
print("向手机号%s发送短信成功!"%num)
time.sleep(5)
return "send_sms OK"
1.2 启动celery beat
PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main beat -l info
输出：
celery beat v5.1.2 (sun-harmonics) is starting.
__ - ... __ - _
LocalTime -> 2022-05-16 14:48:14
Configuration ->
. broker -> redis://127.0.0.1:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. maxinterval -> 5.00 minutes (300s)
[2022-05-16 14:48:14,141: INFO/MainProcess] beat: Starting...
[2022-05-16 14:48:14,224: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:19,227: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:24,235: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:29,244: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:34,252: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
特别注意beat Scheduler输出：Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)，这个是解决的关键之处。
1.3 启动celery worker
因为我是在windows下开发的，所以worker启动命令要加上-P eventlet参数
PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main worker -l info -P eventlet
输出：
-- ******* ---- Windows-10-10.0.19041-SP0 2022-05-16 14:49:18
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: devops:0x2a5f619f2b0
- ** ---------- .> transport: redis://127.0.0.1:6379/0
- ** ---------- .> results: redis://127.0.0.1:6379/1
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. logic_celery.sms.tasks.send_sms
. logic_celery.sms.tasks.send_sms2
[2022-05-16 14:49:18,244: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2022-05-16 14:49:18,254: INFO/MainProcess] mingle: searching for neighbors
[2022-05-16 14:49:19,285: INFO/MainProcess] mingle: all alone
[2022-05-16 14:49:19,294: INFO/MainProcess] pidbox: Connected to redis://127.0.0.1:6379/0.
[2022-05-16 14:49:19,304: INFO/MainProcess] celery@DESKTOP-cjk ready.
[2022-05-16 14:49:19,525: ERROR/MainProcess] Received unregistered task of type 'tasks.send_sms'.
The message has been ignored and discarded.
Did you remember to import the module containing this task?
Or maybe you're using relative imports?
这里可以看到celery已经发现了我的定时任务，说明我的任务注册是没问题的。
[tasks]
. logic_celery.sms.tasks.send_sms
. logic_celery.sms.tasks.send_sms2
但是，为啥后面又会报这个错呢？
[2022-05-16 14:49:19,525: ERROR/MainProcess] Received unregistered task of type 'tasks.send_sms'.
答案就是task名称不匹配。观察beat的输出就能知道，beat发送的任务名称是config.py里配置的beat_schedule的task字段值（tasks.send_sms）；而worker自动发现注册的任务名称是全称（logic_celery.sms.tasks.send_sms）。这两个对不上号，所以worker就报错KeyError，其实worker没错，beat也没错，只是这俩的信息不对口，牛唇不对马嘴。
2. 解决办法
- 修改config.py中beat_schedule的task字段值，写全称。
beat_schedule = {
'send-msg-per-5s': {
'task': 'logic_celery.sms.tasks.send_sms',
'schedule': timedelta(seconds=5),
'args': (2911, ),
}
}
- 重启beat和worker
PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main beat -l info
celery beat v5.1.2 (sun-harmonics) is starting.
__ - ... __ - _
LocalTime -> 2022-05-16 14:51:00
Configuration ->
. broker -> redis://127.0.0.1:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. logfile -> [stderr]@%INFO
. maxinterval -> 5.00 minutes (300s)
[2022-05-16 14:51:46,035: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (logic_celery.sms.tasks.send_sms)
[2022-05-16 14:51:51,041: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (logic_celery.sms.tasks.send_sms)
PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main worker -l info -P eventlet
--- ***** -----
-- ******* ---- Windows-10-10.0.19041-SP0 2022-05-16 14:49:18
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: devops:0x2a5f619f2b0
- ** ---------- .> transport: redis://127.0.0.1:6379/0
- ** ---------- .> results: redis://127.0.0.1:6379/1
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. logic_celery.sms.tasks.send_sms
. logic_celery.sms.tasks.send_sms2
[2022-05-16 14:49:18,244: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2022-05-16 14:49:18,254: INFO/MainProcess] mingle: searching for neighbors
[2022-05-16 14:49:19,285: INFO/MainProcess] mingle: all alone
[2022-05-16 14:49:19,294: INFO/MainProcess] pidbox: Connected to redis://127.0.0.1:6379/0.
[2022-05-16 14:49:19,304: INFO/MainProcess] celery@DESKTOP-Jaquan ready.
[2022-05-16 14:51:01,075: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[39970550-eb78-49a6-a9db-4bf99bfd90ff] received
[2022-05-16 14:51:01,075: WARNING/MainProcess] 向手机号2911发送短信成功!
[2022-05-16 14:51:01,075: WARNING/MainProcess]
[2022-05-16 14:51:05,997: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[707fc868-d472-49f6-b5ea-0693c0fc37b0] received
[2022-05-16 14:51:05,997: WARNING/MainProcess] 向手机号2911发送短信成功!
[2022-05-16 14:51:05,997: WARNING/MainProcess]
[2022-05-16 14:51:06,078: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[39970550-eb78-49a6-a9db-4bf99bfd90ff] succeeded in 5.0s: 'send_sms OK'
[2022-05-16 14:51:11,003: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[707fc868-d472-49f6-b5ea-0693c0fc37b0] succeeded in 5.0s: 'send_sms OK'
完美解决！
