Yes, Celery may be a vegetable, but it's also an open source asynchronous task queue/job queue which is based on distributed message passing. Tasks are the building blocks of Celery applications.
Filipe A Ximenes, partner at Vinta Software, has created a useful checklist for building great Celery tasks! We've repurposed his checklist in Taskade into this celery tasks checklist template so that you can easily use it in conjunction with your other projects and templates.
What is celery distributed task queue?
Celery is a distributed task queue where you can do work asynchronously and process the workload in background. Using celery, we don't need to wait for processing our tasks, but we can run them asynchronously.
To use celery, first you have to install it on your machine – whether that's a server or your local machine.
Celery has a built-in message broker which you can use to pass messages between your application and the worker processes. Celery is often used with a message broker like RabbitMQ or Redis.
What is celery tasks management?
Celery task checklist - Tasks are units of work that your application performs. You can break down your tasks into smaller units of work, and each unit of work becomes a separate task which you can run asynchronously.
Task queues can drastically improve the performance of your application. With a task queue, you push tasks onto the queue, and the task queue pushes them onto one or more worker machines. The workers process the tasks asynchronously in the background.
Celery allows you to run any kind of Python code, including regular functions and class methods, as tasks. You can also pass arguments to your tasks and return values.
Celery makes it easy to scale up your worker machines by delegating the task queue to one or more worker machines, without getting in the way of the application. It simply becomes part of an existing architecture like Django's database layer or Flask's WSGI layer.
Celery gives you fine-grained control of task execution. It lets you run tasks on a single machine, or in a cluster of machines, and it uses the AMQP protocol to get tasks between clients and workers.
Task queues have unlimited potential. They can be used for processing tasks images, uploading files, generating thumbnails, sending email via a transactional email service, transcoding video and audio files with FFmpeg, performing calculations in the cloud using Boto, among many other applications.
What is a celery task?
A Celery Task is basically just a Python function that you want to run in the background. You can use any regular Python function as a Celery Task. A Celery Task must always return a TaskResult, even if that's just the default result (usually None).
Celery allows you to schedule tasks to be executed at any time after they are queued. You can set this delay to be relative (for example 5 seconds from now) or absolute (for example from the time the message was sent).
What is a celery beat?
A beat is a scheduler process, which sends scheduled tasks out to the worker nodes.
Task results are sent back to your app via regular task callbacks or celery events so you can use them in monitoring tools.
How do you monitor celery beats?
Celery comes with a monitoring dashboard that you can use to check the status of your tasks. You can access it from a web browser at http://localhost:5555
The dashboard shows statistics for recently completed tasks, as well as those which are currently being processed by workers. It also graphs task durations and how long it took each worker to process each task.
What is a celery worker?
A Celery Worker is basically just a Python process that executes the tasks you want, as instructed by the Beat scheduler.
You can have as many workers as you like, and configure them all separately. You often have several workers, but one scheduler.
Tasks are executed by one or more worker processes. You can use any programming language that can work with the supported message brokers to write your worker tasks, which makes Celery very flexible.
Celery workers do all of the hard work in a Celery installation. They handle requests from the web server, scheduled tasks and events among other things.