Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 74 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,78 @@ parallelism on CPU-bound work.

## Features

- **Reliability** — retries with exponential backoff, dead letter queue with replay, circuit breakers, exception filtering
- **Scheduling** — priorities, rate limiting, periodic (cron) tasks, delayed execution, job expiration
- **Workflows** — `chain` / `group` / `chord`, task dependencies with cascade cancel
- **Control** — cooperative cancellation, soft timeouts, unique/idempotent tasks, queue pause/resume
- **Observability** — web dashboard, events, HMAC-signed webhooks, structured logging, worker heartbeats
- **Extensibility** — pluggable serializers, per-task middleware, async API, Postgres/Redis backends
Grouped by what you're trying to do — each section has a one-line sample and a
link to its deep-dive guide. New here? Start with
**[Capabilities at a glance](https://docs.byteveda.org/taskito/docs/capabilities)**.

### Reliability

Retries with exponential backoff, per-exception retry rules, soft timeouts, a
dead-letter queue with replay, circuit breakers, and idempotent enqueue.

```python
@queue.task(max_retries=5, retry_backoff=2.0, retry_on=[TimeoutError])
def fetch_url(url: str) -> str: ...
```

[→ Reliability guide](https://docs.byteveda.org/taskito/docs/guides/reliability)

### Workflows

Compose tasks with `chain`, fan out with `group`, fan in with `chord` — plus
task dependency graphs with cascade cancel.

```python
chain(fetch.s(url), parse.s(), store.s()).apply()
```

[→ Workflows guide](https://docs.byteveda.org/taskito/docs/guides/workflows/canvas)

### Concurrency

A thread pool by default (ideal for I/O-bound tasks); switch to a **prefork**
pool of child processes for true CPU parallelism with no GIL contention.

```bash
taskito worker --pool prefork --app tasks:queue # CPU-bound: real parallelism
```

[→ Execution & prefork guide](https://docs.byteveda.org/taskito/docs/guides/advanced-execution/prefork)

### Scheduling

Priorities, rate limiting, periodic (cron) tasks, delayed execution, and job
expiration.

```python
@queue.task(priority=9, rate_limit="100/m")
def notify(user_id: int) -> None: ...
```

[→ Scheduling guide](https://docs.byteveda.org/taskito/docs/guides/core/scheduling)

### Observability

A built-in web dashboard, an events system, HMAC-signed webhooks, Prometheus and
OpenTelemetry exporters, structured logging, and worker heartbeats.

```bash
taskito dashboard --app tasks:queue # Flower-style monitoring UI
```

[→ Dashboard & monitoring guide](https://docs.byteveda.org/taskito/docs/guides/dashboard)

### Extensibility

Pluggable serializers, per-task middleware, a fully async API, and Postgres or
Redis backends for multi-machine workers.

```python
@queue.task(middleware=[MyMiddleware()]) # per-task hooks
def handle(payload: dict) -> None: ...
```

[→ Extensibility guide](https://docs.byteveda.org/taskito/docs/guides/extensibility)

## Examples

Expand Down Expand Up @@ -124,9 +190,11 @@ Coming from Celery? See the **[Migration Guide](https://docs.byteveda.org/taskit
| Rate limiting | **Yes** | Yes | No | Yes | No |
| Dead letter queue | **Yes** | No | Yes | No | No |
| Task dependencies | **Yes** | No | No | No | No |
| Workflows (chain/group/chord) | **Yes** | Yes | No | Yes | No |
| Built-in dashboard | **Yes** | No | No | No | No |
| FastAPI integration | **Yes** | No | No | No | No |
| Cancel running tasks | **Yes** | Yes | No | No | No |
| CPU parallelism (prefork pool) | **Yes** | Yes | Yes | Yes | Yes |
| Postgres backend | **Yes** | Yes | No | No | No |
| Setup | **`pip install`** | Broker + backend | Redis | Broker | Redis |

Expand Down
91 changes: 91 additions & 0 deletions docs/content/docs/capabilities.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: Capabilities at a glance
description: Everything taskito already does, in one place — with a link to the deep-dive guide for each.
---

import { Cards, Card } from "fumadocs-ui/components/card";
import {
ShieldCheck,
Workflow,
Cpu,
CalendarClock,
Activity,
Plug,
} from "lucide-react";

taskito ships a broad feature set out of the box. If you're evaluating it — or
suspect a capability is missing — scan this page first. Every card links to the
guide that documents it.

## If you think it's missing, it isn't

These are the capabilities most often assumed absent. They aren't.

| You might assume… | Reality |
| --- | --- |
| No canvas / workflows — no chain, group, or chord | [`chain` / `group` / `chord`](/guides/workflows/canvas) are exported from the top-level package, backed by a full workflow DSL. |
| No Flower-like dashboard | [`taskito dashboard --app myapp:queue`](/guides/dashboard) serves a built-in monitoring UI. |
| No retry backoff or per-exception retries | [`max_retries`, `retry_backoff`, `retry_on`, `dont_retry_on`](/guides/reliability/retries) are arguments on every task. |
| No soft timeouts | [`current_job.check_timeout()`](/guides/reliability/error-handling) gives cooperative soft timeouts. |
| GIL bottleneck with no escape hatch | [`--pool prefork`](/guides/advanced-execution/prefork) runs child processes with independent GILs for true CPU parallelism. |

## By capability

<Cards>
<Card
icon={<ShieldCheck />}
title="Reliability"
href="/guides/reliability"
description="Retries with exponential backoff, per-exception rules, soft timeouts, dead-letter replay, circuit breakers, and idempotent enqueue."
/>
<Card
icon={<Workflow />}
title="Workflows"
href="/guides/workflows/canvas"
description="Compose with chain, fan out with group, fan in with chord — plus dependency graphs with cascade cancel."
/>
<Card
icon={<Cpu />}
title="Concurrency"
href="/guides/advanced-execution/prefork"
description="A thread pool for I/O-bound work and a prefork pool for true CPU parallelism, with a native async API."
/>
<Card
icon={<CalendarClock />}
title="Scheduling"
href="/guides/core/scheduling"
description="Priorities, rate limiting, periodic (cron) tasks, delayed execution, and job expiration."
/>
<Card
icon={<Activity />}
title="Observability"
href="/guides/dashboard"
description="A built-in dashboard, events, HMAC-signed webhooks, Prometheus, OpenTelemetry, and structured logging."
/>
<Card
icon={<Plug />}
title="Extensibility"
href="/guides/extensibility"
description="Pluggable serializers, per-task middleware, a fully async API, and Postgres or Redis backends."
/>
</Cards>

## One-liners

```python
# Reliability — backoff + per-exception retry rules
@queue.task(max_retries=5, retry_backoff=2.0, retry_on=[TimeoutError])
def fetch_url(url: str) -> str: ...

# Workflows — sequential pipeline (group / chord for fan-out / fan-in)
chain(fetch.s(url), parse.s(), store.s()).apply()

# Scheduling — priority + rate limit
@queue.task(priority=9, rate_limit="100/m")
def notify(user_id: int) -> None: ...
```

```bash
taskito worker --pool prefork --app tasks:queue # CPU-bound: true parallelism
taskito dashboard --app tasks:queue # Flower-style monitoring UI
```
5 changes: 5 additions & 0 deletions docs/content/docs/getting-started/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ description: "Install taskito and prepare your environment."

import { Callout } from "fumadocs-ui/components/callout";

<Callout title="New to taskito?">
See [**Capabilities at a glance**](/capabilities) for everything taskito does —
workflows, retries, the dashboard, prefork pools — each with a link to its guide.
</Callout>

## From PyPI

```bash
Expand Down
4 changes: 3 additions & 1 deletion docs/content/docs/guides/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ import {
} from "lucide-react";

Pick the topic you're working through. Each guide is self-contained, with code
samples, gotchas, and links to the relevant API reference.
samples, gotchas, and links to the relevant API reference. For a one-page
overview of everything taskito does, see
[**Capabilities at a glance**](/capabilities).

<Cards>
<Card
Expand Down
1 change: 1 addition & 0 deletions docs/content/docs/meta.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"title": "Taskito",
"pages": [
"capabilities",
"getting-started",
"guides",
"architecture",
Expand Down