Skip to content

Why Repster?

Repster is the lightweight way to schedule, run, and observe dlt pipelines. dlt itself is a library: it loads data brilliantly, but nothing in it runs your pipeline at 6am, records whether it worked, or retries it when it didn't. That's the gap Repster fills, and this page compares it to the other ways of filling it.

Scheduling dlt pipelines with cron

Cron fires your script, and that's all it does. In practice you end up hand-building the rest:

  • Capturing logs somewhere you can actually find them
  • Recording which runs happened, and whether they succeeded
  • Retrying failures without re-running everything by hand
  • Getting secrets onto the machine safely
  • Answering "did last night's run work?" without SSHing anywhere

Repster gives you all of that with one decorator and a .env file, plus a web UI to see it. And when a laptop or a lone VPS stops being enough, rp deploy moves the same code to Repster Cloud (managed scheduling, a secrets vault, an isolated worker per run), no rewrite required.

Repster vs. Airflow, Dagster, and Prefect

Orchestrators are the right tool when you have DAGs of heterogeneous tasks: ML jobs, dbt runs, custom scripts with dependencies between them. But if what you're orchestrating is dlt pipelines, an orchestrator is a lot of machinery (a scheduler service, workers, a metadata database, and upgrades to manage) before your first pipeline runs.

There are two ways to use Repster here:

Replace the orchestrator you were about to deploy. If your only workloads are dlt pipelines, @rp.flow(schedule=...) covers scheduling, retries, logs, and history, whether on your own machine with rp dev or with no infrastructure at all on Repster Cloud.

Keep your orchestrator, but separate it from your compute. Orchestrators are best at coordination, not at being an execution environment for resource-hungry EL work. Dagster's own guidance makes the case: "We need to separate infra from business-heavy ETL in the data pipeline". The orchestrator keeps the DAG logic and triggers flows via the CLI or API, while Repster Cloud owns the execution layer: an isolated serverless worker per run, secrets injection, log capture. Your orchestrator gets lighter, and your pipelines stop competing with it for resources.

Repster is deliberately not a general-purpose task runner. If you need to run arbitrary non-dlt workloads on a schedule, use an orchestrator for those. See How Repster works for where the boundary sits.

Repster vs. Fivetran and Airbyte

Managed ELT platforms trade control for convenience. Repster keeps the convenience and gives the control back:

  • The code is yours. Pipelines are plain dlt in your repo: reviewed in PRs, versioned in git, portable to anywhere Python runs. No connector configs locked in a vendor GUI.
  • A real local development workflow. The same engine runs your flow locally and in the cloud, so you test the actual pipeline before deploying, instead of configuring a connector in a web console and finding out in production. (This is the parity story tools like Dagster sell, applied to data loading.)
  • Pricing tied to compute, not volume. Fivetran and Airbyte pricing scales with rows or MTU, so your bill grows with your data even when the work doesn't. Repster is free during the design-partner phase; the metered billing it's building toward charges for raw compute — CPU seconds, memory, storage — not rows. See Pricing.

Repster vs. dltHub's hosted platform

dltHub (the team behind dlt) offers its own hosted platform, and it's a good fit if you want a deeply integrated, batteries-included dlt experience from one vendor. Repster is the lightweight alternative: a thin operational layer over dlt that you can run entirely locally, adopt one decorator at a time, and leave whenever you like. Your flows are plain dlt underneath, so nothing locks you in. We compete on simplicity, transparent compute pricing, and code ownership.

Built for AI coding agents

Most data platforms assume a human clicking through a web console, which makes them opaque to AI agents. Repster's entire surface is code and CLI: pipelines and schedules are Python in your repo, and every operation around them (run, deploy, secrets, logs, teams, keys) is an rp command. That's a guarantee, not an accident: no operation ships in the UI without a CLI equivalent, so anything you can do, your agent can do. The package even ships an agent skill, and these docs publish as llms.txt. See Use Repster with AI coding agents.

When not to use Repster

Repster stays small on purpose. Reach for something else when you need:

  • Non-dlt workloads on a schedule: use an orchestrator (Dagster, Airflow, Prefect) or plain cron
  • Transformations: Repster loads data; model it downstream with dbt or SQLMesh
  • A data catalog, lineage, or alerting suite: pair Repster with dedicated tools
  • Streaming: Repster runs batch dlt pipelines

If your problem is "I have dlt pipelines and I want them to run reliably, on a schedule, with logs and retries, without standing up an orchestrator", Repster is built for exactly that. Start with the Quickstart.