Skip to main content

​
Background Workers

Run background jobs, process queues, and execute async tasks without managing infrastructure. Background Workers automatically scale based on workload.

​
What are Background Workers?

Background Workers are specialized services designed to process asynchronous tasks, queues, and scheduled jobs. They run continuously and automatically scale based on queue depth and workload.

​
Key Features

  • Queue Processing: Process jobs from message queues
  • Auto-scaling: Automatically scale workers based on queue size
  • Retry Logic: Automatic retries for failed jobs
  • Dead Letter Queue: Handle permanently failed jobs
  • Monitoring: Track job processing metrics
  • Cost Efficient: Pay only for active processing time

​
Use Cases

  • Processing user uploads and media
  • Sending batch emails and notifications
  • Data synchronization and ETL jobs
  • Report generation and exports
  • Image processing and optimization
  • Payment processing
  • Integration with third-party APIs

​
Supported Queue Types

  • Redis: Redis-compatible queues
  • RabbitMQ: AMQP message queues
  • PostgreSQL: Database-backed queues
  • SQS: Amazon SQS integration
  • Custom: Your own queue implementation

​
Getting Started

  1. Create a Background Worker service
  2. Connect your code that processes jobs
  3. Configure queue settings and connection strings
  4. Set up retry logic and error handling
  5. Deploy and start processing jobs

​
Configuration Example

// Worker configuration
const worker = new AntrykWorker({
  queue: process.env.REDIS_URL,
  concurrency: 10,
  retryLimit: 3,
  retryDelay: 5000
});

worker.on('job', async (job) => {
  // Process the job
  await processJob(job.data);
});

​
Scaling Configuration

Configure auto-scaling with:
  • Min instances: Minimum number of workers
  • Max instances: Maximum number of workers
  • Queue threshold: Queue depth to trigger scaling
  • Scale-down delay: Time before scaling down

​
Pricing

Background Workers are billed by:
  • Worker hours: Time workers are running
  • CPU/Memory: Resource allocation
  • Queue size: Storage for queued jobs

Deploy Your First Worker

Get started with background workers