​
Core Components

​
Engine

The Engine is the central component that:

  • Manages task execution
  • Coordinates resources
  • Handles caching
  • Controls parallelism
from blastai import Engine

engine = await Engine.create(
    settings=settings,
    constraints=constraints
)

​
Scheduler

The Scheduler manages task execution:

  • Tracks task states
  • Handles task dependencies
  • Manages execution order
  • Coordinates parallel tasks

Task priorities:

  1. Tasks with cached results
  2. Tasks with cached plans
  3. Subtasks of running tasks
  4. Tasks with paused executors
  5. Remaining tasks (FIFO)

Tasks can be in different states:

  • Scheduled: Task is queued for execution
  • Running: Task is currently executing
  • Completed: Task has finished execution

​
Resource Manager

Handles system resources:

  • Browser instances
  • Memory usage
  • Cost tracking
  • Resource cleanup

​
Cache Manager

Manages two types of caches both in memory and on disk:

  • Results cache (task outputs)
  • Plans cache (execution plans generated by LLM)

​
Planner

Generates natural language execution plans given user-provided task description.

​
Data Flow

  1. Task Creation

    task_id = scheduler.schedule_task(
    description="Search Python docs",
    cache_control=""
)

  2. Cache Check

    • Check results cache
    • Check plans cache
    • Return cached result if available

  3. Resource Allocation

    • Wait for prerequisites
    • Allocate browser if needed
    • Assign executor

  4. Execution

    • Run task via executor
    • Stream progress updates
    • Cache results

  5. Cleanup

    • Release resources
    • Update cache
    • Handle errors

​
Code Structure

blastai/
├── __init__.py      # Package initialization
├── engine.py        # Main engine implementation
├── scheduler.py     # Task scheduling
├── cache.py         # Caching system
├── config.py        # Configuration
├── planner.py       # Task planning
├── executor.py      # Task execution
├── tools.py         # Tools for parallelism
└── utils.py         # Utilities

​
Configuration

Settings and constraints control behavior:

settings:
  persist_cache: true
  blastai_log_level: "info"

constraints:
  max_memory: "4GB"
  max_concurrent_browsers: 4
  allow_parallelism:
    task: true
    data: true

​
Error Handling

BLAST handles various error types:

  • Browser errors
  • Resource limits
  • Task failures
  • Cache issues

Error recovery:

  1. Log error details
  2. Clean up resources
  3. Retry if possible
  4. Report to user

​
Extending BLAST

You can extend BLAST by:

  1. Adding tools for further optimization
  2. Creating custom executors
  3. Tune better scheduling policy

​
Next Steps