lead-harvester

internal

Built a distributed lead-harvesting pipeline that recursively partitions geographic search areas using a quadtree to overcome API result limits. Cross-references Census boundary data, processes jobs through BullMQ, and sends qualified records into NetSuite.

TypeScriptBullMQNetSuite RESTlet

cat architecture.md

Search API

Geographic search
Business results
Result limits

Quadtree Harvester

Recursive subdivision
Coverage detection
Result aggregation

Census Data

Boundary data
Geographic validation
Area filtering

BullMQ + Redis

Search jobs
Concurrency control
Async processing

NetSuite

Lead creation
Record mapping
CRM workflows

# Why a quadtree?: Geographic search APIs limit the number of results returned for a single area. Recursively subdividing dense regions allows the system to search smaller areas and discover results that would otherwise be hidden by the API's result cap.
# Why a queue?: Large geographic searches can generate many API requests. Queue-backed workers allow searches to run asynchronously while controlling concurrency and keeping long-running workloads out of the application request cycle.