netsuite-lead-generator

internal

A distributed lead-harvesting pipeline that recursively partitions geographic search areas using a quadtree to overcome API result limits. Cross-references Census boundary data, processes jobs through BullMQ, and sends qualified records into NetSuite.

TypeScriptNetSuite SuiteScript 2.1NetSuite SuiteletNetSuite RESTletNetSuite Map/ReduceSuiteQLHMAC-SHA256

cat architecture.md

NetSuite Suitelet Interface

Serves the user-facing lead generation form inside NetSuite.
Injects a React frontend bundle from a script parameter.
Passes current NetSuite user, role, environment, subsidiary, and role subsidiary restrictions to the client.

RESTlet API Layer

Receives normalized action-based POST payloads from the Suitelet and queue workflow.
Routes requests for lead data harvesting, staging record creation, Map/Reduce scheduling, and email notifications.
Returns structured success or error responses while logging failures through NetSuite logging APIs.

Queue Server and Lead Harvesting

Uses an authenticated RESTlet request to start an external queue-server harvesting job.
Harvesting uses a quadtree search strategy to work around API result limits for dense areas.
Batches harvested business results back into NetSuite through the RESTlet for staging.

NetSuite Staging Records

Stores incoming business records in a custom NetSuite staging record before lead creation.
Maps business name, phone, address, city, state, zip code, source URL, processing status, and raw JSON.
Uses the source system business ID as both the external ID and native record name so NetSuite can block duplicate staging entries.

Map/Reduce Lead Processing

Queries pending staging records with SuiteQL for asynchronous processing.
Checks for existing NetSuite customers using address and zip-code matching before creating new leads.
Marks staging records as duplicate, created, or failed and links them to the matched customer or created lead where applicable.

Notifications and Cleanup

Sends harvest and processing summary emails from NetSuite after key workflow milestones.
Counts created leads, duplicate matches, and processing failures during the Map/Reduce summarize phase.
Supports optional cleanup of successfully processed staging records through a script parameter.

Security and Access Controls

Signs queue-server requests with an HMAC-SHA256 signature generated from a NetSuite secret.
Stores integration endpoint and script deployment identifiers in NetSuite script parameters.
Custom-record access is restricted to integration roles and scripts, with administrator audit access and system notes enabled.

# Why stage leads before creating NetSuite records?: The staging record isolates external lead payloads from NetSuite lead creation, reducing timeout risk and giving the system a durable processing status for pending, duplicate, created, and failed records.
# Why use Map/Reduce for processing?: NetSuite Map/Reduce lets pending staging records be processed asynchronously, which fits the need to handle larger imports without blocking the user-facing Suitelet or RESTlet flow.
# Why sign queue-server requests?: The RESTlet signs outbound queue requests with HMAC-SHA256 using a NetSuite-managed secret, giving the external queue workflow a way to verify request integrity without exposing secret values in source code.
# Why keep lead harvesting outside NetSuite?: The queue server owns the lead harvesting job and batching behavior, while NetSuite remains responsible for staging, deduplication, lead creation, and user notifications. This keeps long-running external API work out of SuiteScript execution paths.
# Why deduplicate by address and zip code?: The Map/Reduce processor checks existing NetSuite customer addresses before creating leads, which helps prevent duplicate customer records when lead business names vary but location data still matches.