Lead Verification Workflow: Normalize, Dedupe, Cheapest Check First
Leila owns Friday's outbound queue at a São Paulo BNPL desk. The sheet in her inbox this morning is 18,400 rows from three vendors, and last month she learned that "hit every channel on every cell" is not a verification workflow. The same mobile arrived as +55 11 9…, as 00155…, and as a paste with spaces. She paid five existence checks, then an avatar fetch, then a Telegram peer lookup, for one person already on the March suppress table. What she runs now is a production job: shrink the file, canonicalize once, spend the cheapest check first, write a checkpoint after every batch, and treat a crash as a resume — not a reason to start the bill over.
Opening this page or copying a snippet does not send a lookup. Running the script does, and it spends credits. Sign in, copy an authorized platform key from the dashboard, and export it as OPENDATA_API_KEY on the host that opens the CSV. YOUR_API_KEY in examples is a placeholder. The key rides in the query string, so those URLs do not belong in tickets, screenshots, git, or a column in the sheet.
Shrink the list before you spend
A cross-channel job has one expensive habit: it treats every row as a new person on every platform. Credits go to parse failures, to the same E.164 written three ways, to profile calls on numbers that are not registered, and to a second channel you will never message. The runbook order is the opposite of a demo: drop junk, collapse duplicates, fire the cheapest existence check that actually matches the campaign, persist the raw body, and only then decide whether a second channel or an enrichment call is worth another debit.
HTTP success is not a registration. A missing or unauthorized key, a validation error, or a transport failure shows up as HTTP 4xx/5xx. The gateway envelope is {code, message, data}; a code other than 200 is the same rule — there is no channel verdict in that envelope. Do not write false so the CSV looks complete. HTTP 2xx with gateway code 200 means the check was delivered. Only then read isExist, exists, registered, or a per-row exist. A delivered null is still a business result: undetermined, not a crash. Queue it. An unwrapped body with no such key is empty or malformed, not a list of nos.
Enrichment is a later debit. WhatsApp and Telegram mint a userHash on a positive cheap check; it is valid 24 hours and it authorizes avatar, About, or picture calls. Cache that hash only for the enrichment that follows immediately. Do not fetch a profile to "see if they exist." If the campaign is WhatsApp-only, a true on WhatsApp is enough to route — stop. Signal, Viber, and Snapchat each have their own cheap check; none of them is a proxy for the others. Per-channel field names and privacy quirks live in the WhatsApp lookup, Telegram checker, Signal, Viber, and Snapchat guides. Choosing between the first two is a separate article: WhatsApp vs Telegram number lookup. File-scale WhatsApp resume is in bulk verification.
Keep a suppress table of terminal false results with a timestamp. Re-check on a schedule (monthly for an active list is a starting point), not before every send, and not on every crash.
Normalize and dedupe on one pass
Deduping on the raw cell is how Leila billed the same person three times. One pass has to emit a single canonical key and a list of source rows that collapsed into it. Use digits with the country code, no spaces or punctuation (E.164 without the +). Lookup operations tolerate a leading +; WhatsApp also accepts spaces, dashes, and a 00 prefix; Snapchat, Signal, and Viber normalize digits for you. That leniency is for the wire, not for your index. Guessing a missing country code produces a confident miss on a number you never actually had.
Reject anything you cannot parse. N/A, blanks, extensions, and local numbers without a country are not "maybe later" — they are not this job. Keep the source row ids next to the canonical form so a later true/false can fan back out to every CRM record that pointed at it.
import json
import re
from collections import defaultdict
IN_JSONL = "leads.jsonl" # each line: {"row_id": "...", "phone": "..."}
OUT_MAP = "canonical.jsonl"
def canonical(raw):
text = str(raw).strip()
digits = re.sub(r"\D", "", text)
if text.startswith("00") and len(digits) > 8:
digits = digits.lstrip("0") or digits
if not (8 <= len(digits) <= 15):
return None
return digits
# One pass: drop junk, collapse aliases, remember every source row.
groups = defaultdict(list)
for line in open(IN_JSONL, encoding="utf-8"):
rec = json.loads(line)
key = canonical(rec.get("phone", ""))
if key is None:
continue
groups[key].append(rec["row_id"])
with open(OUT_MAP, "w", encoding="utf-8") as fh:
for number, row_ids in sorted(groups.items()):
fh.write(json.dumps({"number": number, "row_ids": row_ids}) + "\n")
print("unique", len(groups), "from", sum(len(v) for v in groups.values()), "parseable rows")
The numbers that already appear in this guide — +1 555 123 4567, 0015551234567, 15551234567 — collapse to one key. not a number is dropped. Do not pad a short leftover chunk with dummy rows so a batch "looks full"; dummy misses resume as finished work.
Cheapest channel check first
Cheap means two things at once: the existence operation, not the profile operation; and the channel you will actually message, not a tour of every project on the key. One authorized platform key and one credit balance cover WhatsApp, Telegram, Signal, Viber, and Snapchat. That is not a reason to call all of them. Start with the market the campaign uses. In Leila's queue that is WhatsApp. batch-check-10 takes up to ten full international numbers as accounts. One to ten is valid; eleven is a bad request. Each delivered row reports exist and isBusiness. isBusiness is a segmentation hint, not the registration answer, and it does not mint a userHash. For a live form with one person waiting, use check-exist-by-phone instead — holding nine extra numbers so you can "use the batch" is latency you paid for nothing.
If the campaign needs a second channel, fan out only the numbers that are still worth it: WhatsApp unknown after retries, or WhatsApp false when the desk actually sends on Telegram or Viber. Telegram check-exists takes peer as +phone. Signal check and Viber check take number. Snapchat check-phones-batch takes up to 50 comma-separated numbers. Do not bulk-probe Snapchat emails; that path uses account recovery and a hit can send a real recovery mail to the owner.
Checkpoint after every successful batch. Store the requested numbers next to the raw payload, then flush(). On restart, skip numbers that already have a terminal true or false inside your TTL. Retry null, missing rows, short result lists, and any call that never produced a business body. Flattening fields is a second pass after you have inspected a real envelope; index alignment is not a contract.
import json
import os
import time
import requests
ROOT = "https://api.opendata-api.com/api/project"
KEY = os.environ["OPENDATA_API_KEY"].strip()
STATE = "channel_state.jsonl"
# Documented request shape only — running it spends credits.
# GET {ROOT}/whatsapp/batch-check-10?key=YOUR_API_KEY&accounts=15551234567
def unwrap(body):
while isinstance(body, dict) and "code" in body and isinstance(body.get("data"), (dict, list)):
if body.get("code") not in (200, "200"):
return None
body = body["data"]
return body
def load_done(path):
done = {}
if not os.path.exists(path):
return done
for line in open(path, encoding="utf-8"):
rec = json.loads(line)
for n in rec.get("requested", []):
done.setdefault(n, rec)
return done
def whatsapp_batch(numbers):
resp = requests.get(
f"{ROOT}/whatsapp/batch-check-10",
params={"key": KEY, "accounts": ",".join(numbers)},
timeout=30,
)
if resp.status_code >= 400:
return None # no verdict — do not write false
return unwrap(resp.json())
done = load_done(STATE)
todo = [n for n in open("canonical.jsonl", encoding="utf-8") for n in [json.loads(n)["number"]] if n not in done]
chunks = [todo[i:i + 10] for i in range(0, len(todo), 10)]
with open(STATE, "a", encoding="utf-8") as fh:
for group in chunks:
payload = whatsapp_batch(group)
if payload is None:
time.sleep(2)
continue # leave the group out of done; next run retries
fh.write(json.dumps({"requested": group, "channel": "whatsapp", "raw": payload}) + "\n")
fh.flush()
Keep concurrency modest. A thread pool that fires Telegram, Signal, and Viber for every number, in parallel, on the first pass, is how a 2,000-row file becomes 8,000 debits before you notice the WhatsApp column was enough.
Score, route, and the mistakes that re-bill a number
Scoring is a fold over what you already stored, not another round of GETs. A number with at least one delivered true is reachable on those channels — route it in your market order and suppress the rest for this campaign. A number whose only non-miss is null is retry, not unreachable. A number with delivered false on every channel you actually use is unreachable for this cycle; keep it in the suppress table so next week's import does not pay again. A paying customer with no registration anywhere is a typo candidate: flag it for a human, do not auto-enrich it. Empty profile fields are privacy choices, not quality penalties, and they are irrelevant until a cheap check has already returned true.
Registered is not consented. A positive existence check means the number has an account on that platform. It does not mean you may message it. Honor opt-outs, keep a lawful basis, and follow each platform's rules. This is operational guidance, not legal advice.
The mistakes that re-bill a number are mechanical, and they survive every pretty dashboard:
- Calling the second channel, or a profile endpoint, before the list has been canonicalized. Aliases of one E.164 look like new work.
- Writing
falseon HTTP 4xx/5xx, a gatewaycodeother than 200, a missing field, or a short batch. Those are "no verdict." Resume them. A parser that invents a miss will skip the number forever and still look green. - Treating
nullas no. Undetermined needs a backoff and another cheap check, not a suppress. - Starting over after a crash because the checkpoint stored a pretty boolean instead of the requested list plus the raw body. You pay the in-flight batch again, and you cannot see which inputs the server never answered.
- Padding a leftover chunk, or spraying Snapchat email probes across a list. Dummy rows become fake finishes. Email recovery probes can mail the owner.
- Re-verifying the suppress table on every send, or skipping consent because the check was cheap. Freshness is a schedule. Permission is a different system.
Kill the process on purpose once while you still have a small file. If the next run reprints the same numbers, the checkpoint is theatre and the bill will say so.