Crypto Payment Gateway Architecture: How We Built One From Scratch

A crypto payment gateway accepts cryptocurrency for an order: it issues an invoice, watches the chain, matches the amount, then tells the shop with a callback. Peiko built a watch-only one — eight services, five chains, eleven watcher processes, three confirmations by default — with no spending keys in the watching plane.

Read in order the first time. Early sections set the product story. Invoice and architecture are the heart of the build. Incidents and the pending runbook are for when something is already on fire.

Written for engineering leads who are building a gateway or reviewing a build. Same topic as our notes on Web3 ecommerce payments and Node.js backends in fintech, but this one is the machine we actually run.

Key Takeaways

A crypto payment gateway is a system that creates payment invoices, monitors blockchain transactions, verifies that the received transaction matches the invoice, and notifies the merchant when the payment reaches the required confirmation state.

• A watch-only gateway can monitor deposits without holding private keys in the payment-processing plane, reducing the blast radius of a compromised API or monitoring service.
• A production gateway needs more than a checkout UI: it requires invoice management, blockchain watchers, transaction matching, confirmation tracking, callbacks/webhooks, accounting, monitoring, and operational tooling.
• Each blockchain and token network should be treated as a separate integration, because address generation, transaction detection, confirmation rules, fee models, token standards, and reorganization behavior can differ.
• The payment lifecycle should be idempotent: receiving the same blockchain transaction or webhook more than once must not create duplicate payments, commissions, or order fulfillment.
• A payment should not be treated as final merely because a transaction was detected. The gateway should define an explicit confirmation policy and distinguish detected, partially confirmed, confirmed, expired, and rejected states.
• Custodial and watch-only/non-custodial architectures have fundamentally different security, compliance, withdrawal, and operational requirements.
• Production readiness depends as much on observability, queue isolation, retry handling, key management, reconciliation, and incident response as on the blockchain integration itself.
• The right architecture depends on supported networks, custody model, settlement requirements, transaction volume, compliance obligations, merchant integrations, and whether the gateway supports only inbound payments or also payouts.

Who this article is for

This article focuses on the architecture and engineering of a watch-only crypto payment gateway rather than a hosted checkout product. It is most relevant to teams deciding what to build internally, reviewing an existing payment-processing architecture, or defining the technical scope for a custom gateway.

What is a crypto payment gateway

In plain language, a crypto payment gateway is a closed loop: the store hits your API, you return an address and an exact payment_amount, the buyer pays from their wallet, you watch the chain, and you fire a callback when the transfer matches. Unlike cards, a TRC-20 transfer does not reverse with a bank button.

In production, however, the gateway is more than a blockchain listener. It is a payment-processing system that maintains the relationship between a merchant order, an invoice, an on-chain transaction, a confirmation policy, and the final settlement state.

A useful abstraction is:

Merchant order → Invoice → Blockchain address/payment → Transaction detection → Transaction matching → Confirmations → Payment state → Merchant callback → Settlement/reconciliation

The blockchain provides the transaction record; the gateway provides the business logic that determines whether that transaction satisfies a particular payment request.

A crypto payment gateway — crypto processing — is the loop: store hits the API, we return an address and an exact payment_amount, the buyer pays from their wallet, we watch the chain, we fire a callback. Unlike cards, a TRC-20 transfer does not reverse with a bank button. Three Bitcoin confirmations are tens of minutes; the same integer on Solana is a different bet.

How a Crypto Payment Gateway Payment Lifecycle Works

A production crypto payment gateway should model payment processing as a stateful lifecycle rather than a single blockchain transaction.

A typical flow looks like this:

1. Order created
The merchant creates an order and requests a crypto payment.

2. Invoice created
The gateway creates an invoice with an external_id, payment currency, network, exact payment_amount, address and expiration time.

3. Payment address assigned
The gateway associates the invoice with a blockchain address or another deterministic payment-identification mechanism.

4. Customer pays
The customer sends the required asset from their wallet. Network fees are normally paid separately by the sender.

5. Transaction detected
A blockchain watcher detects the transaction and passes the event into the processing pipeline.

6. Transaction matched
The gateway verifies the destination address, asset, amount, network, transaction ID and invoice association.

7. Confirmations tracked
The transaction moves through the gateway’s confirmation states according to the network-specific policy.

8. Invoice completed
When the required business confirmation threshold is reached, the invoice is marked as paid.

9. Merchant notified
A signed or authenticated callback/webhook informs the merchant system.

10. Reconciliation and settlement
The transaction becomes part of the merchant’s accounting and settlement records.

This separation is important because blockchain transaction state and business payment state are not necessarily the same thing.

Watch-only vs custodial

We chose watch-only so a leaked store API key is an address-map and callback problem, not a drained hot wallet.

The custody model should be decided before designing the rest of the architecture because it changes wallet infrastructure, security controls, withdrawal workflows, compliance responsibilities, and incident-response requirements.

The merchant keeps spending keys. We see deposits; we never sign withdrawals inside the watching plane. Custodial gateways hold those keys and can auto-payout — faster UX, larger blast radius.

We went watch-only so a leaked API key is an address-map and callback problem, not stolen funds.

Bitcoin: merchant gives xpub / ypub / zpub (BIP44/49/84), we derive receive, skip index 0, ignore change. Other chains: a watched list or a deposit pool.

We also run a wallet dashboard on the same MySQL and the same watcher. Merchant flow is invoice → webhook. Dashboard flow is history, tickets, withdrawals we do not sign. Mix them once — we did, on a “top up wallet” shortcut — and external_id vanishes.

How to accept crypto payments on a website

This is the path a shop follows without the eight-service diagram first. API key on the server, never in React. Create an invoice, show the exact amount, let the buyer pay gas, verify a signed callback, ship only on status 2.

1. Issue a store API key (2FA on before production).
2. POST /api/v2/address with external_id, fiat or crypto amount, payment_currency, chain if the coin exists on more than one network.
3. Persist address, payment_amount, external_id, chain, expiry. Show that amount in the QR. No rounding.
4. Buyer pays from their wallet. They pay network gas. You do not.
5. Receive callback status 0 → 1 → 2. HMAC or a header secret, then match addr / value / external_id to your copy.
6. Ship the goods only on status 2. Ignore a repeat txid.

Which network to turn on first

USDT TRC-20, USDT ERC-20, and Bitcoin are not three rows in one dropdown. They are three scanners, three gas models, and three ways to issue an address. Network gas is paid by the sender; gateway commission is a separate tariff.

USDT TRC-20, USDT ERC-20 and Bitcoin are not three rows in one dropdown. They are three scanners, three gas models, three ways to issue an address.

If you take one chain live first, take USDT TRC-20 for checkout volume, Bitcoin if the audience actually holds BTC. ERC-20 USDT is a different product, not an alias.

Network gas ≠ gateway commission. The sender pays gas to the chain. Our fee is a tariff on the service balance — that is the 401 when coins are still on the address.

What Changes When You Add Another Blockchain?

A new network should therefore be treated as an engineering integration, not merely as another supported currency in the database.

Crypto Payment Gateway Architecture: Core Components

A production crypto payment gateway typically consists of several logical components. They can be implemented as separate services or combined into a modular monolith depending on transaction volume, team size and operational requirements.

The core components are:

Merchant API — creates invoices, retrieves payment status and manages merchant configuration.

Invoice service — maintains invoice lifecycle, expiration, payment amount and merchant order references.

Blockchain watchers — monitor supported networks and detect relevant transactions.

Transaction matcher — determines whether an observed transaction belongs to an invoice and whether its asset, network, destination and amount are valid.

Confirmation engine — tracks confirmation depth and applies network-specific finality rules.

Callback/webhook service — notifies merchants about payment-state changes with authentication, retries and idempotency.

Accounting/reconciliation layer — maintains balances, fees, commissions, settlement records and exception cases.

Admin and operations console — provides visibility into invoices, transactions, failed callbacks, pending payments and operational incidents.

Observability layer — collects logs, metrics, traces and alerts across the asynchronous payment pipeline.

The important architectural principle is that the API should not be responsible for directly scanning blockchains or deciding payment finality synchronously. Those concerns belong in dedicated processing components.

Build vs buy

Buy when you need hosted checkout this quarter. Build when the merchant must keep the keys, you need amount tails or an xpub watcher, or the callback must match your external_id. Cost is eight services and on-call on ingest — not a plugin license.

We have done both shapes. This write-up is the second.

When Should You Build a Crypto Payment Gateway?

Building a gateway makes sense when payment processing is part of the product’s competitive advantage, when the business requires unusual transaction logic, when merchants need control over custody or settlement, or when the product needs blockchain/network support that hosted providers do not offer.

Buying or integrating an existing provider is usually more appropriate when the primary goal is simply to accept a limited set of cryptocurrencies without owning the payment infrastructure.

The decision should account for more than development cost. Consider:

• provider transaction fees;
• supported assets and networks;
• custody model;
• geographic availability;
• compliance requirements;
• settlement options;
• API/webhook flexibility;
• uptime and operational responsibilities;
• reconciliation;
• refunds and exception handling;
• vendor lock-in.

How a crypto payment gateway invoice works

This is the heart of the money-in path. Issue an invoice with a unique external_id, freeze a rate, publish new_address so the watcher starts caring, then match on-chain by address and — on shared addresses — by an amount tail. Only the orchestrator marks the invoice paid.

Merchant binds an xpub. A tiny Node service (batch of 50 children, skip 0) answers whether the sample address belongs to that tree. We almost put that check in PHP. One BIP path off and you attach someone else’s tree. The generator stays the only HD source, and it stays off the public internet — an xpub is a map of future addresses.

Invoice comes in with external_id unique per store. Next HD child. Fiat amounts use a rate we refresh every 10 minutes; freeze it for the invoice or the buyer will screenshot an exchange and open a ticket. If the address already has a pending bill, we add a unique amount tail. Then new_address into RabbitMQ crypto. Until the watcher inserts the row, the chain transfer does not exist for us.

Watcher: ZMQ rawtx on Bitcoin, JSON-RPC transfer(address,uint256) on ETH/BSC, socket subscriptions on Solana, block batches on Tron. Event goes to from_python. Orchestrator matches address, time window, amount. Percent tolerance only when there is a single pending invoice. Several pending → exact amount plus tail.

EVM, Solana and Tron skip HD. Round-robin store addresses or a free pool slot. BNB sometimes needs a memo. The watcher logs every ERC-20 by selector, not a USDT allowlist. After the dashboard filled with junk tokens, coin filtering moved into the orchestrator.

A worked pass for order 4821, $100 USD as USDT TRC-20. Two pending bills already sat on the store address, so the second one got a tail:

POST /api/v2/address
Authorization: Bearer <store_api_key>

{
  "external_id": "4821",
  "origin_amount": 100,
  "origin_currency": "usd",
  "payment_currency": "usdt",
  "chain": "trx"
}

{
  "address": "TQn9Y2khEsLJW1ChVWFMSMeRDow5KcbLSE",
  "payment_amount": "100.01",
  "payment_currency": "usdt"
}

That 100.01 is the extra payment amount loop in DetailsPayment: while a pending invoice already owns that address + amount + coin, add extra_payment_amount (for stables, one cent). Then we publish to RabbitMQ queue crypto:

content_type: new_address
{"address": "TQn9Y2khEsLJW1ChVWFMSMeRDow5KcbLSE", "chain": "TRX"}

Until the watcher consumes that, an on-chain transfer is invisible to us. After confirmations the callback looks like this (status: 0 unconfirmed, 1 partial, 2 confirmed):

{
  "addr": "TQn9Y2khEsLJW1ChVWFMSMeRDow5KcbLSE",
  "status": 2,
  "txid": "0c3e…",
  "value": "100.01",
  "external_id": "4821",
  "memo": null
}

GET or POST depends on the store flag is_post_callback. We still have shops that round the QR to 100 and then wait forever.

Bitcoin is the other model. New invoice, new HD child. No tail, because uniqueness is the address.

Merchant binds a zpub (BIP84, bc1…, path m/84'/0'/0'/0/n). We skip index 0 so it does not collide with the sample they pasted. Change (path 1) is unused. Node generator, batch 50, is the only place that derivation happens. Then:

POST /api/v2/address
{
  "external_id": "4821",
  "origin_amount": 100,
  "origin_currency": "usd",
  "payment_currency": "btc"
}

{
  "address": "bc1q…n",
  "payment_amount": "0.00115200",
  "payment_currency": "btc"
}

Watcher side is ZMQ rawtx, not JSON-RPC logs. chain on the broker message is BITCOIN. Freeze the fiat rate for the invoice lifetime or the buyer screenshots an exchange ten minutes later and opens a ticket — rates refresh on a 10-minute cron.

Invoice Data Model

The gateway should maintain one authoritative invoice record rather than allowing different services to independently calculate whether an order is paid. This is particularly important when blockchain events, callbacks and retries are asynchronous.

Why a crypto invoice stays pending

Two clocks run at once: invoice pending → completed or expired, and on-chain confirmations as callback status 0 / 1 / 2. A transfer into an expired invoice is a ticket, not auto-complete.

The invoice is pending → completed or expired (0 / 1 / 2 in our table). The on-chain tx is unconfirmed → partially_confirmed → confirmed, which is what the callback sends as status 0 / 1 / 2. Those clocks are not the same.

Expiry is a job every 6 minutes. Buyer can still push a transfer after we flipped the bill to expired. Coins are on-chain. Invoice will not auto-complete. That is a ticket, not a second completed.

We mark completed on product policy, usually when the tx hits confirmed (default 3). Repeating the same txid must update confirmations. It must not fire PaymentTransactionCreated again.

Important: a blockchain transaction being visible does not automatically mean that the merchant order should be fulfilled. Payment detection, transaction validation and payment finalization are separate stages.

Crypto payment gateway architecture we run

Eight services: two Next.js apps (merchant and admin, separate cookies), Laravel orchestrator (~139 MySQL migrations), a second Laravel for the public API with no migrations of its own, Django watcher (11 containers, five scanners), Node generator, WordPress, RabbitMQ, a host Telegram bot.

You can put 12 workers on the API. Ingest of from_python is still one process. from_python (the RabbitMQ routing key the Django watcher publishes detected txs onto — JSON like new_transaction / update_transaction — which a single Laravel ingest process consumes). We learned that the loud way.

Laravel’s scheduler in this repo is often empty. Cron lives in the image: rates, invoice expiry every 6 minutes, pool TTL every minute, unused BTC prune every 30. Default confirmations: 3, for every chain. Amounts: decimal(36, 18) the whole way through PHP, Python and JSON. Floats clip USDT.

The implementation can be understood as five logical planes:

1. Merchant plane — merchant dashboard and API.
2. Payment plane — invoices, payment state and callbacks.
3. Blockchain observation plane — network-specific watchers and transaction ingestion.
4. Accounting/operations plane — balances, commissions, reconciliation and administration.
5. Infrastructure plane — queues, schedulers, monitoring and deployment.

This separation prevents the user-facing API from becoming tightly coupled to blockchain scanning and asynchronous transaction processing.

Crypto Payment Gateway Security Architecture

A crypto payment gateway should be designed around the assumption that blockchain transactions, webhooks, API credentials and infrastructure can all be attacked or fail independently.

The most important security boundaries include:

API credentials
Store API keys server-side, support rotation and never expose merchant secrets in browser applications.

Webhook authentication
Use HMAC signatures or another authenticated mechanism and validate the payload before updating payment state.

Idempotency
Repeated blockchain events and webhook deliveries must not create duplicate payments, commissions or fulfillment events.

Private-key isolation
In a watch-only architecture, private keys should remain outside the monitoring and payment-processing plane.

Queue isolation
Production and sandbox events must not share infrastructure in a way that allows test events to reach production processing.

Access control
Administrative actions such as withdrawal approval, callback changes and key management should require appropriate authorization and audit logging.

Reconciliation
Internal payment records should periodically be reconciled against blockchain data and merchant records.

Monitoring
Alerting should cover transaction-ingestion failures, queue depth, callback failures, stale invoices, node/provider errors and unexpected balance changes.

How much it costs to build a crypto payment gateway

The cost is those eight services, not a checkout plugin. What moves the number is how many chains you watch, whether ingest and prune are real, and whether merchant and admin stay separate products.

The cost to develop a crypto payment gateway is the eight services, not a checkout plugin. We shipped two Next.js dashboards (merchant and admin, separate cookies), a Laravel orchestrator with about 139 MySQL migrations, a public API with no schema of its own, a Django watcher — 11 processes across 5 chains — a Node HD generator, WordPress, RabbitMQ, a host bot. That list is the bill.

What moves it. Chains: Bitcoin is an HD tree and a ZMQ scanner; USDT TRC-20 is a shared address plus an amount tail; the same ticker on ERC-20 is a third scanner. Five chains is five of those jobs, not five rows in a dropdown. Watcher processes: eleven containers, and ingest of from_python is still one process — that one going down is an outage even when the dashboard is green. Two dashboards: merchant flow is invoice → webhook; admin is tickets and withdrawals we do not sign. Mixing them once cost us external_id.

Adding a chain looks cheaper than keeping five alive. It is not, if you skip the eight touchpoints in How to add a network to a crypto payment gateway. A coin in the admin list with no scanner is invoices that never close — we have reviewed that miss. The quiet cost is ingest alerts, prune that actually publishes del_address, pool TTL, rates every 10 minutes, callbacks on their own queue. A one-chain MVP and this five-chain build are different products.

Crypto Payment Gateway Compliance: What Changes by Business Model

Crypto payment gateway compliance depends on the gateway’s business model, jurisdictions, supported assets, custody model and settlement activities.

A gateway that only provides software for a merchant to monitor payments is materially different from a provider that takes custody of customer funds, converts crypto to fiat, executes payouts, or provides payment services to third parties.

Before development starts, define:

• where the company operates;
• which countries merchants and customers can access;
• whether the gateway is custodial or watch-only;
• whether the platform converts crypto to fiat;
• whether it executes payouts;
• which assets and networks are supported;
• whether KYC/KYB is required;
• what transaction monitoring is required;
• how sanctions screening is handled;
• what records must be retained;
• which regulated partners are involved.

Important: technical architecture cannot make a regulated activity compliant by itself. The legal structure, custody model, jurisdictions, asset types and payment flows should be reviewed with qualified legal and compliance professionals before the production architecture is finalized.

How fees, rates and withdrawals work

Four pieces get mixed into “the balance”: deposit pools, rates, commission tariff, and withdrawal tickets. Withdrawals are moderated — we do not sign; see the withdrawals article for create without debit and approve that moves fiat.

Four operational pieces people mix into “the balance”.

Deposit pool. For non-Bitcoin, addresses sit free → busy with expired_at. A monitor runs every minute. There is a 15-minute buffer before we treat a slot as reusable; Solana’s default window in settings is about 30 minutes. An expired TTL means the address is no longer unique for the current bill. If the pool is empty, the API throws “free address not found” — that is an ops problem, not a chain outage.

Rates. Fiat → crypto every 10 minutes from a pivot table. Stables get round() to a whole unit unless a tail is applied, which is why a USDT QR can show 100 or 100.01. Pass payment_amount yourself if you already converted. Otherwise freeze our rate for the invoice window.

Commission. Listener on PaymentTransactionCreated. Percent or flat, in USD tariff units, consumeBalance on the user, a second row with type = commission. Gas is paid by the sender. Repeat the same incoming txid and you should not insert a second commission. That listener is why a negative service balance returns 401 while 100 USDT still sits on the address.

Withdrawal. Watch-only: we do not sign. Ticket statuses moderation → in-progress → success or declined. An operator moves funds outside this plane. If you need auto-payout, that is a different product with a hot wallet we deliberately did not put next to the watcher.

What broke in our crypto payment gateway

The most useful architecture lessons often come from failure modes rather than diagrams. These incidents shaped the production design and are worth considering before implementing a gateway from scratch.

Incidents we still talk about: ingest missing while the dashboard looked fine, callbacks inside the ingest loop, two Laravels drifting, stub prune, sandbox on the live queue, one balance widget hiding a negative fee, rounding, same confirmations everywhere, amount-tail races.

These are the ones we still talk about internally.

Dashboard was green. Payments were not. Ingest was missing from supervisor. Support saw history in the UI from an old sync and thought the chain was fine. First question on “I don’t see the money” is no longer “is the site up”. It is “is ingest running, and how deep is from_python”. We alert on both.

Callback lived inside the ingest loop. Synchronous HTTP to the merchant. One shop timed out, the whole confirmation feed stalled, coins sat on-chain, invoices stayed open. Callbacks belong on their own queue with retries. We still find the old pattern in similar reviews.

Two Laravels drifted. Only the orchestrator has migrations. POST callbacks, amount tolerance and stablecoins landed there first. API kept writing the old invoice shape. Deploy the API before the migration and create-invoice dies. We now ship the schema change and the twin models in one release. The grown-up fix is a shared domain package. We have not extracted it yet. Until we do, every CVE patch is twice the work.

Prune was a stub. The job returned immediately. Solana subscriptions piled up and hit connection limits while the product still looked healthy. Prune has to publish del_address and actually run.

Sandbox closed live invoices. A “test payment” published new_address onto the production crypto queue. Staging must never touch that queue.

Support: API 401, 100 USDT TRC-20 sitting on the address. Three numbers. On-chain. Store ledger (can lag a sync). Service fee balance in tariff units — that one going negative is what returns 401. The coins did not vanish. We split the widgets. One “total” hid the cause for months.

Rounding. Merchant UI rounded USDT to whole units. Invoice pending forever, money already on the address. The other way: a few extra cents, tolerance on, one pending bill — we closed the wrong invoice. Checkout copy is one line: send the amount from the response, no rounding.

Same 3 confirmations on BTC, EVM and Solana. Convenient. Wrong in a merchant contract, especially on large cheques. Policy has to be per chain.

Race on the tail. Two invoices, one EVM address, no row lock, same extra amount. Match is exact tails; without a unique index on an open amount you collide.

A few more we treat as debt, not theory: API keys are 40-character plaintext; the generator we inherited has no auth; migrate on Docker build can shift the database when you publish an image. Generator stays on the internal network. We do not call the public stubs that always answer “valid”.

And we do not mark an invoice paid inside the scanner. Scanner says: address appeared, tx is new, confirmations grew. Orchestrator matches. Otherwise tails and fees invent a second ledger.

Adding a chain is eight touchpoints — catalog, handler, address issue, chain code, scanner, event JSON, dashboard, vectors. Skip one and you get a coin in the dropdown that never closes. That list is the next section.

How to add a network to a crypto payment gateway

Adding a chain is eight touchpoints — catalog, handler, address issue, chain code, scanner, event JSON, dashboard, vectors. USDT on ERC-20, BEP-20, and TRC-20 are three of those lists, not a dropdown alias.

If you want to know how to build a crypto payment gateway that can grow, adding a chain is the test of whether we keep service boundaries. The miss we have reviewed: currency in the admin list, no scanner, invoices pending forever.

1. Currency catalog in the orchestrator: iso_name, decimals, parent for a token, contract, stablecoin flag, fiat rate row.
2. Handler factory: parse the tx, memo fields, amount as decimal — not float.
3. Address issue in the public API: HD, store address, or pool. Pick it on purpose.
4. new_address with a chain string the watcher understands one-to-one (BITCOIN, TRX, …).
5. Scanner process in compose, block cursor, idempotent writes.
6. Same JSON on from_python the ingest command already expects (transaction, to_addr, value, input_msg for tokens).
7. Dashboard icon, store form, admin toggle.
8. Vector table: fixed txids, amounts, a reorg. Without it the chain “kind of works” on staging.

USDT ERC-20, BEP-20 and TRC-20 are three of these lists, not a dropdown alias.

How to Add a Network to a Crypto Payment Gateway

Adding a new blockchain should be treated as a complete integration with its own detection, confirmation and operational requirements.

At minimum, define:

1. Supported asset and token standard
2. Address-generation method
3. Transaction detection mechanism
4. Required RPC/indexer infrastructure
5. Confirmation/finality policy
6. Network fee model
7. Amount precision
8. Reorganization handling
9. Transaction lookup strategy
10. Monitoring and alerting
11. Testnet/sandbox strategy
12. Reconciliation procedure

A network should not be considered production-ready merely because the gateway can generate an address and detect one successful transaction.

How to connect a crypto payment gateway API

The contract is POST /api/v2/address behind the store API key. Persist the response, show the exact amount, verify a signed callback, close only on status 2. Sandbox must never publish new_address to the live crypto queue.

The contract we actually ship is POST /api/v2/address behind the store API key (api_key + api_closed middleware). v1 “just give me an address” should 410. Key stays on your server — not in React.

Recommended API Flow

Merchant Backend
      ↓
POST /api/v2/address
      ↓
Create Invoice
      ↓
Return Address + Exact Payment Amount
      ↓
Customer Pays
      ↓
Blockchain Watcher
      ↓
Transaction Matcher
      ↓
Confirmation Engine
      ↓
Signed Callback
      ↓
Merchant Backend
      ↓
Order Fulfilled

The merchant should not infer payment completion by polling the blockchain independently if the gateway already provides an authenticated payment-state callback. The gateway remains responsible for translating blockchain events into the merchant-facing payment state.

Persist what you got back: address, payment_amount, external_id, chain, expiry. Show that amount in the QR. A handler that does not invent a second source of truth looks like this:

// callback URL — HMAC or a header secret first
$saved = Order::where('external_id', $payload['external_id'])->firstOrFail();

if ($payload['txid'] && $saved->txid === $payload['txid']) {
    return; // already processed
}

$ok = hash_equals($saved->address, $payload['addr'])
    && $saved->payment_amount === $payload['value']
    && (int) $payload['status'] === 2;

if ($ok) {
    $saved->markPaid($payload['txid']);
}

Same idea in a Node shop: compare strings, do not parse USDT as Number. decimal(36, 18) on our side dies the moment your JSON stack turns 100.01 into 100.009999.

Rotate the key. Only the store owner changes the callback URL — a stolen key plus a swapped URL is someone else’s “paid” events. Sandbox must not publish new_address to the live crypto queue; we burned that once.

Why a crypto payment is pending

On-call order we actually use. Dashboard HTTP 200 is check zero, not the finish line. Start with ingest and from_python depth, then scanners, watchlist, amount/chain, expiry, three balances, callbacks, and sandbox contamination.

1. Ingest command in supervisor? How deep is from_python?
2. Which of the five chain containers died?
3. Was new_address published? Is there a row in the watcher Address table?
4. On-chain amount vs payment_amount. Wrong chain, rounded QR, missing BNB memo.
5. Did the 6-minute expiry job already close the invoice?
6. Which of the three balances are they looking at? Negative service fee → 401 by design.
7. Callback log_line on the payment tx? GET vs POST, merchant timeout, no HMAC.
8. Did a sandbox payment hit the live crypto queue?

If 1–3 are healthy and 4 mismatches: do not auto-complete. Coins moved. That is a ticket.

Observability and Reliability for Crypto Payment Gateways

Blockchain payment processing is asynchronous and distributed, so an HTTP health check alone cannot prove that the gateway is working.

A production monitoring strategy should track at least:

• blockchain watcher health;
• RPC/node availability;
• transaction ingestion latency;
• queue depth;
• invoice age;
• confirmation latency;
• callback success/failure rates;
• duplicate transaction attempts;
• reconciliation discrepancies;
• expired invoices;
• unexpected balance changes;
• address-pool exhaustion;
• failed scheduled jobs.

The key operational principle is simple:

Monitor the money flow, not only the application infrastructure.

A dashboard can return HTTP 200 while blockchain ingestion is completely broken.

After go-live

What still has to be true on a quiet Tuesday: ingest in supervisor, prune that publishes del_address, sandbox isolation, frozen rates, queued callbacks with HMAC, key rotation, three balance widgets, withdrawals as tickets.

The runbook above is for incidents. This is what still has to be true on a quiet Tuesday.

• Ingest is in supervisor. Alert on from_python depth, not only dashboard HTTP.
• Address prune actually publishes del_address.
• Sandbox cannot write to the live crypto queue.
• Invoice rate is frozen or the UI shows tolerance. QR shows payment_amount with no rounding.
• Callbacks are queued with retries, HMAC, idempotent on txid.
• API key rotation exists. Only the owner changes the callback URL.
• Three balances are three widgets. Commission is not gas.
• Withdrawal is a ticket. Nobody expects the watcher to sign.