Design a Payment Checkout System (Idempotent, Saga-Based)

The payment system is where idempotency, sagas, outboxes, and resilience handlers all show up at once. Get any of them wrong and customers get double-charged or stuck with paid-but-unshipped orders. This chapter designs a checkout flow in .NET that survives network glitches, webhook storms, and orchestrator crashes - and explains why every block in the series earns its keep here.

When does payment become its own service?

Three signals.

Multiple products feeding one checkout. A marketplace, multi-tenant SaaS, or e-commerce site with subscriptions and one-offs. Centralising payment logic stops three implementations from drifting in subtle, expensive ways.

Compliance requirements. PCI DSS scope shrinks dramatically if card data never touches your servers (use Stripe Elements / Adyen Web Drop-In). The payment service is the well-defined boundary.

Refund and reconciliation are first-class operations. Manual ops people need to refund, retry, and trace charges. A dedicated service with audit trails and admin UI makes that scalable.

What numbers should I budget for?

Checkouts / day             100K
Avg amount                  $50
Payment provider latency    300-1000 ms p99
Webhook duplication rate    1-5% (Stripe retries)
Idempotency window          24-72 hours
Refund window               90 days typical
Audit log retention         7 years (financial records)

The two numbers that drive everything: provider latency forces async UX (polling or webhook for confirmation), and webhook duplication makes idempotency mandatory. Audit retention shapes storage decisions.

What does the architecture look like?

flowchart LR
    Client --> Web[ASP.NET Core checkout]
    Web --> SagaOrch[Saga Orchestrator]
    SagaOrch -->|reserve| Inv[Inventory svc]
    SagaOrch -->|authorise| Pay[Payment svc]
    Pay --> Stripe[(Stripe API)]
    Stripe -. webhook .-> Web
    SagaOrch --> OB[(Outbox)]
    OB --> Notif[Notification svc]
    SagaOrch --> Audit[(Audit log)]
    Stripe -. async webhook .-> Web

Five services. Web accepts the checkout, the saga orchestrator runs the steps, payment talks to Stripe, inventory reserves stock, notification fans out emails. Audit log captures every state transition. Webhooks come back asynchronously and update the saga.

What is the .NET 10 wiring for the saga?

public class PaymentSaga : MassTransitStateMachine<PaymentSagaState>
{
    public State Reserving { get; private set; } = null!;
    public State Authorising { get; private set; } = null!;
    public State Capturing { get; private set; } = null!;
    public State Completed { get; private set; } = null!;
    public State Failed { get; private set; } = null!;

    public Event<CheckoutStarted> Started { get; private set; } = null!;
    public Event<InventoryReserved> Reserved { get; private set; } = null!;
    public Event<PaymentAuthorised> Authorised { get; private set; } = null!;
    public Event<PaymentCaptured> Captured { get; private set; } = null!;
    public Event<PaymentFailed> Failed { get; private set; } = null!;

    public PaymentSaga()
    {
        InstanceState(x => x.CurrentState);

        Initially(
            When(Started)
                .Then(ctx => { ctx.Saga.IdempotencyKey = ctx.Message.IdempotencyKey;
                                ctx.Saga.Amount = ctx.Message.Amount; })
                .Publish(ctx => new ReserveInventory(ctx.Saga.OrderId, ctx.Message.Items))
                .TransitionTo(Reserving));

        During(Reserving,
            When(Reserved).Publish(ctx => new AuthorisePayment(
                ctx.Saga.IdempotencyKey, ctx.Saga.Amount))
                .TransitionTo(Authorising));

        During(Authorising,
            When(Authorised).Publish(ctx => new CapturePayment(ctx.Saga.IdempotencyKey))
                .TransitionTo(Capturing),
            When(Failed).Publish(ctx => new ReleaseInventory(ctx.Saga.OrderId))
                .TransitionTo(this.Failed));

        During(Capturing,
            When(Captured).TransitionTo(Completed));
    }
}

The Stripe call inside the AuthorisePayment consumer:

public class AuthorisePaymentConsumer(IStripeClient stripe, AppDbContext db)
    : IConsumer<AuthorisePayment>
{
    public async Task Consume(ConsumeContext<AuthorisePayment> ctx)
    {
        var options = new PaymentIntentCreateOptions
        {
            Amount = (long)(ctx.Message.Amount * 100),
            Currency = "usd",
            ConfirmationMethod = "automatic",
        };
        var requestOptions = new RequestOptions
        {
            IdempotencyKey = ctx.Message.IdempotencyKey.ToString()  // Stripe-side dedup
        };
        var intent = await stripe.PaymentIntents.CreateAsync(options, requestOptions);
        if (intent.Status == "requires_action") { /* 3DS challenge */ }
        await ctx.Publish(new PaymentAuthorised(ctx.Message.IdempotencyKey, intent.Id));
    }
}

Three details. The same idempotency key flows from client to your service to Stripe, deduping at every layer. The saga orchestrator persists state on every transition. Webhook updates from Stripe publish into the same saga via correlation on the payment intent ID.

How do webhooks integrate with the saga?

app.MapPost("/webhooks/stripe", async (HttpRequest req, IPublishEndpoint bus,
                                        AppDbContext db, IConfiguration config) =>
{
    var json = await new StreamReader(req.Body).ReadToEndAsync();
    var sig = req.Headers["Stripe-Signature"].ToString();
    var stripeEvent = EventUtility.ConstructEvent(json, sig,
        config["Stripe:WebhookSecret"]);

    // Idempotency on Stripe event ID
    var seen = await db.WebhookEvents.AnyAsync(e => e.StripeEventId == stripeEvent.Id);
    if (seen) return Results.Ok();

    db.WebhookEvents.Add(new() { StripeEventId = stripeEvent.Id, ReceivedAt = DateTimeOffset.UtcNow });
    await db.SaveChangesAsync();

    if (stripeEvent.Type == "payment_intent.succeeded")
    {
        var intent = stripeEvent.Data.Object as PaymentIntent;
        await bus.Publish(new PaymentCaptured(Guid.Parse(intent!.Metadata["idempotencyKey"])));
    }
    return Results.Ok();
});

The signature check authenticates the webhook. The idempotency table on event ID dedupes Stripe's retries. The published message correlates back to the saga via the idempotency key.

What scale-out path does this support?

The shape stays mostly the same at scale. What changes:

• Saga state DB: partition by user ID; per-user sagas are serialisable on one partition.
• Webhook handler: stateless; scale replicas. Stripe limits the inbound rate by retry, so the load is bounded.
• Audit log: append-only, partition by month; archive after the retention window.
• Stripe rate limits: ~25 req/s per account; multi-account routing is the answer if you exceed.

The bottleneck at scale is rarely your code; it is the payment provider's API.

What failure modes does this introduce?

• Double charge - retry without idempotency. Mitigation: enforced at three layers (client UUID, server idempotency, Stripe idempotency).
• Charged but no shipment - capture succeeded, fulfilment failed. Mitigation: the saga publishes RefundCharge as compensation; a manual review queue catches edge cases.
• Webhook lost - Stripe sent an event your endpoint never received. Mitigation: poll payment_intents.list nightly for unreconciled charges and replay missing events.
• 3DS authentication abandonment - user starts checkout, never completes 3D Secure. Mitigation: timeout the saga after 30 min, release inventory, mark order as abandoned.

The observability chapter exposes saga-state metrics so all four are visible in dashboards.

When is a custom payment service overkill?

When you sell one product with one provider and have no reconciliation needs. A direct Stripe Checkout integration with a single webhook handler is sufficient. Build the dedicated service when complexity (multi-product, refunds, ops UI, multi-provider) justifies it.

Where should you go from here?

Last case study: analytics events pipeline

• the firehose problem, where every chapter's queue and stream patterns combine. After that the series wraps with how to answer interview questions and conclusion.