Ten Things That Always Break - ProductFTW #51

When I built Wallaby with my co-founder, Todd Zino, from 2012 to 2014, we made a list of things that always caused problems. I can’t find the original list, which feels apropos for a piece like this. (Item no. 1: You lose your checklist.)

Fintech software is a unique beast. You’re not just building apps; you’re building trust, handling money, and navigating a minefield of technical and regulatory complexity. Here are ten things (plus a few honorable mentions) that frequently go wrong in fintech development, even for the most seasoned teams.

I recommend running through this if you’re building something new or trying to troubleshoot. It’s like checking to ensure your computer is plugged in and your power-strip is turned on.

The 10 Things

1. Time Zone (or Time Zone Conversion)

Time is a lie. Especially in fintech. Whether you’re calculating interest, triggering cutoffs, or just showing a timestamp to a user, time zone issues can lead to misfires. Daylight saving? Regions that don’t observe it? Systems that assume local time? Chaos.

2. Firewalls, IP Tables, or Security Tools

You deployed your app, and everything broke, but it was only in production. Security layers like firewalls, rate limiters, or outbound connection restrictions often block key dependencies (like payment processors or fraud tools). These problems rarely appear in dev environments and are notoriously hard to debug.

3. Off-by-One Errors

The sneakiest of bugs. Did that loop include the last item? Did your date range capture the complete end of the day? These tiny mistakes often surface in pagination, balances, or transaction windows, and they can erode user trust quickly when money appears “missing.”

4. Currency Conversion Errors

Money math is brutal. Using the wrong rate, applying it in the wrong direction, or rounding incorrectly can lead to frustrating inconsistencies. Things get even trickier with zero-decimal currencies or multi-leg conversions (e.g., JPY → USD → BTC).

5. Pending Transactions Posting Early (or Late, or Never)

Banking systems don’t always behave the way users expect. A charge shows up as pending and then vanishes. Or it posts for a different amount. If your app doesn’t explain this nuance well or handles it inconsistently, it quickly breaks user trust.

6. Date/Time Formatting Errors

Is that 01/02/2024, January 2, or February 1? Date formatting across locales is a breeding ground for bugs. Add parsing issues from third-party APIs, and you’ve got a perfect storm for transactions getting timestamped — or rejected — incorrectly.

7. Race Conditions in Asynchronous Systems

In distributed systems, timing is everything. But weird things happen when two systems try to update the same record — or process the same event — simultaneously. Especially dangerous when dealing with webhooks, ledgers, or stateful workflows.

Please don't make your microchips compete.

8. Decimal vs Float Representation

Never use floats for money. Ever. Using floating-point math instead of fixed-point or decimal libraries can cause minor rounding errors that compound, and fractions of a cent matter in fintech. Always use the correct data types.

9. Inconsistent State Between Systems

One service thinks a transaction succeeded; another thinks it failed. This kind of data drift can stem from retries, partial failures, or lack of atomic operations — reconciling these mismatches is painful and error-prone.

10. Poor Handling of Retries or Idempotency^[1]

Retrying an API call without proper idempotency can lead to double charges or duplicate records. Conversely, not retrying when you should can result in dropped or incomplete actions. Fintech systems must be deliberate about handling retries safely.

Honorable Mentions

(But wait - there's more!)

“Just Use the Test Environment”...That Doesn’t Match Prod

Sandbox APIs often behave differently from production, causing development teams to miss edge cases entirely. Never assume parity without testing.

Compliance Reviews That Derail Timelines

Shipping a product is hard enough without late-stage rewrites from legal. Building in compliance review early (and often) is key.

Magical Retry Logic with No Observability

Retries are great, until they silently fail. Always log and trace background jobs and failure handling to avoid invisible data loss.

Too Much Trust in Third-Party APIs

Fintech is built on APIs, but brittle dependencies can break your product. Always have fallback logic, health checks, and alerts.

Shadow Data Drift

When your ledger and reporting don’t match, you'll have trouble figuring out where things broke. Keeping a single source of truth (and syncing regularly) is essential.

Peter learned about shadow drift the hard way.

Cron (Scheduled) Jobs that Don’t Make it on Time

Scheduled jobs are critical in most applications, running maintenance, updating information, and calculating values. These scheduled tasks run in the background and success (or errors) are often logged to logs that no one is looking at. Always document the schedule, scope, and success criteria. Generate loud alerts when they don’t work.

Audit Tables No One Looks At

Audit tables are designed to track when something breaks or fraud slips through. You have to look at them for them to be valuable! Make sure to review audit tables periodically and after each release. Otherwise, I can guarantee the logs will be broken when you need them.

Final Thoughts

Fintech teams face a unique blend of engineering complexity, regulatory pressure, and high user expectations. The problems on this list aren’t exotic; they’re common, recurring, and often subtle, and that’s precisely why they deserve attention.

Investing time in better observability, defensive coding, and operational discipline can prevent a world of pain down the line and make your product (and your team) far more resilient.

Did I miss your favorite fintech bug? Put it in the comments.

1. Not sure what the hell idempotency is? Developers love to use this word and PMs love to pretend like they get it. Idempotency means that performing the same action multiple times has the same effect as doing it once. In software, this is crucial for safely handling retries, especially in APIs. For example, if a user tries to make a payment and the network times out, your system might retry the request. If that request isn’t idempotent, you could accidentally charge the user twice. But if it is idempotent, the system recognizes the retry as a duplicate and avoids performing the action again. ↩︎

About ProductFTW

ProductFTW is a weekly newsletter about product management with a focus on real-life experiences in startups. We want to help product leaders be successful by giving realistic approaches that aren’t for giant tech companies. We know you don’t have a full-time product designer on each team. We know your software probably hasn’t been used by millions of people worldwide–yet. We’re here to bridge the content gap from building your product and team to scaling it.

Part of the Product Requirements Field Guide — ProductFTW's collected essays on the six phases of writing requirements, from problem definition to launch.