Lessons
Short notes on working better, learning faster, and staying effective over time.
Our on-call engineer muted the pager after 200 false alarms that month. Alarm 201 was the real outage.→
An alert should mean exactly one thing. A human needs to act right now. Everything else belongs in a dashboard or digest.
The library our auth system depended on was archived on GitHub two years ago. We found out during a compliance audit.→
Dependency health should be checked on a schedule, especially for anything security-sensitive. Nothing tells you a library went quiet.
I've read a hundred 'add login' PRs from side projects. They all break the same way.→
Auth isn't a login form, it's an identity and session lifecycle. Reset tokens, session invalidation, and rate limits included.
I used to leave 15 comments on every pull request. Half were about semicolons. My team started dreading my reviews.→
Automate style and formatting completely so human review time is reserved exclusively for logic, edge cases, and intent.
Our dashboard got slower the longer someone left it open. We assumed it was their laptop. It was every event listener we never cleaned up.→
Every listener, subscription, or interval needs a matching teardown tied to the same lifecycle event that created it.
Our CPU was at 12%. Our database was idle. Every request still hung for 30 seconds. We weren't out of resources. We were out of connections.→
Monitor connection pool checkout time and utilization as first-class metrics. CPU and DB load can look fine while the pool is starved.
Embrace Continuous Learning→
The tech world evolves fast. Keeping up with trends, frameworks, and languages is key to staying relevant.
Our error rate dropped to zero overnight. We hadn't fixed anything. We'd added a try/catch that swallowed every failure.→
Never catch an exception without deciding what happens to that information. A quiet catch block deletes the only signal something is broken.
I explained event-driven architecture to our CEO using a food delivery app. It landed faster than my last three whiteboard diagrams combined.→
Start with a system the listener already understands intuitively, introduce the benefit before the terminology, and only name the pattern last.
A feature flag stayed in the codebase two years after the feature fully shipped. Cleaning it up caused the outage.→
Give every feature flag an owner and an expiry date at creation, or it becomes permanent untested config.
We added an index to fix one slow query. Every write got 3x slower for a year before anyone connected the two.→
Before adding an index, check the table's read/write ratio and measure write latency, not just the query you were trying to fix.
We shipped an API key in our frontend bundle. It took nine months for someone to notice, and abuse it.→
The security boundary is "does this ever reach the browser," not "is it in an env file." Client-reachable code needs scoped, revocable keys or a backend proxy.
I restructured our backend's folders exactly once. Three years and 20 engineers later, I still haven't touched it again.→
Organize by domain/feature first and technical layer second. A folder should map to an owner, not a technical concern.
We asked AI to write a database migration. It passed every test. It would have silently dropped six months of customer data.→
AI tools are confident about the code they can see and blind to what they can't. Grep the whole org for a column name before merging a migration, not just the service being changed.
Two engineers argued for twenty minutes about a variable name. The unhandled null case three lines down shipped anyway.→
Review against whether a bug fails loudly, whether someone else can safely change it later, and whether it's easy to reason about, not naming taste.
A user signed up with an emoji in their name. Nothing broke for eight months. Then our export job started crashing every night at 3am.→
Use full UTF-8 end to end and test with emoji and non-ASCII strings as normal QA. Bad encoding can sit silently for months before it surfaces.