How We Use Flow PT CustomEntities to Improve Our Code Quality

Lines of Code

Our codebase size is growing linearly with time, by roughly 1.5k LOC a day. This is expected for a startup, but exposes us to the universal law that the number of bugs grows with codebase size. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10].

The above chart serves as a stark reminder that all software teams have to deal with: tame complexity or progress grinds to a halt.

Type Coverage

Type coverage here means explicit types rather than any. Normally the quest for strict type coverage is viewed as futile, pedantic and extremely restrictive. However, with the advent of LLMs, every time we replace an any with a concrete type, we remove ambiguity for the models. So we settled with 96% type coverage, which feels like a good high mark, without hitting diminishing returns.

The models know exactly what arguments functions / methods expect, and what they return. Strong and explicit typing removes a vast majority of the guesswork for LLMs.

Code Complexity

Cyclomatic complexity measures how many independent paths exist through our code. Higher numbers mean more branches, more edge cases, and more bugs. We track both average and maximum complexity to catch files that are getting out of hand.

Click "Complexity Max" in the legend to overlay the maximum values.

When the max complexity spikes, it's usually a single file that needs attention. We can catch these with the chart below.

Complexity Hit List

Code Duplication

This is where LLMs struggle the most. Models default to the path of least resistance: rather than finding and reusing an existing utility, they'll write a new one. Rather than importing a shared type, they'll redeclare it inline. The result is subtle (the code works, tests pass) but duplication creeps up across the codebase. Left unchecked, you end up with multiple sources of truth for the same logic, and a bug fix in one place doesn't propagate to the copies.

We use jscpd to detect duplicated blocks and enforce a threshold in CI. When duplication spikes, it's almost always a sign that a refactor is overdue: extracting a shared module or consolidating types.

Test Coverage

Getting a single coverage number for a full-stack app is harder than it sounds. Unit tests cover backend logic, E2E tests exercise the frontend through a browser, and each produces its own coverage report in a different format. We merge them using Istanbul via nyc, which gives us one unified percentage across the entire codebase: lines, branches, functions, and statements.

The sharp step shifts in the chart below correspond to refactoring sprints, where we targeted the least-covered files (more on that below). Gradual climbs reflect day-to-day test writing as part of normal feature work.

To find files that need covering, we use the chart below.

Least Covered Files

This is where coverage stops being an abstract percentage and becomes a to-do list. The table ranks files by how little coverage they have, so a test sprint starts by picking the worst offenders from the top. We write tests for those files, push, and watch the overall coverage number climb in the chart above.

This part is rather easy, as it mostly involves asking our agentic setup to write tests, and another more adversarial pass that ensures the tests aren't being lazy, and are truly testing functionality.

Playwright E2E Testing

The metrics above tell you about your code at rest: how complex it is, how much is duplicated, how well it's typed. But none of that matters if the app doesn't actually work. That's where E2E tests come in.

Playwright launches 8 real browsers in parallel, each with a bot that navigates the app like a user would: clicking buttons, filling forms, checking that the right data appears on screen. This lets us verify that core user flows survive even substantial refactors. The downside is that E2E tests are inherently fragile. Network hiccups, server latency, timeouts, and race conditions mean that a passing test today can fail tomorrow with no code change. This flakiness became a real problem for our CI reliability.

Test Results Over Time

The first thing we wanted to see was the big picture: are our tests getting more reliable or less? This chart shows passes, failures, and retries over time.

Blended Flakiness Score

This is our CI reliability hit list, and arguably the single most valuable chart in this entire post. It uses FlowPilot's calculate_field transform to compute a weighted score: 10 points per failure, 2 points per retry. The AI generated this computed metric from a natural language prompt.

Having one ranked table that blends failures and retries into a single score makes prioritization trivial: fix the top item, watch CI reliability improve, repeat. Before this chart existed, flaky tests were a vague frustration. Now they're a sorted backlog.

Failures by Spec File

Sometimes the problem isn't a single test; it's an entire spec file. Maybe it's testing a flaky feature, or the test setup is unreliable. This bar chart shows which spec files fail most often.

When one file dominates this chart, it's usually a sign that the feature itself needs attention: either better test isolation or fixing the underlying flakiness.

Filtering by Timeframe

All the charts and tables above become significantly more useful when you can narrow the window. Did flakiness get worse after last Tuesday's deploy? Is coverage trending up over the past month, or just the past week? FlowPilot supports date range filtering on any chart, so we can answer these questions instantly.

What We Learned

Dogfooding our own dashboards for quality tracking taught us a few things:

Build Your Own Charts

FlowPilot's AI assistant can generate charts like these from natural language. Describe what you want to see, and it builds the recipe for you, including computed fields, filters, and visualizations.