01—The problem nobody wants to admit
100% coverage. Every test green. And still, every time you refactor something you know is safe, three or four tests blow up for reasons that have nothing to do with the change you made. That's not a coverage problem. That's a trust problem, and it's a much bigger one.
I spent years on teams that leaned hard on mocks before I admitted something uncomfortable to myself: a lot of the time, mocks make you feel like you're testing behavior when you're really just checking that a method got called. Once you're verifying implementation, every internal change becomes a threat, even when the actual behavior hasn't moved an inch.
None of this means mocks are bad. It means we reach for them far more than we should, and this piece is my attempt to show exactly where that starts to hurt, one added dependency at a time.
02—Test doubles: a taxonomy, not a hierarchy
People use "fake" and "mock" interchangeably for "some pretend object in a test," which causes more confusion than it should. There are five real categories, but the split that matters isn't between all five individually — it's between state-based doubles (dummy, stub, fake) and interaction-based doubles (spy, mock). A fake isn't a fancier mock. It's a different way of thinking about what a test should even check.
| Type | What it does | Verifies interactions | Typical example |
|---|---|---|---|
| Dummy | Fills a parameter slot, never used | No | null or an empty object |
| Stub | Returns canned values | No | every { repo.findById(any()) } returns customer |
| Spy | Like a stub, but records usage | Passively | A manual call counter |
| Mock | Like a spy, with hardcoded expectations | Actively | verify(exactly = 1) { repo.save(any()) } |
| Fake | A real, lightweight implementation | No — you assert on state | FakeCustomerRepository backed by a HashMap |
mockk() can end up as any of the top four rows, depending on how you configure it, so "I used a mocking library" doesn't actually tell you much. The question that matters is whether your test checks what happened, or how it happened.
03—Why tests matter, briefly
Coverage tells you which lines executed. It says nothing about whether you checked the right thing. A good test reads like documentation that can't go stale — if it passes, the behavior it describes is real; if it fails, something changed. There's a less obvious effect too: tests quietly shape the production code around them. Chase mockability and you end up with interfaces everywhere and logic sliced into tiny methods purely so something can be stubbed out. Chase behavior instead, and the code tends to end up doing the right thing, because that's the only thing being rewarded.
04—Kent Beck's Test Desiderata
Kent Beck laid out twelve properties of a good test, first in a 2019 essay and later refined in a 2022 newsletter. Two of them do most of the heavy lifting for this argument.
Structure-insensitive
His own words: "Tests should be coupled to the behavior of the code and decoupled from its structure." Strict mocks fail this constantly. Call counts, ordering, exact arguments — none of that is behavior. It's structure wearing a disguise.
Behavioral
A test should fail when behavior changes. Here's the trap: a mock-based test can pass while the code underneath it is flatly broken, because all it confirmed is that a method got called.
// Passes even if the code is broken
@Test
fun `should save order`() {
val orderRepository = mockk()
every { orderRepository.save(any()) } returns Order()
useCase.execute(validOrder)
verify { orderRepository.save(any()) } // green... but was the order actually saved correctly?
} 05—What refactoring actually means
Refactoring changes structure without changing observable behavior. It's how something happens, not what happens. Keep that definition close — everything below gets judged against it.
| Safe refactor (test shouldn't break) | Behavior change (test should break) |
|---|---|
| Renaming an internal variable | Changing the order of business validation |
| Extracting logic into a private method | Changing which exception gets thrown |
| Splitting a long method into steps | Changing the result of an operation |
| Moving code around inside a method | Adding a new step to the process |
| Adding caching | Changing a business rule |
| Making a call asynchronous, if nobody downstream ever observed the wait | Making a call asynchronous, if a caller depended on the ordering or the block |
That last row is the messy one on purpose. Sync-to-async only counts as a pure refactor if nothing outside the method actually noticed the synchronous behavior. It's a decent gut check before you assume any change is "just structure."
06—The story: five branches, one growing system
Same use case throughout — PlaceOrderUseCase — gaining exactly one new kind of dependency per branch. At each level we run the same three tests: happy path, a rename, and a private-method extraction. Once with mocks, once with fakes. Then we look at what actually broke.
07—Level 0 — one dependency (branch: level-0-single-repository)
class PlaceOrderUseCase(private val customerRepository: CustomerRepository) {
fun execute(customerId: String): Customer =
customerRepository.findById(CustomerId.fromString(customerId))
?: throw CustomerNotFoundException(customerId)
}At this scale, mocks and fakes look basically indistinguishable, which is the point. Start here if you're new to this.
// Mock
val repo = mockk()
every { repo.findById(any()) } returns customer
assertEquals(customer, useCase.execute(customer.id.value()))
// Fake
val repo = FakeCustomerRepository().also { it.save(customer) }
assertEquals(customer, useCase.execute(customer.id.value())) Rename the parameter, pull the lookup into a private method — both survive without complaint. Plenty of engineers stop here and conclude the whole fakes-vs-mocks debate is overblown. It isn't. It's just that one dependency isn't enough to expose the difference yet.
08—Level 1 — two dependencies, the first crack (branch: level-1-two-repositories)
Add StockRepository. Now the use case checks stock before creating anything. Pull the lookup-and-validate logic into a private method — a refactor with zero behavioral impact.
| Mocks | Fakes | |
|---|---|---|
| Happy path | passes | passes |
| Rename a variable | passes | passes |
| Extract private method | fails — verify(exactly = 1) now sees two calls | passes — the stock state is unchanged |
This is the smallest possible version of the whole argument, and it's worth sitting with before moving on, because every branch after this is the same failure at a bigger scale.
09—The broken black box
There's a cost to strict mocks that never shows up in a timing report but is probably the one that wears a team down the fastest: friction.
When a test fails because behavior actually changed, fixing it feels fine — the test did its job. But when a test fails over a refactor you know is harmless, and you still have to stop and dig into why, that time feels stolen. Do this enough times and people start resenting their own tests. They get commented out. Marked @Ignore. "Fixed" by copy-pasting whatever verify() call makes the red go away, without anyone really understanding what changed. A test that breaks for no business reason doesn't just cost the minutes it takes to fix — it costs some of the trust the team had in every other test in the suite, and that trust doesn't come back cheap.
There's a concrete technical reason behind that friction: fixing a broken mock means giving up on treating the use case as a black box.
Look at what happened in Level 1. The public shape of execute() didn't change. Its contract — same input, same output or same exception — didn't change either. But to update verify(exactly = 1) { stockRepository.findByProductId(any()) } into verify(exactly = 2), you have to open the method, count the actual calls, and figure out why there are two now instead of one. You're debugging the internals of something that, by design, shouldn't require you to know its internals at all.
A fake sidesteps this because it never made that promise in the first place. The test only needs the public interface of StockRepository — which it already knew, since that's the type it compiles against — and it checks the resulting state. Whether the method calls the repository once or five times internally is the use case's own business, and its tests have no opinion on it.
One thing worth being precise about: adding a new collaborator breaks compilation either way. Add a NotificationService parameter to the constructor and both the fake-based and mock-based tests stop compiling — that's just how a statically typed language like Kotlin or Java works, and neither approach gets a pass on it.
The difference shows up right after that compile error goes away.
// Fake: one line, and the test is green again
val useCase = PlaceOrderUseCase(
customerRepository, stockRepository, orderRepository,
notificationService = FakeNotificationService()
)// Strict mock: now you have to decide what to expect
val notificationService = mockk()
every { notificationService.notify(any()) } returns Unit
// Called once? With what argument? To know, you have to read
// the body of execute() — so much for the black box. Satisfying the constructor with a fake is mechanical. Satisfying it with a mock is only step one — you still have to open the method and decide what behavior you're willing to demand from that call, or the test will pass without checking anything real. The fake spares you the work of understanding the implementation. The mock, at best, spares you the work of writing a real implementation, but it still makes you understand it well enough to mock it convincingly.
This is really the same idea Beck calls "cheap to change," just seen from the angle of who has to make the fix and what they need to know to do it. A test that demands you read production code just to repair it isn't only expensive in minutes — it's expensive because it defeats the abstraction the code was trying to offer in the first place.
10—Level 2 — the full checkout (branch: level-2-full-checkout)
PlaceOrderUseCase at full scope now: customer lookup, per-item stock checks, order creation, stock reservation. Three collaborators. Reorder the validation — check everything up front for a better error message — and any mock asserting call order or count breaks. Fakes don't even notice, because the final state (order saved, stock reserved) is identical regardless of the order things happened in.
// Fake assertion, unaffected by internal reordering
val updatedStock = stockRepository.findByProductId(productId)
assertEquals(98, updatedStock.availableQuantity)By now the mock-based suite has accumulated enough verify() calls that a single reordering turns four tests red for no behavioral reason at all. This is usually the point where a team starts calling its own suite "brittle" — without quite putting together that fixing it means breaking the exact encapsulation the use case was supposed to have.
11—Level 3 — side effects show up (branch: level-3-domain-events)
PlaceOrderUseCase now publishes an OrderPlacedEvent. This branch exists on purpose to show mocks doing their actual job well. You don't want a fake event bus quietly delivering events in memory so you can assert on business state — an event has no state worth asserting on. Its whole reason to exist is the side effect.
verify { eventPublisher.publish(match { it is OrderPlacedEvent && it.orderId == expectedId }) }The lesson here isn't "mocks are bad." It's "use the tool that matches what you're actually checking." Side effects get verified. State gets asserted.
12—Level 4 — a system you don't own (branch: level-4-payment-gateway)
Add a call to a third-party PaymentGatewayClient. This is where Google's old guidance about not mocking types you don't own stops being trivia and starts mattering. Mock that client directly and every test now bakes in assumptions about a library's API shape — the moment that library changes a method signature, your mocks keep compiling happily while the real integration quietly breaks in production. The fix isn't to avoid mocking it. It's to wrap it behind an interface you control, and fake that:
interface PaymentGateway {
fun charge(amount: Money, customerId: CustomerId): PaymentResult
}
class FakePaymentGateway : PaymentGateway {
var nextResult: PaymentResult = PaymentResult.Approved
override fun charge(amount: Money, customerId: CustomerId) = nextResult
}This is also where "narrow contract vs wide contract" stops being theoretical. CustomerRepository has two methods — trivial to fake in full. A real payment SDK might expose forty. At that width, faking the whole thing isn't worth doing. Fake the narrow slice your use case actually touches, hidden behind your own interface, and let the mock live only at the boundary you genuinely don't control.
13—Fakes need contract tests too
A fake can drift from what the real implementation actually does — Marcelo Chiaradia calls this the divergence problem, and it's a fair criticism. The fix is an abstract test suite that both implementations have to pass:
abstract class CustomerRepositoryContractTest {
abstract fun createRepository(): CustomerRepository
@Test
fun `should save and find customer by id`() {
val repo = createRepository()
val customer = Customer.create(...)
repo.save(customer)
assertEquals(customer, repo.findById(customer.id))
}
}
class FakeCustomerRepositoryContractTest : CustomerRepositoryContractTest() {
override fun createRepository() = FakeCustomerRepository()
}
class JpaCustomerRepositoryContractTest : CustomerRepositoryContractTest() {
override fun createRepository() = JpaCustomerRepository(dataSource)
}If that contract passes against the fake and fails against the real repository, you've caught a persistence bug at test time, not three weeks later in production.
None of this is free, by the way. Every time the real dependency's contract shifts — a new required field, a new failure mode — the fake and its contract test need updating too. Plan for that cost up front. A fake nobody maintains anymore is arguably worse than a mock, because it still looks trustworthy while quietly lying to you.
14—Repository tests: unit vs integration
Repository tests shouldn't be unit tests. Testing PlaceOrderUseCase against a fake proves the use case talks to its repository correctly. It proves nothing about whether the real implementation actually persists anything to Postgres.
| Test type | What it verifies | With what |
|---|---|---|
| Unit test | Business logic | Fakes |
| Integration test | Real persistence | Real database (Testcontainers) |
15—Numbers you can reproduce yourself
Clone the project, run ./scripts/benchmark.sh. It runs each branch's suite 30 times, throws away JVM warmup, reports median and p95. So these numbers are yours to check, not mine to be believed:
| Branch | Mocks (median) | Fakes (median) |
|---|---|---|
| level-0 | ~40 ms | ~5 ms |
| level-2 | ~530 ms | ~17 ms |
| level-4 | ~890 ms | ~22 ms |
The gap widens as dependencies pile up, because every mockk
16—When each one actually earns its place
| Situation | Reach for |
|---|---|
| Domain logic, business rules, aggregate behavior | Fake |
| Tests need to survive structural refactors | Fake |
| Non-technical people read tests as specs | Fake |
| Verifying a side effect happened (email sent, event published) | Mock |
| A type you don't own (SDK, vendor client) | Wrap it, fake your own interface, mock only the boundary |
| Rare, hard-to-reproduce failure conditions | Mock or stub |
| Legacy code where a proper fake isn't realistic yet | Mock, as a stepping stone |
17—A short checklist before your next test
- Am I checking behavior (final state) or structure (how it got there)?
- If I refactor without touching behavior, does this test survive?
- Does the test name read like the business, or like the code?
- If this passes, would I actually feel good deploying?
- Does my fake pass the same contract test as the real implementation?
- Is this "unit test" secretly an integration test wearing a costume?
- If it's hard to build a fake for this, is that telling me the interface is too wide?
- If this test breaks, will fixing it force me to read production code I shouldn't need to understand?
18—Where this lands
Mocks aren't the villain of this story — Level 3 and Level 4 show exactly where they earn their spot: side effects, systems you don't own, legacy constraints you haven't fixed yet. The trouble starts when they become the default for business logic that has real, checkable state. Beyond the raw numbers, there's a cost that's harder to put on a slide: a team that keeps fighting tests it doesn't trust eventually stops trusting testing at all. Go run the five branches yourself and watch where things actually turn red. That'll teach you more than any rule of thumb in this article.
19—References
- Beck, K. (2019), Test Desiderata
- Beck, K. (2022), Desirable Unit Tests
- Chiaradia, M., Using Fakes for Testing
- Meszaros, G. (2007), xUnit Test Patterns
- Winters, Manshreck & Wright (2020), Software Engineering at Google, ch. 13
- Google Testing Blog, Don't Mock Types You Don't Own, Know Your Test Doubles
- Feathers, M. (2004), Working Effectively with Legacy Code