TestinGil - Agile SW Testing Consultancy

"Here's an API, test it".

Ok, it works. And let's say I really put in the work to see that it does.

What does it say about the system? The feature? The workflow?
All the things that are a lot more than one API call?

Most of the time - not a lot. Yes we have a passing test or even a few. But do I know how the workflow containing that API call works?

No. We need a lot more than singular API tests.
Here's the blog:
https://testingil.com/2026/06/test-the-feature-not-the-endpoint.html

What's the difference between a unit test and an API test?
That's an easy one, right?

Here's a harder one. What's similar between a unit test and an API test?

No, it's not the test framework. And if you get this, you get a more confident test result.

Read the new blog:
https://testingil.com/wp-content/uploads/2026/06/au2c751.jpg

Do you know when you're making an assumption?
Because if you do, thinking through it can bring up interesting ideas.

Let's say we write a test that has State Pollution assumption in it - we're using a unique ID, that we expect not to be there, when we create an item.

If we're not saying to ourselves: "There is not going an item with that ID when the test runs" - Nothing will happen, until that ID shows up out of nowhere.

But if we're having the conversation and say "Well, we assume the ID won't be there. But what if does?", we can also ask:
- Will it fail our test?
- Do we need to guard against it?
- How can it show up uninvited?
- Can we make sure it doesn't show up uninvited?

Just by thinking about it, making the assumption explicit, we can start handling problem that haven't happened yet.

The "Fighting Flakiness" webinar recording is up. I walk through seven suspects (including State Pollution) — each one is an assumption we don't want to leave implicit.
https://youtu.be/RXvrOVv0kXo

Fix Flaky API Tests: 7 Assumptions Breaking Your CI [Webinar] Flaky tests. We've all got them. We rerun, they pass. We shrug and ...

You just finished writing your 8535th test. Congrats!

When we write tests, we rely on our experience, other people's experience, and use the past as a guide to the future.

We know, from experience, this works most of the time. What experience will teaches us, is that we don't get the full picture every time. What we don't, we fill with assumptions. (there's a webinar for that, coming soon!)

And now, we don't need to write anymore. AI will do this for us!
More tests, more coverage, more... assumptions?

Yes, dude. The assumptions are not limited to your experience. No, you just inherited assumptions of so many people in the world. Whatever model you use, it was trained on a whole lot of code, and newsflash: people who had assumptions wrote it. And I'm not going into the inception angle now - generated code from generated code.

That means, that tests - our source of truth - need to be reviewed a lot more surgically. Not just if the tests do what they need to check, and check it correctly - what assumptions have stallaway-ed inside? What does it mean for future failures and log reading sessions?

AI amplifies everything - from productivity to complexity and flakiness?

Have you noticed that in your tests?

For years and years I've been telling people (especially my students, they paid for it) that tests are the real documentation of the system.

You can have all kinds of stale documents, collecting digital dusts. Sometimes real dust.

But if you have tests - running and passing tests, I need to be more specific here - that's current documentation.

That means they better be readable and covering a lot of cases. Because up until now, that was already a lot to ask for.

But with AI? Tests - as documentation - become even more important. AI generates more code than can be read, and fewer people over time know what the system actually does (The knowledge void).

So, in a couple of years, when someone will need to maintain the code, how well can they do it? With tests as good documentation, they have a better chance.

I know, AI pundits will tell you AI will have all the answers. But even if it did (I don't see that happening), it still needs good documentation. And I'd still want answers from a human that worked on the system for a couple of years.

So, are you treating your tests as documentation? Do you (a human) review all the tests? Makes sure names are good, and workflows explained? What else do you do to future proof your app?