The Ministry of Testing is running a 30 days of API Testing challenge. I’m going to give it a go…
I always believed that if you shared your goals, you’re more likely to achieve them, but whilst trying to find science to back up my thoughts I found several articles stating the opposite – sharing you’re goals makes you less likely to achieve them.
Well, time to test my hypothesis then!
I’ve done a lot of testing of APIs as part of software development roles I’ve undertaken, but I can’t say how deeply I thought about the topic. I learned enough to be effective in my role!
This exercise provides the perfect opportunity to do that deep thinking!
I’ll use this post for running commentary as I attempt each of the challenges. Let’s see what happens…
1. Define what API testing is
API testing is exploring the behaviour of Application Programmable Interfaces which send & receive messages with the endpoints of other components such as in the UI or DB layers. API testing checks that expectations are met & unexpected behaviour is uncovered.
API Testing is not only functional – it is possible to perform security, load & error handling tests through the API.
2. How would you approach API exploratory testing?
I’d use exploratory testing techniques I already use for testing systems across the entire stack such as:
- Map out the endpoints & associated params of the API under test.
- Map out the endpoints & associated params of APIs that communicate with the API under test.
- Compare responses with their requests
- Experiment with different variations of params to see the API behaviour.
- Identify example messages from typical user journeys to derive variations of those journeys (e.g in/valid tokens).
- Replay troublesome user journeys from production through the API to reproduce undesired behaviour.
- Identify data being passed through the API & use the STRIDE threat model to discover vulnerabilities of that data.
- Compare the API with it’s documentation to identify discrepancies.
- Force the API to fallover – how is the error handled? What is the impact to the customer?
- Open the console in Postman! (Top tip in Alan’s video)
EDIT I’ve just re-read Ash Winter’s ICEOVERMAD mnemonic,
Daniel Donbavand’s SPEEDLOADS mnemonic & Linda Roy’s VADER mnemonic – these would serve as great starting checklist from which I could launch deeper explorations.
3. Read an API testing related book
Aside from Heather’s recommendation, I’m also reading through Mulesoft’s “Undisturbed REST“.
I chose this book so I can help with & add value to API design conversations – this is part of my strategy for testing upstream.
4. Share a resource on HTTP & how it works
Understanding HTTP basics – One Month
Basic concepts of web applications, how they work and the HTTP protocol – Natural Programmer (I’m a sucker for RSA style animation!)
5. Contribute to the list of publicly available APIs
6. Read & share an interesting blog post on API testing
I went down a (positive) rathole on Nordic APIs site & found several interesting posts. These 2 stood out for me:
Mulesoft also has a stack of interesting posts, but I’ll choose this one:
And to overcome the vendor bias – I’m also sharing a post
from Danny Dainton on using Fiddler with Postman to set environments. I chose this post because setting environments is something I could have used in a previous role so I imagine I’ll need it again… 🙂
7. Complete exercise 1
8. Explore the API thread on The Club and contribute to the conversation
I opted for an easy option and asked Mark W a question about the automation in testing online resource:
9. Share some tools we can use to discover what API calls our applications are making
Here’s a resource I found useful which provides 10 tools:
Out of the 10, I’ve used:
I’ve been through a demo of Tricentis Tosca.
I’m keen to try Karate DSL
10. Share your favourite API testing tool & why
Has to be Postman for me – its the tool I’m most familiar with as it is the tool of choice for those programmers I have worked with. Using the same tool helps with collaboration & sharing assets – a programmer can navigate around my instance of postman as well as they can on their instance.
I do also use Fiddler for real time monitoring of traffic between browser & server.
17. API Testing Experts on Twitter
19. A useful API testing video
I’m interested in how I can take the next steps with Postman & get smarter with Newman. This video takes you through to running collections with Newman in Jenkins – a mission i’m looking to accomplish!
23. Resources for understanding status codes
The most concise list I could find! 🙂 HTTP Status Code Summary (via @stevelosh)
1xx: hold onThe Spiceworks Community
2xx: here you go
3xx: go away
4xx: you f$%ed up
5xx: I f$%ed up
I found this post with status codes as pictures helpful as well.
Finally the HTTP Status Map is a useful interactive resource.
Before I start, here are some resources I already have in my toolkit that I believe will come in useful:
- API, Web Services & Microservices Testing Pathway by Katrina Clokie
- Danny Dainton‘s Postman GitHub repo
- Alan Richardson‘s Thingifier
- API Framework by Mark Winteringham
- Automation In Testing Online by Mark Winteringham
- ICEOVERMAD mnemonic by Ash Winter
- SPEEDLOADS mnemonic by Daniel Donbavand
- API Testing Heuristics for Developers presentation by Linda Roy
- Exploratory Testing of an HTTP REST API with Postman for Response Validation and BurpSuite proxy video by Alan Richardson
- Exploratory Testing an API post by Maaret Pyhäjärvi
Pair Exploring an API with Thomas post by Elizabeth Hocke
- RestMud text adventure game by … Alan Richardson
- Let’s Talk – UKStar James Whittaker and Automating with Intent, From Data, and without GUIs video by … Alan Richardson