Some requests to API endpoints may generate a very large output in JSON format.
As an example, the crAPI vulnerable web app that I have been practicing with lately has an endpoint that lists the details of all recent posts in the community section of the app. These details happen to include the e-mail addresses of the registered users that have posted in the forum.
Now suppose you want to extract these e-mail addresses to test your password bruteforcing skills, without having to read through all the JSON syntax and potentially miss out some data.
Here’s a quick, dirty and very effective way to speed things up.
Export a JSON file
If you are testing the API in Postman and got a large JSON output as a response to one of the requests you sent, go to the Save Response menu at the top right of the response section and choose Save to a file.
You will obtain a file called response.json.
If you are using another tool to probe your target API, you can just cut and paste the JSON output into a text file.
Use a regex to search the file
Now cd into the directory where you saved the file and run:
cat response.json | grep -oe "[a-zA-Z0-9._]\+@[a-zA-Z]\+.[a-zA-Z]\+"
The regex we are using will allow the grep command to extract from the file all the strings in an e-mail address format.
Get a clean list
You will see in the output that some addresses may appear several times. To get a clean list without duplicates, just pipe the output into a
sort -u command:
cat response.json | grep -oe "[a-zA-Z0-9._]\+@[a-zA-Z]\+.[a-zA-Z]\+" | sort -u
Voila! You have a list of addresses ready for a bruteforce attempt.
Also, you will notice in the video that Corey Ball is using this slightly different command:
grep -oe "[a-zA-Z0-9._]\+@[a-zA-Z]\+.[a-zA-Z]\+" response.json
This syntax works just as well. Enjoy!