cURL is one of the most ubiquitous tools used by developers to test REST APIs. In this comprehensive guide, you‘ll learn how to fully utilize cURL for testing API requests and responses.
We‘ll start with a quick overview of cURL, then jump into hands-on examples, tips and tricks, detailed debugging techniques, and finally wrap up with discussing alternatives and limitations of using cURL for testing APIs.
A Brief History of cURL
cURL first emerged in 1997 as an open source project started by Swedish developer Daniel Stenberg. The name stands for "Client URL", and it was designed as a command line tool for transferring data using various protocols.
Over the last 25+ years, cURL has become practically synonymous with making HTTP requests. Its adoption steadily grew thanks to being included in many Linux distributions and macOS. Today, cURL remains one of the most used HTTP client libraries and tools.
Let‘s explore what makes cURL so ubiquitous for testing REST APIs.
An Overview of cURL Capabilities
cURL supports a wide range of use cases:
- Making HTTP requests with any method – GET, POST, PUT, DELETE, etc.
- Customizing requests with headers, body data, parameters
- Authentication – user credentials, API keys, secure tokens
- Downloading and uploading files via FTP, SFTP
- Scraping data from web pages
- Automating transfers with scripting capabilities
cURL has over 100 command line options and supports around 22 protocols including HTTP, HTTPS, FTP, FTPS, SFTP, LDAP, MQTT, POP3, RTSP, RTMP, and SMTP.
This makes cURL a very versatile tool both for developers working with APIs as well as sysadmins, DevOps engineers, and testers interacting with various systems.
Next, let‘s go through the steps of using cURL for API testing.
cURL Command Syntax and Structure
The basic format of a cURL command is:
curl [options] [URL]
For example:
curl https://api.example.com/users
This will execute a simple GET request.
Some common options you‘ll want to use:
-X
– Specify request method like GET, POST, PUT-H
– Add request header(s)-d
– Add POST data-u
– Set basic authentication username and password-i
– Include response headers in output-o
– Save output to file
You can also chain options together like:
curl -iX POST -H "Content-Type: application/json" -d ‘{"title":"Post Title"}‘ https://api.example.com/posts
This shows sending a POST request with headers and data.
Next, let‘s walk through some examples of interacting with a sample API using various cURL commands.
Using cURL to Test REST API Calls
We‘ll be making requests to a dummy JSONPlaceholder REST API that allows adding, deleting and updating fake resources.
Let‘s go through both simple and more advanced examples of hitting endpoints with different HTTP methods.
Fetching Resources with GET
Let‘s start by making a GET request to retrieve all posts:
curl https://jsonplaceholder.typicode.com/posts
By default cURL will make a GET request. This will return a JSON array of posts.
To add extra output, we can include response headers with -i
:
curl -i https://jsonplaceholder.typicode.com/posts
And to view request headers as well, use -v
:
curl -iv https://jsonplaceholder.typicode.com/posts
We can also save response to a file instead of printing to stdout:
curl https://jsonplaceholder.typicode.com/posts -o posts.json
Creating Resources with POST
To create a new resource, we‘ll need to make a POST request by passing some data.
curl -X POST -H "Content-Type: application/json" -d ‘{"title":"My New Post","body":"Post content"}‘ https://jsonplaceholder.typicode.com/posts
Here we use -X POST
to change the method, -H
to set a header, and -d
to pass the request body.
To add basic access authentication:
curl -u username:password -X POST -d ‘{"title":"Post Title"}‘ https://jsonplaceholder.typicode.com/posts
Updating Resources with PUT
For updating existing resources, we can use PUT requests:
curl -X PUT -d ‘{"title":"Updated Post Title"}‘ https://jsonplaceholder.typicode.com/posts/1
This will update the post with ID 1.
Deleting Resources
To delete a resource, pass the -X DELETE
option:
curl -X DELETE https://jsonplaceholder.typicode.com/posts/1
And we have now covered making all major CRUD operations using the cURL command line tool!
Of course, there are many additional options for fine-tuning requests which we‘ll get into next.
Common cURL Options and Usage Tips
Beyond basic GET and POST requests, there are many helpful cURL options to customize and debug API calls.
Setting Request Method
Use -X
or --request
to change HTTP method:
curl -X DELETE https://api.example.com/posts/123
Adding Request Headers
Set one or more headers with -H "Header:Value"
:
curl -H "Content-Type: application/json" -H "Authorization: Bearer token" https://api.example.com
Data in Body vs. URL Parameters
For POST requests, data can be passed directly on the command line with -d
:
curl -d ‘{"key":"value"}‘ https://api.example.com
Or, for GET requests, parameters can be passed in the URL:
curl https://api.example.com?key1=value1&key2=value2
Authentication
For basic auth, use -u username:password
:
curl -u myusername:password https://api.example.com
Or set an Authorization header manually:
curl -H "Authorization: Bearer <token>" https://api.example.com
Output Options
View response headers with -i
:
curl -i https://api.example.com/posts/123
Enable verbose mode to see request headers as well with -v
:
curl -v https://api.example.com
Save response body to a file instead of printing to terminal with -o
:
curl https://api.example.com/file.pdf -o file.pdf
There are many more options for controlling how curl functions – far more than we can cover here!
Next, let‘s go over some best practices when using cURL to interact with REST APIs.
cURL Best Practices for Testing REST APIs
When testing REST APIs, here are some tips for making effective use of cURL:
Use proper HTTP methods – Leverage the built-in verbs like GET, POST, PUT, DELETE to perform the appropriate operation instead of relying solely on POST.
Structure requests properly – Include Content-Type and Accept headers to specify formats like JSON or XML. Set other headers like Authorization to match what API expects.
Validate error handling – Check that the API returns appropriate status codes like 400 or 500 for invalid requests and errors.
Inspect raw responses – Look at the raw response body, headers, and status codes instead of relying solely on cURL‘s exit code.
Test with different request sizes – Try small and large payloads to verify the API handles length properly.
Parameterize and script requests – Save frequently used cURL commands to files or scripts to re-run them with different parameters.
Learn from HTTP traces – Use verbose output and traffic inspection to understand the raw HTTP conversation in detail.
Now let‘s look at some ways to debug requests and responses when issues occur.
Debugging REST APIs with cURL
When tests fail or problems occur, you‘ll need to dive deeper into requests and responses to troubleshoot.
Here are techniques for unlocking additional insight using cURL:
Enabling Verbose Output
The -v
flag prints full request and response details:
curl -v https://api.example.com
Inspecting HTTP Traffic
Proxy tools like Charles allow viewing all traffic between your machine and APIs.
Capturing Packets
Lower-level inspection using tcpdump and Wireshark to analyze network packets.
Saving Responses
Dump responses to files for further analysis:
curl https://api.example.com/data > response.json
Formatting JSON
Pipe JSON output to jq for easy parsing and formatting:
curl https://api.example.com | jq
Correlating with Application Logs
Match backend application log entries with cURL requests and responses.
Development teams will often combine multiple debugging techniques to pinpoint issues when working with REST APIs.
Limitations of cURL for API Testing
While being a ubiquitous tool for testing REST APIs, cURL does have some limitations:
- Not very intuitive or user-friendly interface
- Difficult to perform complex or sequential test scenarios
- No built-in assertions or test reporting
- Lack of developer-focused features like environments, runners, documentation
For API testing needs beyond simple request/response inspection, more full-featured tools are recommended:
- Postman – Provides complete API development environment
- Insomnia – Feature-rich REST client with beautiful UI
- SoapUI – Specialized in SOAP/XML testing as well as REST
- REST Assured – Java DSL tailored for API testing
- Karate – Combines API test automation, mocks, and reports
These tools build on cURL‘s foundations but provide enhanced workflows and outputs tailored for testing teams.
Conclusion
While newer API testing tools continue to emerge and evolve, cURL remains a staple that developers rely on for inspecting REST APIs during development. Its flexibility through command line usage and scripting makes cURL invaluable.
By mastering cURL options for constructing requests, inspecting responses, generating traces, and scripting repetitive test cases, you can debug APIs with ease and automate validation checks.
Combined with understanding how to leverage its output for troubleshooting, cURL will remain a "must have" tool for any REST API developer‘s toolbox for the foreseeable future.