How to Use Curl Command in Linux: From Basics to Advanced

Curl is an incredibly powerful command-line tool that’s indispensable for anyone working with web technologies. It allows you to send and receive data using a variety of protocols, making it a versatile tool for developers, system administrators, and anyone else who needs to interact with web services.

In this guide, we’ll explore how to use curl to interact with REST APIs, a common task that showcases its utility. We’ll use the JSONPlaceholder API as our example, demonstrating how to fetch a list of posts and then add a new one.

curl command in Linux

curl stands for Client URL, which might already give you a hint about its functionality. It’s a command-line tool used to transfer data to or from a server using various protocols such as HTTP, HTTPS, FTP, and more. One of the things I absolutely love about curl is its simplicity for quick tasks yet its extensive set of features for complex operations. On the flip side, its plethora of options can sometimes be overwhelming for beginners.

Basic usage of curl

Let’s start with the basics. The simplest way to use curl is to fetch the content of a webpage. Here’s an example that also showcases the beauty of the Ubuntu terminal:

curl http://example.com

This command will output the HTML content of “http://example.com” right into your terminal. It’s a straightforward way to quickly peek into a webpage’s source without opening a web browser.

Downloading files with curl

One common task I often use curl for is downloading files. To save a file to the local filesystem, you can use the -o (or --output) option:

curl -o example.html http://example.com

This command fetches the HTML content of “http://example.com” and saves it as example.html in your current directory. Simple, yet powerful!

Sending data with curl

curl isn’t just for downloading; it’s also incredibly efficient at sending data. Here’s how you can make a POST request with form data:

curl -d "param1=value1&param2=value2" -X POST http://example.com/form

In this example, -d specifies the data you want to send, and -X specifies the HTTP method (in this case, POST). It’s a handy way to interact with APIs or submit forms from the command line.

Working with headers

Sometimes, you need to include HTTP headers in your requests. curl has you covered with the -H option:

curl -H "Content-Type: application/json" -X POST -d '{"username":"example","password":"1234"}' http://example.com/api/login

This command sends a JSON object to the server, which is typical when dealing with REST APIs. The ability to set headers easily is something I appreciate a lot about curl.

Saving session cookies

For websites requiring authentication, managing session cookies is essential. curl can handle this with the -c (cookie jar) option:

curl -c cookies.txt -d "username=example&password=1234" http://example.com/login

This will save the session cookies to cookies.txt, which you can then use in subsequent requests with the -b option to stay authenticated.

Why I love and hate curl

To sum up, curl is a tool I find indispensable for web-related tasks in Linux. Its versatility and power are unmatched in the realm of command-line HTTP clients. That said, its vast array of options can sometimes be a double-edged sword, making it intimidating for beginners and even causing occasional frustration for seasoned users when a particular flag is forgotten.

Handling redirects

When you request a URL that leads to a redirect, curl by default doesn’t follow it. However, you can use the -L option to tell curl to follow redirects until it reaches the final destination:

curl -L http://example.com/redirect

This is particularly useful when downloading files from sources that use redirects to manage downloads. Personally, I find the -L flag a lifesaver in scripts where I need to download resources without manual intervention.

Verbose and silent modes

Understanding what curl is doing behind the scenes can be crucial for debugging. The -v (verbose) option provides a detailed trace of the request, showing headers and more:

curl -v http://example.com

On the flip side, if you prefer a cleaner output, especially when you’re incorporating curl in scripts, the -s (silent) mode is incredibly handy:

curl -s http://example.com > example.html

I have a love-hate relationship with the verbose output. It’s invaluable for debugging but can be overwhelming with information overload for simple tasks.

Uploading files

curl isn’t just great at downloading; it’s also efficient at uploading files to a server. Using the -F option, you can upload files via multipart/form-data (common in web forms):

curl -F "file=@path/to/local/file" http://example.com/upload

The @ symbol is key here, indicating the local file path. This feature has been incredibly useful for automating backups and file transfers in my projects.

Using curl with HTTPS

In today’s web, HTTPS is a must for secure communication. curl works seamlessly with HTTPS, but sometimes you might encounter self-signed certificates or other SSL issues. To bypass these for testing purposes (note: this is not recommended for production use), you can use the -k or --insecure option:

curl -k https://example.com

While the -k option is helpful in certain situations, I always encourage fixing the certificate issues rather than bypassing them for long-term solutions.

JSON and APIs

Interacting with REST APIs is a common task that often requires dealing with JSON data. curl makes it easy to send JSON requests using the -H header option along with application/json and the -d data option:

curl -H "Content-Type: application/json" -d '{"key":"value"}' -X POST http://example.com/api/resource

Dealing with APIs and JSON efficiently is probably one of my favorite uses of curl. It exemplifies the power and flexibility of the command line for modern web tasks.

Sure, let’s walk through a real-world example where we’ll use curl to interact with a REST API. We’ll simulate fetching a list of posts from a JSON placeholder API, a common task for developers testing API interactions. This example will showcase how to use curl to make a GET request, interpret JSON output, and then make a POST request to add a new post.

Environment Setup

For this walkthrough, ensure you have curl installed on your system. Most Linux distributions come with curl pre-installed. You can check by typing curl --version in your terminal. If it’s not installed, you can usually install it using your distribution’s package manager.

Step 1: Fetching Data with a GET Request

First, we’re going to fetch a list of posts from the JSONPlaceholder API, an online REST API that you can use for testing and prototyping.

Open your terminal and enter the following command:

curl https://jsonplaceholder.typicode.com/posts

This command sends a GET request to the API. You’ll see a list of posts in JSON format displayed in your terminal. This output represents the data fetched from the API. It’s quite a lot, so feel free to add | less at the end of the command to paginate the output, like this:

curl https://jsonplaceholder.typicode.com/posts | less

Step 2: Understanding the Output

The JSON output you see is a list of posts, each with several fields: userId, id, title, and body. Here’s a snippet of what one post might look like:

{
  "userId": 1,
  "id": 1,
  "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
  "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
}

Step 3: Adding a New Post with a POST Request

Now, let’s simulate adding a new post. We’ll use a POST request with JSON data for this. The command looks a bit more complex because we’re including data with the -d flag and specifying that we’re sending JSON with the -H flag.

Type the following in your terminal:

curl -X POST -H "Content-Type: application/json" -d '{"title": "Hello World", "body": "This is a test post from curl", "userId": 1}' https://jsonplaceholder.typicode.com/posts

This command sends a POST request to add a new post with the title “Hello World”, a body of “This is a test post from curl”, and a userId of 1.

Step 4: Interpreting the Response

After sending the POST request, you’ll receive a JSON response from the server. This response includes the data you sent along with a new id for the post. Here’s an example of what you might see:

{
  "title": "Hello World",
  "body": "This is a test post from curl",
  "userId": 1,
  "id": 101
}

The API has assigned an id of 101 to our new post, indicating it was successfully created.

Conclusion

Our exploration of curl, particularly through a practical example, has highlighted its critical role and flexibility within the tech landscape. By engaging in data retrieval and submission with a REST API, we’ve merely begun to uncover the breadth of curl’s capabilities. This experience serves as a powerful illustration of the tool’s value across a diverse range of tasks, from API testing to automating the processes of fetching and sending data. As we further investigate its features and potential uses, curl continues to stand out as an essential tool for streamlining web interactions, proving itself to be an invaluable asset for efficiently navigating the digital world.