How to use `java.net.URLConnection` for HTTP requests?

General Discussions
, ,
1

The java url connect functionality is commonly asked about, but Oracle’s tutorial only covers basic GET requests. It doesn’t explain how to handle POST requests, set request headers, read response headers, manage cookies, submit HTML forms, or upload files. How can java.net.URLConnection be used to perform more advanced HTTP requests?

2

I’ve worked with java.net.URLConnection quite a bit, and to start with something simple like a GET request, here’s how you can easily add request headers and handle cookies. With java url connect, it’s straightforward to fetch data and handle things like user-agent and response cookies.

import java.io.*;
import java.net.*;
import java.util.*;

public class URLConnectionExample {
    public static void main(String[] args) throws IOException {
        URL url = new URL("https://example.com");
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("GET");

        // Set request headers
        conn.setRequestProperty("User-Agent", "Mozilla/5.0");
        conn.setRequestProperty("Accept", "application/json");

        // Handle cookies
        Map<String, List<String>> headers = conn.getHeaderFields();
        List<String> cookies = headers.get("Set-Cookie");
        if (cookies != null) {
            for (String cookie : cookies) {
                System.out.println("Received Cookie: " + cookie);
            }
        }

        // Read response
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
        }

        conn.disconnect();
    }
}

:white_check_mark: Why?

  • Supports request headers like User-Agent to mimic a browser.
  • Handles cookies by capturing Set-Cookie headers.
  • Can handle JSON or other response formats, making it more flexible.
3

As I continued working with java url connect, the next step was handling POST requests, especially when dealing with forms or APIs. Here’s how you send POST data, typically used when submitting forms.

import java.io.*;
import java.net.*;

public class URLPostRequest {
    public static void main(String[] args) throws IOException {
        URL url = new URL("https://example.com/login");
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("POST");
        conn.setDoOutput(true);

        // Set headers
        conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

        // Send form data
        String postData = "username=user&password=secret";
        try (OutputStream os = conn.getOutputStream()) {
            os.write(postData.getBytes());
            os.flush();
        }

        // Read response
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
        }

        conn.disconnect();
    }
}

:white_check_mark: Why?

  • Simulates form submissions similar to how a web browser would.
  • Works perfectly for APIs that expect form-encoded data (application/x-www-form-urlencoded).
4

Once you get familiar with basic GET and POST, file uploads are the next logical step. Uploading files using java url connect can be done via multipart/form-data. Here’s how to handle it when you need to send both text fields and files.

import java.io.*;
import java.net.*;

public class MultipartUpload {
    public static void main(String[] args) throws IOException {
        String boundary = "----WebKitFormBoundary";
        File file = new File("test.txt");
        URL url = new URL("https://example.com/upload");

        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("POST");
        conn.setDoOutput(true);
        conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);

        try (OutputStream os = conn.getOutputStream();
             PrintWriter writer = new PrintWriter(new OutputStreamWriter(os))) {

            // Write text field
            writer.println("--" + boundary);
            writer.println("Content-Disposition: form-data; name=\"description\"");
            writer.println();
            writer.println("Test file upload");
            writer.flush();

            // Write file field
            writer.println("--" + boundary);
            writer.println("Content-Disposition: form-data; name=\"file\"; filename=\"" + file.getName() + "\"");
            writer.println("Content-Type: text/plain");
            writer.println();
            writer.flush();

            // Write file content
            try (FileInputStream fis = new FileInputStream(file)) {
                byte[] buffer = new byte[1024];
                int bytesRead;
                while ((bytesRead = fis.read(buffer)) != -1) {
                    os.write(buffer, 0, bytesRead);
                }
            }

            writer.println();
            writer.println("--" + boundary + "--");
            writer.flush();
        }

        // Read response
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
        }

        conn.disconnect();
    }
}

:white_check_mark: Why?

  • Handles file uploads through multipart/form-data.
  • Can send both text data and files, which is essential for API endpoints accepting file uploads.