What is the best way to send a GET request with Python request headers?

General Discussions
4

Both methods are solid, but if you’re in a situation where you need to stick to Python’s standard library and prefer something a bit more manual, you can also use urllib.request. While it’s not as user-friendly as requests, it’s handy when you’re working with libraries that don’t support requests. Plus, if you’re looking to implement your own logic around how requests are sent, it offers that flexibility.

Here’s the implementation:

import urllib.request

url = "https://example.com/api/content/v1/products/search?page=20&page_size=25&q=&type=image"

headers = {
    "Host": "example.com",
    "Connection": "keep-alive",
    "Search-Version": "v3",
    "Accept": "application/json",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
    "Referer": "https://example.com/search/?q=&type=image&page=20",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-US,en;q=0.9"
}

request = urllib.request.Request(url, headers=headers)
response = urllib.request.urlopen(request)

print(response.read().decode())

This method works well when you need to rely solely on Python’s built-in modules and still want to customize how you handle the python request headers. It gives you more flexibility if you don’t mind a little extra setup.