Massively Speed Up Requests with HTTPX in Python
If you are looking to speed up your HTTP requests in Python, HTTPX is the library for you. HTTPX is a next-generation HTTP client for Python that is built on top of the popular Requests library but with a focus on performance and compatibility with asynchronous programming.
One of the main reasons why HTTPX is so fast is because it supports HTTP/2 and HTTP/1.1 natively, allowing for faster and more efficient communication with web servers. In addition, HTTPX also has built-in support for connection pooling and timeouts, which can greatly improve the speed and reliability of your requests.
Another key feature of HTTPX is its support for asynchronous programming with Python’s asyncio framework. This allows you to make multiple HTTP requests in parallel, greatly speeding up the overall performance of your application. By taking advantage of HTTPX’s asynchronous capabilities, you can make hundreds or even thousands of requests per second with ease.
Here’s an example of how you can use HTTPX to make a simple HTTP request in Python:
import httpx
async def main():
async with httpx.AsyncClient() as client:
response = await client.get('https://jsonplaceholder.typicode.com/posts/1')
print(response.json())
if __name__ == '__main__':
import asyncio
asyncio.run(main())
As you can see, making an HTTP request with HTTPX is as simple as creating an instance of the AsyncClient class and using its get method to make a request. The response object contains the JSON data returned by the server, which you can then process as needed.
In conclusion, if you are looking to massively speed up your HTTP requests in Python, HTTPX is the library you need. With its support for HTTP/2, connection pooling, timeouts, and asyncio, HTTPX can help you make requests faster and more efficiently than ever before.
books to scrape is huge malware site now
4:30
Hi ,
I tried executing HTTPX POST method, but it is returning 419 instead of 200. the calls are similar to what I used to do with request.Session.. can you please create a view on POST call. what is the trick to make it work ?
Very cool.
What's your take on concurrent.futures?
“`
import concurrent.futures
import time
import requests
start = time.monotonic()
with concurrent.futures.ThreadPoolExecutor() as executor:
executor.map(lambda u: print(requests.get(u).status_code), urls)
end = time.monotonic()
print(f'{end – start:.2f}s')
“`
Comment for the algorithm 🙂 Perfect video lesson
This is INSANE!
very interesting, it is possible to also have the url that we are testing with the return 200
wow, it works amazingly. Thanks for sharing this
Big ups for all the stuff you do man! All videos are very clear, concise and on point. Keep it up
Can you solve rubik cube with python if yes then please make video
Ich habe mit einem 2 Jahre alten Video von dir gestartet – und hatte gehofft dass du immer noch postest … klasse das du es tust!
I’m happy you still posting! Thanks for what you do.
You're awesome dude, loving your new pace. Keep up the great work.
Is this efficient on getting access to different ip camera addresses
Thanks a lot for the video! Does "httpx" work with webdriver from selenium?
Great video!
This was a good one, really interesting.
Have you covered python queue.Queue() with threading?
Python 3.11 + httpx = 🔥🔥🔥🔥