Re: How to benchmark a HTTP connection with requests module?

[Etienne Robillard]

Hi Dennis,

Here's my code so far:
import os
import requests
import threading
import time

from Queue import Queue
from test_support import unittest

test_queue = Queue()

def worker(ident, url, queue):

    while True:
        start = time.clock()
        r = requests.get(url)
        end = time.clock()
        queue.put((ident, r.status_code, end - start))
        queue.task_done()
    return queue

class HTTPBenchmarkTestCase(unittest.TestCase):

    url = 'http://localhost/benchmark/'
    threads = 5

    def setUp(self):
        self.hold = True


    def test_benchmark_concurrency(self):
        for i in range(self.threads):
            t = threading.Thread(target=worker, args=(i, self.url, 
test_queue))
            t.daemon = True
            t.start()
            obj = test_queue.get()
            print obj
        test_queue.join()


erob@marina:~/src/django-hotsauce-0.9/tests$ pypy ./run.py -C benchmarks
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
(0, 200, 0.12999503499999943)
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
(1, 200, 0.053242928999999606)
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
(0, 200, 0.11221248100000025)
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
(1, 200, 0.12075822500000033)
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76
DEBUG:Starting new HTTP connection (1): localhost
(0, 200, 0.12590276600000028)
.
----------------------------------------------------------------------
Ran 1 test in 0.468s

OK


What do you think? :-)

Etienne


Le 2018-02-16 à 10:58, Dennis Lee Bieber a écrit :
On Fri, 16 Feb 2018 06:22:04 -0500, Etienne Robillard <tkad...@yandex.com>
declaimed the following:

Hi Dennis,

Nice pseudo code! :-)

Is it possible benchmark/measure the latency of a HTTP connection for
each threads by timing the duration of the requests.get method to complete?

        Wrap each .get with calls for start time and end time? (Need to check
if time.time or time.clock is more precise on your hardware, docs recommend
time.clock for benchmarking)

def worker(ID):
        while hold: pass
        st = time.clock()
        r = requests.get(...)
        en = time.clock()
        resultQ.put( (ID, r.status_code, en - st) )
        #return duration as third parameter


        for _ in range(NUMBEROFREQUESTS):
                (ID, code, duration) = resultQ.get()
                requestTasks[ID].join()

--
Etienne Robillard
tkad...@yandex.com
https://www.isotopesoftware.ca/

--
https://mail.python.org/mailman/listinfo/python-list