python - 将自定义数据传递给 request_futures 中的异常

Question

我正在运行以下代码来发出异步“获取”请求。 CustomSession 类用于添加对每个请求计时的支持。

如果发生异常或请求运行正常，我希望能够访问附加到 futures 列表中的 test_id 以及要请求的 URL 。换句话说，当请求运行或抛出异常时，我想找到与 session.get 调用关联的 test_id。

from datetime import datetime
from requests_futures.sessions import FuturesSession

class CustomSession(FuturesSession):

    def __init__(self, *args, **kwargs):
        super(CustomSession, self).__init__(*args, **kwargs)
        self.timing = {}

    def request(self, method, url, *args, **kwargs):
        background_callback = kwargs.pop('background_callback', None)
        test_id = kwargs.pop('test_id', None)

        # start counting
        self.timing[test_id] = datetime.now()

        def time_it(sess, resp):
            # here if you want to time the server stuff only
            self.timing[test_id] = datetime.now() - self.timing[test_id]
            if background_callback:
                background_callback(sess, resp)
            # here if you want to include any time in the callback

        return super(CustomSession, self).request(method, url, *args,
                                                  background_callback=time_it,
                                                  **kwargs)


session = CustomSession()

futures = []
for url in ('http://httpbin.org/get?key=val',
            'http://httpasdfasfsadfasdfasdfbin.org/get?key2=val2'):

    futures.append(session.get(url, test_id=1))
for future in futures:
    try:
        r = future.result()
        print(r.status_code)
    except Exception as e:
        print(e)

Answer 1

我为 future 对象的 result() 函数创建了一个装饰器:

def mark_exception(fn, id, url):
    def new_fn(*args, **kwargs):
        try:
            return fn(*args, **kwargs)
        except:
            raise Exception("test id %d with url %s threw exception" % (id, url))
    return new_fn

并将其应用到 CustomSession.request() 函数的末尾，替换原来的 return 语句:

future =  super(CustomSession, self).request(method, url, *args,
                                          background_callback=time_it,
                                          **kwargs)
future.result = mark_exception(future.result, test_id, url)
return future

输出:

200
test id 1 with url http://httpasdfasfsadfasdfasdfbin.org/get?key2=val2 threw exception

我希望这会有所帮助。

编辑:

如果您想获取每个 future 的测试 ID，可以通过以下两种方式实现:

futures = []
for url in ('http://httpbin.org/get?key=val',
            'http://httpasdfasfsadfasdfasdfbin.org/get?key2=val2'):
    tid = 1
    future = session.get(url, test_id=tid)
    # option 1: set test_id as an attrib of the future object
    future.test_id = tid
    # option 2: put test_id and future object in a tuple before appending to the list
    futures.append((tid, future))
for tid, future in futures:
    try:
        r = future.result()
        print("tracked test_id is %d" % tid) #option 2
        print("status for test_id %d is %d" % (future.test_id, r.status_code)) #option 1
    except Exception as e:
        print(e)