Python Interactive brokers IB API 非常非常慢

Question

我正在试用新的 Python Interactive Broker API，但我在第一步遇到了一些严重的速度问题...

以下代码(见下文)次

0:00:08.832813 直到数据接收完毕

0:00:36.000785 直到应用程序完全断开...

为什么这么慢？加快速度的最佳方法是什么？

from ibapi import wrapper
from ibapi.client import EClient
from ibapi.utils import iswrapper #just for decorator
from ibapi.common import *
from ibapi.contract import *
import datetime
from datetime import timedelta


class DataApp(wrapper.EWrapper, EClient):
    def __init__(self):
        wrapper.EWrapper.__init__(self)
        EClient.__init__(self, wrapper=self)

    @iswrapper
    def historicalData(self, reqId: TickerId, date: str, open: float, high: float,
                            low: float, close: float, volume: int, barCount: int,
                            WAP: float, hasGaps: int):
        super().historicalData(reqId, date, open, high, low, close, volume,
                                barCount, WAP, hasGaps)
        print("HistoricalData. ", reqId, " Date:", date, "Open:", open,
               "High:", high, "Low:", low, "Close:", close, "Volume:", volume)

    @iswrapper
    def historicalDataEnd(self, reqId: int, start: str, end: str):
        super().historicalDataEnd(reqId, start, end)
        print("HistoricalDataEnd ", reqId, "from", start, "to", end)
        print(datetime.datetime.now()-startime)
        self.done = True # This ends the messages loop - this was not in the example code...

    def get_data(self):        
        self.connect("127.0.0.1", 4002, clientId=10)
        print("serverVersion:%s connectionTime:%s" % (self.serverVersion(),
                                                self.twsConnectionTime()))

        cont = Contract()
        cont.symbol = "ES"
        cont.secType = "FUT"
        cont.currency = "USD"
        cont.exchange = "GLOBEX"
        cont.lastTradeDateOrContractMonth = "201706"
        self.reqHistoricalData(1, cont, datetime.datetime.now().strftime("%Y%m%d %H:%M:%S"),
                               "1800 S", "30 mins", "TRADES", 0, 1, [])
        self.run()        
        self.disconnect()
        print(datetime.datetime.now()-startime)

global starttime
startime = datetime.datetime.now()
DA = DataApp()
DA.get_data()

我还尝试在后台持续运行它，以便只在运行中提交请求

def runMe():
    app.run() # where run() has be removed from the class definition

import threading
thread = threading.Thread(target = runMe)
thread.start()

但它也非常慢。任何建议表示赞赏

Answer 1

我建议您在 ibapi 模块的连接类中修改连接套接字锁。推荐来自github上的heshiming；如果您有权访问私有(private)交互式经纪人存储库，则可以在此处访问讨论 https://github.com/InteractiveBrokers/tws-api/issues/464

我这样做了，它显着提高了性能。

和士铭建议您减少套接字锁对象的超时时间，每次发送或接收消息时都会调用该对象。要修改套接字锁，请转到 ibapi 的站点包文件夹并修改 connection.py 中的连接函数，将“self.socket.settimeout(1)”更改为“self.socket.settimeout(0.01)”。这是我拥有的版本的 connection.py 中的第 48 行。

如果您看不到 heshiming 的帖子，我已将其包含在本文的底部。

备选方案:另一个有趣的解决方案是将 asyncio 用于异步事件循环。我没有这样做，但看起来很有希望。请参阅 Ewald 放在一起的示例 https://github.com/erdewit/tws_async

和士铭评论:

The implementation of Connection /ibapi/connection.py has a Lock object shared in both sendMsg and recvMsg. Since connect, self.socket.settimeout(1) is called, therefore the underlying self.socket.recv(4096) only times out once per second.

Such implementation creates a performance problem. Since the lock is shared, the socket cannot send data while receiving. In the scenario where the message received is less than 4k bytes long, the recvMsg function will wait for 1 second before releasing the lock, making subsequent sendMsg wait. In my experiment, most messages appear to be shorter than 4k bytes. In other words, this imposes a cap of one recvMsg per second.

There are couple strategies to mitigate this. One can reduce the receive buffer to a number much less than 4k, or reduce the socket timeout to something like 0.001 second to make it block less.

Or according to http://stackoverflow.com/questions/1981372/are-parallel-calls-to-send-recv-on-the-same-socket-valid , the socket itself is actually thread-safe. Thus no locks are necessary.

I tried all three strategies. Removing the lock works the best. And reducing the timeout to 0.001 works in similar ways.

I can only vouch for linux/unix platforms, and I haven't tried it on Windows. Would you consider to change the implementation to improve this?