Recommended approach for adding custom tasks & events that work with ib_async

[blair-anson]

With ib_async running asyncio underneath, and me being a noob with asyncio, it leaves me unsure as to what the recommended approach is for extending the asyncio event loop in ib_async to handle custom tasks / events / callbacks for non-IB data feeds.
If I have some code that polls a data source, what do I need to do to run it as an asyncio task that works with ib_async, and when there is new data how do I inform the event loop in ib_async of that new data event?

[mattsta]

A correct, but probably not as useful as you want, answer is "learn how everything works together" by trying different approaches until you see a dozen different ways how everything can break and the three ways everything works.

For the most flexibility and performance, the library should be used in an async-native fashion with all the *Async() methods you manually call using await ib.connectAsync() etc (or it can use async "behind the scenes" transparently with just ib.connect() but then the library starts and owns the event loop itself and it's more difficult to fully interoperate other async services with).

The general pattern with python asyncio is you use asyncio.run() to start the event loop at an entry point coroutine which then has all the further async calls for your entire program.

import asyncio

def main():
    async def start():
        await asyncio.sleep(1)

    asyncio.run(start())

So then we end up with things like this (just an example structure, it's cleaner wrapping all this logic in proper classes for real usage):

import asyncio
from ib_async import IB

def main():
    async def start():
        ib = IB()
        # attach callback event handlers as needed outside of the reconnect loop

        async def connect():
            await ib.connectAsync(host, port, clientId)
            await ib.reqAccountSummaryAsync()

        await connect()

        # Once your program reaches here, you can do anything else you want.
        # You can use websockets, httpx, prompt-toolkit, any of the other ib_async interfaces, etc.
        # Just note your program will _exit_ when this `start()` task returns, so you need _something_
        # either running, waiting on events or network data, or just sleeping to stop from exiting.

    host, port, clientId = ...
    asyncio.run(start(host, port, clientId))

Then it's up to you to figure out obviously what the system does and more importantly how to make the system interactive. Do you need to modify parameters (symbols, features, algos, timeframes, durations) while your system is running? Do you need to enable or disable features when the platform is running? Or do you want everything to just run in a loop forever when running until the program exits?

I've also found it useful to run multiple ib_async programs at once. IBKR allows you to attach many clients to the local IBKR gateway interface as long as each of them use their own clientId (so you could have one client for forwarding market data, one for trading one group of stocks, another client for trading an even different group, etc). Just watch out for one IBKR catch: each clientId controls its own orders, so resting orders on clientId 4 are not editable or visible on clientId 5 (the only exception is clientId 0 is a special "controller id" where it can see everything from other clients, but it can't modify them).

The main problems to watch out for:

• make sure you always use asyncio.sleep() if you are using sleep because asyncio.sleep() will correctly yield to the event loop so other tasks can run while your current task is sleeping. The regular time.sleep() will block the event loop and stop all processing for its duration. Even if it doesn't look like your program is doing work, the IBKR gateway and ib_async library are communicating sometimes dozens of times per second to keep data updated transparently using the event loop.
• if you are doing external network fetches, make sure to use an async network library like httpx or aiohttp or the async methods of websockets because any regular blocking I/O operations could be slow or have long timeouts (then those act like a regular blocking sleep too).
• if you need to wait for user input or some event, make sure you are using an async interface or event handler callbacks and not just while True: (check for reply).

These are good resources to fully read through and experiment with to figure out how python async systems can work together:

• https://docs.python.org/3/library/asyncio.html
• https://realpython.com/python-async-features/
• https://docs.aiohttp.org/en/stable/index.html
• https://websockets.readthedocs.io/en/stable/index.html
• https://www.python-httpx.org/async/
• https://python-prompt-toolkit.readthedocs.io/en/master/pages/advanced_topics/asyncio.html

[blair-anson]

Thank you! This is exactly the type of guidance I was looking for, I appreciate you taking the time to write a comprehensive answer.