Fundamentals of TWS API - Interactive Brokers Part 2

Generally Interactive Brokers releases the installer for both the TWS API and IB Gateway simultaneously. For this blog I will use TWS API ( Version: API 9.79 Release Date: Feb 05 2020 ) and not IB Gateway. Check out my previous blog post for detailed installation guide of TWS API on Win 10 Machine that uses the Anaconda Distribution for Python.

 

IB also has a host of API's. A recent one that they added is a REST based API which they refer to as Client Portal WebAPI . However this blog post deals solely with the TWS API

                            Table of Contents :

• EClient and EWrapper Class
• Establishing an API Connectivity
• Choice of IDE
• Threads and Messages
• Contracts in TWS API
• Orders in TWS API
• Live Market Data & Historical Data in TWS API
• Codebase on GitHub

The general operation of the TWS API application is:

1. Establish connection with IB server.
2. Request information from IB server or execute an action
3. If a response is provided by the IB server, then receive the response and process the response
4. Repeat steps 2 & 3 until all the required information has been received and all the operations have been executed.
5. Terminate the Connection with IB server.

From the developer's perspective, this is the communication process:

1. Connect to TWS API by creating a socket
2. Send requests (messages) to TWS API through the socket
3. Receive responses (messages) from the TWS API through the socket.

A physical electrical socket in our house is an entry point for a power cable. A software socket is also an entry point, but instead of accepting a power cable, it accepts a network connection from another computer. IP addresses, ports, and TCP/IP are the related technologies that made sockets possible. To give a simplistic explanation, a socket is the combination of an IP addrress and a port. So when an application sends a socket based request to a specific IP address and port number, the receiving application can reply with a response. Communicating with TWS uses this approach.

If a Python application sends a message to the IP address of a system running TWS API and sets the port number to 7497 (for paper trading in IB gateway, port number is set to 4002), then TWS will send the message to one of IB' servers which will send an appropriate response to TWS API that is sent back to the Python application. This process is explained in the diagram below:

Each Arrow in the above process represents a step in the socket communication process: 

1. The Application (Client) send a message(request) to the IP address of the system running TWS API.
2. The TWS API receives the message, authenticates it and sends another message to the IB server
3. The IB server receives the message , processes it and sends a message to the TWS API
4. TWS API forwards the message (response) to the application

 

As developers we are only concerned with the Step 1 and Step 4. So we write an application that send requests to the TWS API and receives responses.

EClient and EWrapper Class

In Win 10 path C:\TWS API if you open the TWS API directory, you will find a folder named "source". This contains a sub-directory named "pythonclient". This has two sub-directories named "ibapi" and "tests". The "ibapi" folder contains source files for the Python API. Among the programs are client.py  and wrapper.py 

 

And when you run a Python application, you will have to set the sys.path or PYTHONPATH variable to the location of ibapi. If you have configured the path correctly, you will be able to import  classes using the following import statements:

from ibapi.client import EClient
from ibapi.wrapper import EWrapper

Schematically any Python program that we will write to use the IB API will have two main classes and it is illustrated below:

The EClient class handles all outgoing requests while the EWrapper class handles incoming messages.

The wrapper.py module defines the EWrapper class which contains more than 100 methods. And if you peruse them , you will find that most of them merely write a message to the log.

 

To understand how these functions can be used, consider the following code, which prints the argument received by the currentTime method:

def currentTime(self, curr_time):
    t = datetime.fromtimestamp(curr_time)
    print('Current time: {}'.format(t))

Now if the client calls reqCurrentTime and the response is stored in the message queue, the decoder will access the clients wrappper and call currentTime function. These response methods of the EWrapper class are called callbacks. 

As application developers, our work is to create a custom sub class of EWrapper and override the callbacks that we are interested in. As an example, if the client application calls reqOpenOrders, the custom wrapper should provide code in the openOrder call back. To put it in simple language - As its name goes, EWrapperclass  acts like a wrapper for incoming messages and in most cases a function from it will need to be overwritten in your Python Script to redirect the output to where you want it to go.

When our application program gets too big, it might get difficult to distinguish EWrapper callback methods from other methods. So I find it helpful to precede callback methods with the @iswraper annotation. This is declared in utils.py  program and for Win 10 users, you can find it in  C:\TWS API\source\pythonclient\ibapi . Import this in the application with the following import statement:

from ibapi.utils import iswrapper

As an example in the below code, the historicalData is a callback function to the reqHistoricalData function. Now as you begin to spend more time with the TWS API, this becomes very evident. But what about error function ? Is it a callback or is userdefined ? What if I create another user defined function def is_error() ? So to ensure that such confusion is done away with, I use the @iswrapper annotation to the error function. When a TWS API error occurs, the callback provides the ID of the request that produced the error, the error code and the error message. Only EWrapper callback methods have the @iswraper annotation.

@iswrapper
def historicalData(self, reqId, bar):
	''' Callback to reqHistoricalData '''
	print('\nOpen: {}, High: {}, Low: {}, Close: {}'.format(bar.open, bar.high, bar.low, bar.close))
@iswrapper
def error(self, reqId, code, msg):
	print('Error {}: {}'.format(code, msg))

Establishing an API Connectivity

Try out the following Code to check the Connectivity: 

# Establish a Connection with IB APi
from ibapi.client import EClient
from ibapi.wrapper import EWrapper  
import time

class TestAPI(EWrapper, EClient):
     def __init__(self):
         EClient.__init__(self, self) 

client = TestAPI()
client.connect('127.0.0.1', 7497, 123)
client.run()

First we have imported EClient and EWrapper. Next we have created a class and both these scripts are passed through it. Then we instantiate the class using the client variable and then call the client.connect()  command and specify the parameters to create a connection. 

client.connect(host, port, clientId)

The above function  establishes the connection to TWS. It accepts three parameters: a string identifying the IP address of the system running TWS, a port number, and a number that identifies which client is connecting. TWS supports up to 32 simultaneous client connections

In the Python Script above,  127.0. 0.1 is the loopback Internet protocol (IP) address also referred to as the localhost.  The address is used to establish an IP connection to the same machine or computer being used by the end-user.

The client.run() command reads message from the queue and executes everything. It executes in a loop that repeats as long as the client is connected. For this reason, applications frequently execute the clients run function in a separate thread using code like shown below:

thread = Thread(target=self.run)
thread.start()

The above connection can be disconnected by using the app.disconnect()  command. 

In the event that TWS is not active and you are using the TWS API (for this tutorial we shall be using the TWS  API) , then very likely you will receive a message like the one shown below: 

In case the TWS is active and all is fine, then you will receive a message like the one shown below: 

In case you have not received a message like the one above, its could be becos the python script ended before the connection was established. If so then use a sleep timer at the end of the code. Also change the clientid to something a little more unique and try again.

Choice of IDE

After running the above Python script, you would have noticed that the IPython terminal of the Spyder IDE (that comes with anaconda distribution) continues to be active. That’s becos the connection is still open and we have not run the app.disconnect()  command.

 

With Interactive Brokers, a connection is first made to the  IB Client software which acts as an intermediary to the IB servers. This requires an Open and Constant  connection and that is one of the reason why we use separate threads to handle incoming messages (EWrapper) and outgoing messages (EClient).

 

This presents a challenge when we are using interactive IDE like Spyder or Jupyter Notebooks. The EClient functions (outgoing calls) tend to work fine but EWrapper functions (incoming data) present issues due to the lack of an open connection. There is a workaround for this in case you want to use interactive IDE like Spyder - Use IB-insync library which uses the asyncio library to provide an asynchronous single thread to interact with the API. In case you want to post queries or get clarifications, check out the discussions in the Group for the IB-insync Python framework

 

Some other popular IDE/Code Editor to consider include VSCode, Sublime Text, PyCharm, Atom etc., I prefer VSCode and you can check out the VSCode tutorials in case you want to experiment with it.  Check them and use the one that suits you.

Threads and Messages

We know that applications communicate with IB through sockets. An application writes data to the socket and reads data from the socket. Packages of received data are called messages. And applications store messages in a first-in first-out (FIFO) data structure called message queue.

 

Applications cannot control when new data will be available. So they create a separate execution thread to read from the socket and write to the message queue. For this reason most applications perform their processing in two threads:

• Client Thread - Send data to socket,  Read messages from the queue.
• Reader Thread - Read data from Socket , Write messages to the queue.

 

Generally these threads work in the following manner:

1. The application starts the client  thread, which send requests to the TWS API.
2. The client thread goes to sleep while TWS API transfers data to its socket.
3. When the client thread is asleep, the reader thread reads data from the socket and writes a message to the message queue.
4. As messages enter the queue, a decoder processes each of them and invokes corresponding call back function of the clients wrapper.

 

As a developer, we will probably never write any Python code that interacts directly with the reader thread, message queue or the decoder. However as you develop applications,  you will find it helpful to understand how a clients requests results in wrapper functions being invoked.

Contracts in TWS API

Essentially a contract is a Financial Instrument. Anything that is available for trading in IB like Stocks, futures and options, bonds, IB special products like Contract For Difference(CFD), etc., is called a Contract.  An Order is used to buy and sell a Contract.

 

In TWS API , the contracts are represented by instances of the Contract structure as described in the contract.py file in the directory C:\TWS API\source\pythonclient\ibapi

 

If you open the TWS terminal and right click on a symbol in the market watch and check Financial Instrument Info -> Description , then you will get all details on that symbol. A typical Python code will look like this: 

    #Create contract object
    contract = Contract()
    contract.symbol = 'INFY'
    contract.secType = 'STK'
    contract.exchange = 'SMART'
    contract.currency = 'INR'
    contract.primaryExchange = "NSE"

The first four fields are the fundamental fields. The field secType with value STK refers to stock and the currency is set to Indian Rupees (INR). IB provides a service called SMART routing wherein the router searches exchanges and directs orders to the best exchange for a given contract. However since a stock like INFY is traded both in NSE and BSE stock exchange, it is preferable to mention primaryExchange to ensure that there is no ambiguity (who knows, IB might start trading in BSE also in the future!!!. )

 

In case you want more details about any given financial instrument programmatically, you can use the reqContractDetails  method. After TWS API receives the request, the python application will receive the response through the contractDetails callback function. If you can't remember a contract's symbol, you can request possible symbols by calling the client's reqMatchingSymbols method. After an application calls reqMatchingSymbols it can obtain the server's response through the symbolSamples callback function. The point to note is that every function has a necessary callback (in some cases more than one callback) and in the corresponding callback, we can retrieve the information we seek.

 

Lets check out a  python program to read a Contract:

# Imports for the Program
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.utils import iswrapper

import time
import threading

class TestContract(EWrapper, EClient):
    ''' Serves as the Client and the Wrapper '''

    def __init__(self, addr, port, client_id):
        EWrapper.__init__(self)
        EClient.__init__(self, self)

        # Connect to TWS API
        self.connect(addr, port, client_id)

        # Launch the client thread
        thread = threading.Thread(target=self.run)
        thread.start()
        
    @iswrapper
    def symbolSamples(self, reqId, contractDescriptions):
        # Print the symbols in the returned results
        print ('...')
        print('Number of Contract descriptions in List: {}'.format(len(contractDescriptions)))
        for contractDescription in contractDescriptions:
            print('Symbol: {}'.format(contractDescription.contract.symbol))
            print('Contract ID: {}'.format(contractDescription.contract.conId))
            print('Contract Security Type: {}'.format(contractDescription.contract.secType))
            print('Contract Primary Exchange: {}'.format(contractDescription.contract.primaryExchange))
            print('Contract Currency: {}'.format(contractDescription.contract.currency))

        # Select the first symbol
        self.symbol = contractDescriptions[0].contract.symbol

    @iswrapper
    def contractDetails(self, reqId, details):
        print('\nDetails of first symbol follows:')
        print('Market Name: {}'.format(details.marketName))
        print('Long Name: {}'.format(details.longName))
        print('Industry Classification: {}'.format(details.industry))
        print('Industry Category: {}'.format(details.category))
        print('Subcategory: {}'.format(details.subcategory))
        print('Contract ID: {}'.format(details.contract.conId))
        print('Time Zone for the Product: {}'.format(details.timeZoneId))
        print('Trading Hours of the Product: {}'.format(details.tradingHours))
        print('\nFull Contract Details, Unformatted:', reqId, " ", details)
        
    @iswrapper
    def contractDetailsEnd(self, reqId):
        print ('...The End of Contract Details...for reqId:{} '.format(reqId))

    def error(self, reqId, code, msg):
        print('Error Code: {}'.format(code))
        print('Error Message: {}'.format(msg))

def main():
    # Create the client and connect to TWS API
    client = TestContract('127.0.0.1', 7497, 700)
    time.sleep(3)

    # Request descriptions of contract starting with "goog"
    client.reqMatchingSymbols(0, 'goog')
    time.sleep(3)

    # Request details for the Stock
    contract = Contract ()
    contract.symbol = client.symbol
    contract.secType = "STK"
    contract.exchange = "SMART"
    contract.currency = "USD"

    client.reqContractDetails(1, contract)
    
    time.sleep(3)
    client.disconnect()
   
if __name__ == "__main__":
    main()

In the above program we first create a TestContract instance and then call its  reqMatchingSymbols function to search for all contracts starting with with "goog". The application accesses the IB servers response using the callback symbolSamples . This callback is used to print all the symbols and store the first symbol. The first symbol is used for the creation of a contract for a stock. Then viz a request using the  reqContractDetails  method, the response in ContractDetails callback is used to obtain information about the stock. This callback prints the Contract's marketName, longName etc., as is mentioned in the print statements. In the last print statement, the full details are then printed to get a feel of the information available.

This is the output that we receive when we run the program:

Now replace "goog"  with "TCS"  in the above program and check out the details of the symbol. !!! That's why it is important to explicitly set the contract.primaryExchange field as NSE  in case you are operating in the Indian markets.

Orders in TWS API

In TWS API , the orders are represented by instances of the Order structure as described in the order.py file in the directory C:\TWS API\source\pythonclient\ibapi . To execute Orders programmatically, this is typically how a code would look:

order = Order()
order.action = "BUY"
order.totalQuantity = 100
order.orderType = "MKT"

In the above code,   "MKT" stands for market order, which is the simplest of all. 

 

Besides the plain vanilla MKT orders, there is another variant which is of interest to me. Particularly becos many of my strategies use Stop Loss Orders (Stop Order) along with Target Price which I code manually . This is the Order Type MIT (Market if Touched). Here we also have to set the auxillary price which is essentially the Trigger Price.

order = Order()
	
# Set the order's type to Market if Touched (MIT)
order.orderType = "MIT"
	
# Set the trigger price to 90
order.auxPrice = 90

In the above code, IB will wait until the market price touches 90 before submitting the market order

Although Market Orders are the easiest to place, particularly when we want a fill for sure, it is always problematic for an illiquid contract where the Bid-Ask spreads are high. That's where the Limit Order becomes helpful. As an example, if we place a Buy order at a Limit Price of 100, then IB will execute the order at any price Less than or equal to 100. We need to set order.orderType = "LMT" to place a limit order.

 

Another variant of the limit order is Limit if Touched (LIT)

order = Order()
	
# Set the order's type to Limit if Touched (LIT)
order.orderType = "LIT"
	
# Set the limit price to 90
order.lmtPrice = 90
	
# Set the trigger price to 110
order.auxPrice = 110

In the above code, IB won't submit the limit order until the market price reaches the trigger price given by order.auxPrice which is 110.

Lets see how a Stop Order (Stop Loss Order) is placed.

order = Order()
	
# Set the order's type to Stop Order (STP)
order.orderType = "STP"
	
# Set the trigger price to 90
order.auxPrice = 90

Imagine a case where we have made a BUY for a security and placed a STP with trigger price 90.  In this case, if the market price of the security touches 90, then the Stop Order is triggered at Market Price. Likewise a variant also exists for triggering a Stop Order at limit price, although I always use a Stop Order at Market Price (Better Safe than Sorry !!!)

Just as we have Stop orders, we also have Trailing Stop Orders.

order = Order()
	
# Set the order's type to Trailing.
order.orderType = "TRAIL"
	
# Set the trailing Stop Price to 80
order.trailStopPrice = 80
	
# Set the trailing percentage to 5
order.trailingPercent = 5

Some variant's are order.orderType = "TRAIL LIT"  and order.orderType = "TRAIL MIT" where MIT represents Market if Touched  and LIT represents Market if Touched. There are also a host of other order types in IB and click here for details on the Basic Orders in TWS .

Once we have the Order ready, the next step is to place the Order. For this we use the placeOrder method of the EClient Class.

placeOrder(orderId, contract, order)

In the above code, orderId parameter serves as a unique identifier for the order, contract  identifies the Contract to be traded and order is the Order that performs the trade.

Once the IB server receives the order and executes it, it will provide the client with status information by invoking the callback function openOrder and another callback orderStatus

openOrder (orderId, contract, order, orderState)

In the above code, orderState argument contains the status about the order and the account that placed it. And this argument is an instance of the class OrderState Class  defined in the order_state.py file. For a developer, the initial margin ( the equity that must be present when the order is placed) and the Maintenance margin( the equity that must be present during each trading day following the order) can be accessed through the OrderState Class.

The second callback invoked after order placement is orderStatus

orderStatus(orderId: OrderId, status: str, filled: float, remaining: float, avgFillPrice: float, permId: int, parentId: int, lastFillPrice: float, clientId: int, whyHeld: str, mktCapPrice: float):

The status  argument tells the application what has happened to the order. This can take one of the values mentioned below:

• PendingSubmit: Order transmitted but not accepted
• PendingCancel: Order cancellation request sent but not confirmed
• PreSubmitted: Simulated order accepted but not submitted
• Submitted: Order accepted by IB
• ApiCanceled: Order cancellation requested by API client but not confirmed
• Canceled:Order canceled
• Filled: Order completely filled
• Inactive: Order received but not active because of rejection or cancellation

 

If the order is filled, the filled  argument identifies the number of filled positions. If the order is only partially filled, then the remaining argument gives the number of positions that were not filled. The avgFillPrice argument gives the average price at which the order was placed. permId    is used by TWS and orderId is the Id of the order whose status is being provided. parentId is the Id of the orders parent and clientId is the Id of the API client that submitted the order.

 

Generally in TWS API, there is a one-to-one relationship between a request function and its corresponding callback function. However in the case of Orders, the relationship is many-to-many: many request functions have multiple callback's and some callback's are invoked in response to multiple request functions.

 

Request Functions	Corresponding Callback's	Notes
reqOpenOrders()     reqAllOpenOrders()     reqAutoOpenOrders(True)	openOrder(orderId, contract, order, orderState)     orderStatus(orderId, status, filled, remaining, avgFillPrice, permId, parentId, lastFillPrice, clientId, whyHeld, mktCapPrice)	If an application calls reqOpenOrders , it will only receive data about the orders submitted by the client that called reqOpenOrders. However if an application calls reqAllOpenOrders, it will receive the state of all open orders for each client transmitting orders to the target TWS.     Active orders will be delivered via The openOrder callback and The orderStatus callback. When all orders have been sent to the client application you will receive a IBApi.EWrapper.openOrderEnd event:   reqAutoOpenOrders requests status of future orders placed by TWS. This is only available for the master client (Client 0). If its argument is set to true, future orders will be assigned IDs associated with the client calling reqAutoOpenOrders
reqExecutions(reqId, ExecutionFilter())	execDetails(reqId, contract, execution)   execDetails(reqId, contract, execution):	After an application calls reqExecutions , two callback functions are invoked in response: execDetails  and commissionReport . execDetails contains information about the executed order and commissionReport identifies the order's commission, profit, loss, and yield.   The reqExecutions function requests information about orders that were successfully executed since midnight. This accepts an ExecutionFilter that identifies which executed orders should be identified.   execDetails  is invoked once for every order executed in the last 24 hours that meet the criteria set by the ExecutionFilter

The documentation of TWS API provides more specific details to the above request functions and their corresponding callback's.  Now lets dwell straight into some more code for placing orders and cancelling the placed order.

# Imports for the Program
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.order import Order
from ibapi.utils import iswrapper

import time
import threading
import sys

class TestLimitOrder(EWrapper, EClient):
    ''' Serves as the Client and the Wrapper '''

    def __init__(self, addr, port, client_id):
        EWrapper.__init__(self)
        EClient.__init__(self, self)
        self.orderId = None

        # Connect to TWS API
        self.connect(addr, port, client_id)

        # Launch the client thread
        thread = threading.Thread(target=self.run)
        thread.start()

    @iswrapper
    def nextValidId(self, orderId):
        ''' Provides the next order ID '''
        self.orderId = orderId
        print('\nThe order id is: {}'.format(orderId))

    @iswrapper
    def openOrder(self, orderId, contract, order, state):
        ''' Callback for the submitted order '''
        print('Order Status: {}'.format(state.status))
        print('The accounts current initail margin: {}'.format(state.initMarginBefore))
        print('The accounts current maintanence margin: {}'.format(state.maintMarginBefore))
        print('Comission Charged: {}'.format(state.commission))
        print('Completed Time: {}'.format(state.completedTime))
        print('Warning Text: {}'.format(state.warningText))
        
    @iswrapper
    def orderStatus(self, orderId, status, filled, remaining, avgFillPrice, permId, parentId, lastFillPrice, clientId, whyHeld, mktCapPrice):
        ''' Check the status of the submitted order '''
        print('Status of Order: {}'.format(status))
        print('No. of Filled Positions: {}'.format(filled))
        print('No. of remaining positions: {}'.format(remaining))
        print('Why Held: {}'.format(whyHeld)) # value 'locate'used when trying to locate shares for short sell
        print('Average Fill Price: {}'.format(avgFillPrice))
        print('permId: {}'.format(permId))

    @iswrapper
    def position(self, account, contract, pos, avgCost):
        ''' Provides the portfolios Open Positions'''
        print('No. of Positions held in {} : {} '.format(contract.symbol, pos))
        print('The average cost of the position: {}'.format(avgCost))

    @iswrapper
    def accountSummary(self, reqId, account, tag, value, currency):
        '''Read Information about the Account'''
        print('Account {} : {} = {}'.format(account, tag, value))
        print('The currency on which the Value is expressed: {}'.format(currency))

    @iswrapper    
    def error(self, reqId, code, msg):
        print('Error Code: {}'.format(code))
        print('Error Message: {}'.format(msg))        

def main():
    # Create the client and Connect to TWS API
    client = TestLimitOrder('127.0.0.1', 7497, 7)
    time.sleep(3)
    client.orderId = None

    # Define a Contract
    contract = Contract()
    contract.symbol = 'SBIN'
    contract.secType = 'STK'
    contract.exchange = 'SMART'
    contract.currency = 'INR'
    contract.primaryExchange = "NSE"
    
    # Define the Limit Order for BUY/SELL
    order = Order()
    order.action = 'BUY'
    order.totalQuantity = 42
    order.orderType = 'LMT'
    order.lmtPrice = '10.2' # Place the limit order far off from the actaul market price. For Test
    #order.transmit = False # when set to false, the order wont actually be executed when the program is run
    
    # Request from TWS, the next valid ID for the order
    client.reqIds(1)
    time.sleep(3)

    # Place the order
    if client.orderId:
        client.placeOrder(client.orderId, contract, order)
        time.sleep(3)
    else:
        print ('Order ID not received. Terminating Application')
        sys.exit()

    # Obtain information about open positions
    client.reqPositions()
    time.sleep(3)

    # Obtain information about Account
    client.reqAccountSummary(7, 'All', 'AccountType')
    time.sleep(3)

    # Cancel Order
    print ('Cancelling the Order')
    client.cancelOrder(client.orderId) # Cancel the above order which has been placed

    # Disconnect from TWS
    time.sleep(3)
    client.disconnect()
    
if __name__ == "__main__":
    main()

In the above program, the main  function calls reqIds to obtain a valid ID for the next order. When the ID becomes available, main calls placeOrder with the Contract structure, Order structure, and ID. As a result of the placeOrder call, two callbacks are invoked. The first is openOrder, and the second is  orderStatus. After submitting the order, main calls reqPositions to request information related to the account's open positions and reqAccountSummary to request information related to the account. The application accesses the requested data through the position callback and accountSummary callbacks. Lastly the order is cancelled with  cancelOrder before disconnecting from TWS.

Now let us take a look at a program to implement a Bracket Order, which is essentially an order that is placed simultaneously with  a Stop Loss and/or a Target (take profit). When we place such an order, if the Target is hit, then the entire order stands "executed" , in the sense that the Stop Loss is automatically cancelled.

 

Bracket Orders are designed to help limit your loss and lock in a profit by "bracketing" an order with two opposite-side orders. A BUY order is bracketed by a high-side sell limit order and a low-side sell stop order. A SELL order is bracketed by a high-side buy stop order and a low side buy limit order. Note how bracket orders make use of the TWS API's Attaching Orders mechanism.

 

Advanced orders such as Bracket Orders or Hedging involve attaching child orders to a parent. This can be easily done via the IBApi.Order.ParentId attribute by assigning a child order's IBApi.Order.ParentId to an existing order's IBApi.Order.OrderId. When an order is attached to another, the system will keep the child order 'on hold' until its parent fills. Once the parent order is completely filled, its children will automatically become active.

 

One key thing to keep in mind is to handle the order transmission accurately. Since a Bracket consists of three orders, there is always a risk that at least one of the orders gets filled before the entire bracket is sent. To avoid it, make use of the IBApi.Order.Transmit flag. When this flag is set to 'False', the TWS will receive the order but it will not send (transmit) it to the servers. In the example below, the first (parent) and second (takeProfit) orders will be send to the TWS but not transmitted to the servers. When the last child order (stopLoss) is sent however and given that its IBApi.Order.Transmit flag is set to true, the TWS will interpret this as a signal to transmit not only its parent order but also the rest of siblings, removing the risks of an accidental execution.

# Imports for the Program
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.order import Order
from ibapi.utils import iswrapper

import time
import threading
import sys

class TestBracketOrder(EWrapper, EClient):
    ''' Serves as the Client and the Wrapper '''

    def __init__(self, addr, port, client_id):
        EWrapper.__init__(self)
        EClient.__init__(self, self)
        self.orderId = None

        # Connect to TWS API
        self.connect(addr, port, client_id)

        # Launch the client thread
        thread = threading.Thread(target=self.run)
        thread.start()

    @iswrapper
    def nextValidId(self, orderId):
        ''' Provides the next order ID '''
        self.orderId = orderId
        print('\nThe order id is: {}'.format(orderId))

    @iswrapper
    def openOrder(self, orderId, contract, order, state):
        ''' Callback for the submitted order '''
        print('Order Status: {}'.format(state.status))
        print('The accounts current initail margin: {}'.format(state.initMarginBefore))
        print('The accounts current maintanence margin: {}'.format(state.maintMarginBefore))
        print('Comission Charged: {}'.format(state.commission))
        print('Completed Time: {}'.format(state.completedTime))
        print('Warning Text: {}'.format(state.warningText))
        
    @iswrapper
    def orderStatus(self, orderId, status, filled, remaining, avgFillPrice, permId, parentId, lastFillPrice, clientId, whyHeld, mktCapPrice):
        ''' Check the status of the submitted order '''
        print('Status of Order: {}'.format(status))
        print('No. of Filled Positions: {}'.format(filled))
        print('No. of remaining positions: {}'.format(remaining))
        print('Why Held: {}'.format(whyHeld)) # value 'locate'used when trying to locate shares for short sell
        print('Average Fill Price: {}'.format(avgFillPrice))
        print('permId: {}'.format(permId))

    @iswrapper
    def position(self, account, contract, pos, avgCost):
        ''' Provides the portfolios Open Positions'''
        print('No. of Positions held in {} : {} '.format(contract.symbol, pos))
        print('The average cost of the position: {}'.format(avgCost))

    @iswrapper
    def accountSummary(self, reqId, account, tag, value, currency):
        '''Read Information about the Account'''
        print('Account {} : {} = {}'.format(account, tag, value))
        print('The currency on which the Value is expressed: {}'.format(currency))

    @iswrapper    
    def error(self, reqId, code, msg):
        print('Error Code: {}'.format(code))
        print('Error Message: {}'.format(msg))

    @iswrapper
    def execDetails(self, req_id, contract, execution):
        print('Order Executed: ', req_id, contract.symbol, contract.secType, contract.currency, execution.execId, execution.orderId, execution.shares, execution.lastLiquidity)
    
    @staticmethod
    def BracketOrder(parentOrderId:int, action:str, quantity:float, limitPrice:float, takeProfitLimitPrice:float, stopLossPrice:float):
        print('Bracket Order parentOrderId: {}'.format(parentOrderId))
        #This will be our main or "parent" order
        parent = Order()
        parent.orderId = parentOrderId
        parent.action = action
        parent.orderType = "LMT"
        parent.totalQuantity = quantity
        parent.lmtPrice = limitPrice
        #The parent and children orders will need this attribute set to False to prevent accidental executions.
        #The LAST CHILD will have it set to True, 
        parent.transmit = False

        takeProfit = Order()
        takeProfit.orderId = parent.orderId + 1
        takeProfit.action = "SELL" if action == "BUY" else "BUY"
        takeProfit.orderType = "LMT"
        takeProfit.totalQuantity = quantity
        takeProfit.lmtPrice = takeProfitLimitPrice
        takeProfit.parentId = parentOrderId
        takeProfit.transmit = False

        stopLoss = Order()
        stopLoss.orderId = parent.orderId + 2
        stopLoss.action = "SELL" if action == "BUY" else "BUY"
        stopLoss.orderType = "STP"
        #Stop trigger price
        stopLoss.auxPrice = stopLossPrice
        stopLoss.totalQuantity = quantity
        stopLoss.parentId = parentOrderId
        #In this case, the low side order will be the last child being sent. Therefore, it needs to set this attribute to True 
        #to activate all its predecessors
        stopLoss.transmit = True

        bracketOrder = [parent, takeProfit, stopLoss]
        return bracketOrder

def main():
    # Create the client and Connect to TWS API
    client = TestBracketOrder('127.0.0.1', 7497, 7)
    time.sleep(3)
    client.orderId = None

    # Define a Contract
    contract = Contract()
    contract.symbol = 'INFY'
    contract.secType = 'STK'
    contract.exchange = 'SMART'
    contract.currency = 'INR'
    contract.primaryExchange = "NSE"

    # Request from TWS, the next valid ID for the order
    client.reqIds(1)
    time.sleep(3)

    # Place a Bracket Order
    if client.orderId:
        bracket = TestBracketOrder.BracketOrder(client.orderId, "BUY", 100, 30, 40, 20)
        for o in bracket:
            client.placeOrder(o.orderId, contract, o)
        time.sleep(3)
    else:
        print ('Order ID not received. Terminating Application')
        sys.exit()
    
    # Obtain information about open positions
    client.reqPositions()
    time.sleep(3)

    # Disconnect from TWS
    client.disconnect()
    
if __name__ == "__main__":
    main()
	

Check out the Orders windows on TWS application and you should see a bracket order placed, with its corresponding profit taker and stop loss.

Live Market Data & Historical Data in TWS API

There are different kinds of Data that can be accessed viz the TWS API. That includes Financial News,  Data required for Technical Analysis and Fundamental Analysis. Of interest to us is Live Market Data and Historical Data.

 

There are presently 4 functions to receive Market Data. The difference between these four actually lay in Delay between Updates, the number of simultaneous requests that each function can make and the Market Depth provided (Level 1 Data / Level 2 Data)

Function	Data Details
reqTickByTickData	Tick-by-tick data corresponding to the data shown in the TWS Time & Sales Window
reqMktData	Requests real time market data
reqRealTimeBars	Real time and historical data functionality is combined through the IBApi.EClient.reqRealTimeBars request.   No more than 60 API queries in more than 600 seconds
reqMarketDepth	Requests the contract's market depth (order book). This request must be direct-routed to an exchange and not smart-routed

In case you are an Indian National like me , an explicit market data subscription for NSE is not required as it comes by default for all those who open account with IB. . This is subject to maintaining a minimum account balance of USD 500 or equivalent.  This is as per the information available at the time of writing this blog post.  Detailed information for Indian Individuals is available on this link.

Level II data costs more than Level I data. Level II market data contains all of the prices and sizes available for a given security, not just the NBBO prices. Because all of a security's orders are provided, this type of data is referred to as order book data, deep book data or market depth. Remember that Level II entries represent submitted orders, not executed orders. Savvy traders may submit orders and then adjust or cancel them to influence other traders

 

Of primary interest to us is the reqMktData function which doesn't provide data as quickly as reqTickByTickData. The delay between updates depends on the type of contract and in Stocks it is generally about 250 milli seconds. Despite the small time delay, which I find acceptable to retail traders, the  reqMktData function has multiple advantages over reqTickByTickData . Firstly you can request data for more contracts at a time and you can ask for more types of data with each request. The maximum number of requests is limited by the market data lines and the 50 messages per second limit.

reqMktData(reqid, contract, genericTickList, snapshot, regulatory, mktDataOptions)
	

The The TWS API provides six different callbacks for reqMktData. Each provides a different type of data, as explained in the below table:

tickPrice(reqId, field, price, attribs)	Handles all price related ticks such as a security's ask price and bid price. Every tickPrice callback is followed by a tickSize. A tickPrice value of -1 or 0 followed by a tickSize of 0 indicates there is no data for this field currently available, whereas a tickPrice with a positive tickSize indicates an active quote of 0 (typically for a combo contract).
tickSize(reqId, field, size)	Handles all size-related ticks such as a security's ask sizes, bid sizes, and volumes.
tickString(reqId, field, value)	This provides tick data that can be expressed as strings Every tickPrice is followed by a tickSize. There are also independent tickSize callbacks anytime the tickSize changes, and so there will be duplicate tickSize messages following a tickPrice.
tickOptionComputation(reqId, field, impliedVolatility, delta, optPrice, pvDividend, gamma, vega, theta, undPrice)	Provides data and statistics related to options
tickEFP(reqId, field, basisPoints, formattedBasisPoints, impliedFuture, holdDays, futureLastTradeDate, dividendImpact, dividendsToLastTradeDate)	Provides data for exhange-for-physical contracts
tickGeneric(reqId, field, value)	Provides generic tick data requested in reqMktData

In all the above callback's , the "field" parameter is the essentially the "Tick Type" . Check this link for The complete list of Available Tick Types . 

Lets see some quick code to see how this is used in practice: 

def tickPrice(self, reqId, field, price, attribs):
    if field == 1:
        self.bid_price = price
    elif field == 2:
        self.ask_price = price
    elif field == 4: 
        self.last_price = price # Last price at which the contract traded
    elif field == 6:
        self.high_price = price
    elif filed == 7:
        self.low_price = price
    elif  field == 9:
        self.close_price = price # The last available closing price for the previous day
	

Another important point is that not all values printed in the callback tickPrice are prices and not all values printed in callback tickSize are sizes. 

The reqRealTimeBars Function is where we get the OHLC information which is generally used by traders of the Technical Analysis school of thought. This request will return a single bar in real time every five seconds and the limitation is that no more than 60 *new* requests for real time bars can be made in 10 minutes. This is how a typical call is made:

reqRealTimeBars(reqId, contract, barSize, whatToShow, useRTH, realTimeBarsOptions) 	

The The barSize parameter in the above function is always set to 5 (seconds) . The whatToShow parameter can be set to 'BID',  'ASK' or 'MIDPOINT'. It can also be set to 'TRADES' to get information about recent transactions. Caution when using the useRTH. Setting the value of useRTH to True tells TWS to access data inside and outside of regular trading hours.  Setting the value to False tells TWS to restrict data to that generated within regular trading hours. An example follows below:

reqRealTimeBars(id, contract, 5, 'ASK', True, [])	

The corresponding callback function for the above request is shown below: 

realtimeBar(reqId: TickerId, time:int, open: float, high: float, low: float, close: float, volume: int, wap: float, count: int)

reqId	the request's identifier
Date	the bar's date and time (Epoch/Unix time)
Open	the bar's open point
High	the bar's high point
Low	the bar's low point
Close	the bar's closing point
Volume	the bar's traded volume (only returned for TRADES data)
WAP	the bar's Weighted Average Price rounded to minimum increment (only available for TRADES).
Count	the number of trades during the bar's timespan ie; last 5 seconds (only available for TRADES)

After making an initial request with the reqRealTimeBars, an application can cancel the subscription with a cancelRealTimeBars .

For accessing Historical Data, the TWS API has three  functions as given below:

• reqHistoricalData :Provides bars from a specified time interval

• reqHistoricalTicks :Provides ticks from a specified time interval

• reqHistogramData :Provides a histogram containing data from a given period

Shown below is the function reqHistoricalData:

reqHistoricalData (int 	reqId, Contract contract, string endDateTime, string durationStr, string barSizeSetting, string whatToShow,  int useRTH, int formatDate, bool keepUpToDate, List< TagValue > chartOptions )

reqId	the request's identifier
contract	the contract for which we want to retrieve the data.
endDateTime	request's ending time with format yyyyMMdd HH:mm:ss {TMZ}
durationStr	the amount of time for which the data needs to be retrieved " S (seconds) - " D (days) " W (weeks) - " M (months) " Y (years)
barSizeSetting	the size of the bar: 1 sec ,5 secs , 15 secs , 30 secs, 1 min, 2 mins, 3 mins, 5 mins,  15 mins, 30 mins, 1 hour, 1 day
whatToShow	the kind of information being retrieved: TRADES, MIDPOINT, BID, ASK, BID_ASK, HISTORICAL_VOLATILITY, OPTION_IMPLIED_VOLATILITY ,FEE_RATE, REBATE_RATE
useRTH	set to 0 to obtain the data which was also generated outside of the Regular Trading Hours, set to 1 to obtain only the RTH data
formatDate	set to 1 to obtain the bars' time as yyyyMMdd HH:mm:ss, set to 2 to obtain it like system time format in seconds
keepUpToDate	set to True to received continuous updates on most recent bar data. If True, and endDateTime cannot be specified.

The callback function for reqHistoricalData is historicalData.

historicalData	(int reqId, Bar bar )

For The bar parameter in the above function is the OHLD historical bar.

After an application has requested historical data, it can cancel the subscription by calling cancelHistoricalData . The only required argument is the ID of the original request. 

reqHistoricalTicks is similar to reqHistoricalData , but provides less information. In fact, each tick only provides the price and size for each requested time. This function accepts nine arguments.

Shown below is the function reqHistoricalTicks:

reqHistoricalTicks (int	reqId, Contract	contract, string startDateTime, string endDateTime, int numberOfTicks, string whatToShow, int useRth, bool ignoreSize, List 	miscOptions)

reqId	id of the request
contract	contract object that is subject of query
startDateTime.	"20170701 12:01:00". Uses TWS timezone specified at login.
endDateTime	"20170701 13:01:00". In TWS timezone. Exactly one of start time and end time has to be defined.
numberOfTicks	Number of distinct data points. Max currently 1000 per request.
whatToShow	(Bid_Ask, Midpoint, Trades) Type of data requested.
useRth	Data from regular trading hours (1), or all available hours (0)
ignoreSize	A filter only used when the source price is Bid_Ask
miscOptions	should be defined as null, reserved for internal use

For whatToShow=MIDPOINT, The callback for the above function is  historicalTicks. For for whatToShow=BID_ASK, the callback function is historicalTicksBidAsk. For whatToShow=TRADES,  the callback function is historicalTicksLast

reqHistogram is similar to reqHistoricalData ,but instead of providing prices at different times, it provides the number of trades at different prices. This function accepts four arguments

• reqId: the request ID

• contract: the contract of interest

• useRTH: whether to use regular trading hours

• timePeriod: the time period of interest

After requesting histogram data, an application can access the results through the histogramData callback.

So now, lets get straight into some code to put this in action.

# Imports for the Program
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.order import Order
from ibapi.utils import iswrapper

import time
import threading
import datetime
import sys

class TestTradingData(EWrapper, EClient):
    ''' Serves as the Client and the Wrapper '''

    def __init__(self, addr, port, client_id):
        EWrapper.__init__(self)
        EClient.__init__(self, self)
        self.orderId = None

        # Connect to TWS API
        self.connect(addr, port, client_id)

        # Launch the client thread
        thread = threading.Thread(target=self.run)
        thread.start()

    @iswrapper
    def nextValidId(self, orderId):
        ''' Provides the next order ID '''
        self.orderId = orderId
        # print('NextValidId: '.format(orderId))
        print('The order id is: {}'.format(orderId))

    @iswrapper
    def tickByTickMidPoint(self, reqId, tick_timestamp, midpoint):
        ''' Callback to reqTickByTickData '''
        print('\nThe Midpoint Tick from tickByTickMidPoint Callback: {}'.format(midpoint))
        print('The timestamp of the realtime tick from tickByTickMidPoint: {}'.format(tick_timestamp))

    @iswrapper
    def tickPrice(self, reqId, field, price, attribs):
        ''' Callback to reqMktData '''
        print('\nFrom tickPrice callback - field is: {}'.format(field))
        if field == 1:
            self.bid_price = price
            print('Bid Price is: {}'.format(price))
        elif field == 2:
            self.ask_price = price
            print('Ask Price is: {}'.format(price))
        elif field == 4:
            self.last_price = price # Last price at which the contract traded
            print('Last Price is: {}'.format(price))
        elif field == 6:
            self.high_price = price
            print ('High Price is: {}'.format(price))
        elif field == 7:
            self.low_price = price
            print ('Low Price is: {}'.format(price))
        elif field == 9:
            self.close_price = price # The last available closing price for the previous day
            print('Close Price is: {}'.format(price))
        elif (field == 66 or field == 67 or field == 68 or field == 72 or field == 73 or field == 75): # Codes used for delayed data
            self.delayed_price = price 
            print('Delayed  Price is: {}'.format(price))       
    
    @iswrapper
    def tickSize(self, reqId, field, size):
        ''' Callback to reqMktData '''
        print('\nFrom tickSize callback - Requested field is: {} and Size is: {}'.format(field, size))

    @iswrapper
    def realtimeBar(self, reqId, bar_timestamp, open, high, low, close, volume, WAP, count):
        ''' Callback to reqMktData '''
        print('\nOpen: {}, High: {}, Low: {}, Close: {} at Bar Timestamp: {}'.format(open, high, low, close, bar_timestamp))

    @iswrapper
    def historicalData(self, reqId, bar):
        ''' Callback to reqHistoricalData '''
        print('\nOpen: {}, High: {}, Low: {}, Close: {}'.format(bar.open, bar.high, bar.low, bar.close))

    @iswrapper    
    def error(self, reqId, code, msg):
        print('Error Code: {}'.format(code))
        print('Error Message: {}'.format(msg))

def main():
    # Create the client and Connect to TWS API
    client = TestTradingData('127.0.0.1', 7497, 7)
    time.sleep(3) #Sleep interval to allow time for connection to server
    client.orderId = None

    # Define a Contract
    # contract = Contract()
    # contract.symbol = 'AAPL'
    # contract.secType = 'STK'
    # contract.exchange = 'SMART'
    # contract.currency = 'USD'
    contract = Contract()
    contract.symbol = 'INFY'
    contract.secType = 'STK'
    contract.exchange = 'SMART'
    contract.currency = 'INR' 
    contract.primaryExchange = "NSE"

    # Define 10 Ticks containing midpoint data
    client.reqTickByTickData(1, contract, 'MidPoint', 10, True)

    # Request market data
    client.reqMarketDataType(4) # Switch to live (1) frozen (2) delayed (3) delayed frozen (4).
    client.reqMktData(2, contract, '', False, False, [])

    # Request Real Time Bars of 5 seconds
    client.reqRealTimeBars(3, contract, 5, 'MIDPOINT', False, [])

    # Request Historical Bars
    now = datetime.datetime.now().strftime("%Y%m%d, %H:%M:%S")
    client.reqHistoricalData(4, contract, now, '2 D', '5 mins', 'BID', False, 1, False, [])

    # Disconnect from TWS
    time.sleep(10)
    client.disconnect()
    
if __name__ == "__main__":
    main()

Point to remember here is that if you are using the Free Trial version of IB, you may not have access to reqRealTimeBars and  reqHistoricalData if you are using the Paper Trading Account.  It may throw up errors messages like :

	Error 354: Requested market data is not subscribed.
	Error 162: Historical Market Data Service error message: No market data permissions for NSE STK
	Error 10167: Requested market data is not subscribed. Displaying delayed market data...

So yeah, if you are still testing on the Demo Account of IB or using the paper trading without actually Opening an account with IB, then you cannot proceed further. Open the account and deposit the minimum balance required so as to get the Live data. Use this Live Market Data in the paper trading account and test your strategy.

Happy Trading with IB !!!

Codebase on GitHub