13 August 2017

Facebook Cleaner - Automate deletion of FB Posts/Likes/Comments

The Selenium WebDriver API is a great choice when it comes to automating browsers for auto / semi-auto trading. It also has some other quick fix applications like automating the deletion of facebook posts/likes/comments etc. To give a rough analogy -  As CCleaner is to the PC, this Facebook Cleaner is to your facebook  profile.
For the specific purpose of automating Facebook,  using the  Facebook SDK for Python is another  good option for consideration. However at the time of the development of this script,  the  Graph API Version 2.7 was the latest released version for the  facebook SDK for Python. But the official Graph API Version of facebook was 2.10. And so Selenium it was !!!!


26 July 2017

FAQ on Google Sheets & Google Finance for Traders

Many of us have used Google Sheets sometime or the other for EOD/RTD data. There are quite a few Screener's out there in the marketplace that are used by retail traders in NSE. One such is the FataFat Stock Screener

There are quite a few things that one can do with Google Sheets and this post seeks to demystify Google Finance for NSE Traders

How do we get EOD/RTD data for NSE by using Google Sheets
Google sheets use a function called GOOGLEFINANCE which is built-in into the google sheets. GOOGLEFINANCE fetches current or historical securities information from Google Finance


How does GOOGLEFINANCE fetch securities information?
GOOGLEFINANCE retrieves data that is stored on Google Servers

Who is the data provider for the data that Google uses?
End of day prices is provided by SIX Financial Information and Intra-day data may be provided by Interactive Data Real-Time Services, Inc.

14 July 2017

Syntax Highlighter for Source Code on Blogs - Google's code-prettify

 A few months back, I started sharing my open source codes on my blog and was looking around to find the best means to highlight the code samples.  Sometimes it helps to have code on the blog for a more detailed  step by step  explanation rather than just post a link to the whole code on a github repo.
There are many tools out there which are suited for the task. I finally zeroed in on Google's code-prettify becos of the simplicity of the solution that it offers. 

As the documentation says, code-prettify is "An embeddable javascript that makes source-code snippets in HTML prettier."


This is the way to setup your blogger for using code-pretiffy :
Step 1:   Login to your blogger account , click on Theme -> Edit HTML

25 June 2017

Scrap/Automate the Windows Terminal of Zerodha PI Scanner

Scenario:
Consider a case where you already have your Trading Strategy on Zerodha PI. This is written in TadeScript. Normally this code can be re-written in any other platform. But there is a catch - What if you are using some TradeScript specific built-in function like TREND . As per the manual, the TREND function is built-in to the core of the programming language ,  "primitives" , as they call it.

Recreating the TREND  in a programming language like Python calls for reverse engineering.  And despite my efforts, I was not able to do so accurately. My take is that identifying the trend of any financial instrument goes a long way In determining the profitability of a trader. And the Trend function in TradeScript is pretty good.  So the best way forward is to retain the strategy "as is"  in TradeScript and merely Scarp the data from the Scanner output of Zerodha. 


Approach:
I first tried to scarp the data using AutoIt . But then the reality dawned upon me that AutoIt does not support  .NET based WinForms and Windows Presentation Forms (WPF).   This can be determined with the help of the Inspect tool provided in   Windows 10SDK .  ( On Win10, the following path should help you to locate Inspect.exe: C:\Program Files (x86)\Windows Kits\10\bin\10.0.15063.0\x86 ). The  UIA Verify can also be used and is installed with the Windows Software Development Kit (SDK). It is located in the \bin\<platform>\UIAVerify folder of the SDK installation path (VisualUIAVerifyNative.exe).

Although AutoIt didn’t help, thankfully Pywinauto was up and ready with version  0.6.1 that fully supports  MSUI Automation (UIA support for WinForms, WPF, Qt, browsers, Store apps and more).

A point to note here is that once we identify the type of Controls of the Zerodha Pi Scanner Output  using Inspect / Spy ++ / UIAVerify / AutoIt Window Info , put these tools to rest. For use in the Python program,  use the name of the specific dialog by using the  Control Identifiers from   print_control_identifiers() method of Pywinauto 


03 June 2017

Get Real Time Data from Google Finance for NSE - Multi Threaded

This program is very similar to Get Real Time Data from Google for NSE . The difference is that this program uses concurrent.futures module in Python 3.5 to launch parallel tasks
  • The purpose of this program is to retrieve  Google Intraday Data for a list of symbols from the NSE stock exchange . Designed to run until 15:30 hrs Indian Standard Time(IST)
  • The Input file for this program is an Excel Spreadsheet. Name it as "stock_symbols". Enter the list of stocks in the  first column of the sheet for which you would like to retrieve the Google Intraday Data. 
  • This is a multi threaded program that uses the  concurrent.futures module (only from version Python 3.2) for asynchronously executing callables (Synchronous tasks are essentially Sequential in nature ie; when you do something synchronously, you wait for it to finish before moving on to another task. When you do something asynchronously, you can move on to next task before it finishes.)
  • The following arguments are required for the program :  sheetname time_in_minutes
  • Write's the output in OHLCV format in a CSV file
            When tested on a Intel I3-2367M processor for a list of 200 stocks:
            • For sequential processing , the time taken was about 30 seconds.
            • For asynchronous processing, the time taken was about 16 seconds.



            Test it yourself and verify the results for yourself. The results will vary based upon the machine that you use and your net speed.

            Program on GitHub  

            Cheers Maadi !!!

            17 February 2017

            Automation of the Windows Terminal of NSE NOW

            Disclaimer:
            1. Try this at your own risk.
            1. If you do decide to use this, Test  it thoroughly before you do so.
            2. At the time of writing, the NSE NOW version is : 1.13.3.5
            1. This has been tested by me  only for the above mentioned version for the NSE NOW software provided by my broker Finvasia.
            1. This has been tested on  Win 10 64 bit OS
            1. The automation script is designed for day trading on the NSE CASH segment

            Acknowledgement:
            This brilliant article from Stocksonfire set me thinking. Its perfectly suited for those of us who like to trade on the FnO segment  for a fixed lot size.  Although limited in its functionality, X-Mouse Button Control is a good choice for this.
            But what if I need to trade on the NSE Cash segment with a clear cut position sizing and a dynamic Stop Loss ? Therein lay the origin of this script written in AutoIt.  Some of the other tools that I have explored for this automation are SikuliX, Selenium and AutoHotkey


            Purpose of this Automation:
            1. Create a Hotkey for placing a Buy/Sell order. Here (Shift + B) is used for placing a Buy Order. And (Shift + S) is used to place a Sell Order
            2. Automatically place a SL (either SL-Limit Order or a SL-Market Order) along with each Buy/Sell Order.  The default is  SL-Limit Order.
            3. For a Long position, the Low of the day is taken as Stop Loss . For a Short position, the High of the day is taken as SL.
            1. A rudimentary Position Sizing .  For the input trading capital , the number of shares to Buy/Sell are determined by the formulae: (ltp - stoploss)*noofshares = risk per trade. For finer details on position sizing, check with the writings of GuruJi Van Tharp

            07 January 2017

            Calculating Exponential Moving Average (EMA) using Google Sheets

            The concept of Exponential Moving Average is explained here in this link

            The calculation of the Exponential Moving Average(EMA) can get tricky, unlike the calculation of the Simple Moving Average(SMA). If you take the same set of data points and determine the EMA on Google Sheets, Excel and Python, you will find that there is a difference in the EMA. That's becos to calculate the EMA there are different methods in adjusting the weights, which is handled differently (internally) in Excel , Google Sheets and Python.

            Or from a traders perspective, you would have noticed that there is a difference between the values of EMA in Sharekhan, Zerodha PI, Amibroker etc. This is becos: 
            1. There is a variance in the data feeds that these trading terminals use. This also leads to a difference in calculation of SMA or any other technical indicator.
            2. There is a variance in the method that they use internally to determine the calculation of EMA.
            3. Or probably one of them is using Excel, another is using Google Sheets and the third is using Python !!!

            Thanks to the efforts of  Marcello, we now have an easier google sheets method to calculate the EMA for use in trading. Here is the link:  https://docs.google.com/spreadsheets/d/1iBxaU-yeiG_ta4waj0YD6O8mn6DgbKUfE5Ap6W4aIis/edit#gid=0

            In case you want a Python based method, check out this oneliner code I mentioned in Stackoverflow . Or scroll below for more details.

            Here is a good explanation for the calculation of EMA using Pandas:

            Now lets get to the Python Code for the Calculation of SMA and EMA. Lets have an input file in the standard OHLCV format with Date/Datetime includedlabelled as Open, High, Low, Close, Volume. Now lets say that we want to calculate the SMA and the EMA for the Close price
            #imports and Declarations used in the program
            
            import pandas as pd
            input_file = "test data.csv"
            period = 10
            sma = 'SMA' + '_' + str(period)
            ema = 'EMA' + '_' + str(period)
            

            These would be the functions that we use in the program
            def SMA(df_stock, base_column, target_column, period=14):
                """
                Function to compute Simple Moving Average (SMA)
                
                Arguments :
                    df_stock : This is the Pandas DataFrame which contains ['Date', 'Open', 'High', 'Low', 'Close', 'Volume'] columns
                    base_column : The String that indicates the column name from which the SMA is computed.
                    target_column : The string that indicates the column name in which the computed data will be stored.
                    period : This integer that indicates the period of computation in terms of number of candles
                    
                Returns :
                    df_stock : Pandas DataFrame with new column added with SMA for the period mentioned.
                """
                
                df_stock[target_column] = df_stock[base_column].rolling(window=period).mean()
                df_stock[target_column].fillna(0, inplace=True)
            
                return df_stock
             
            def EMA(df_stock, base_column, target_column, period=14, alpha=False):
                """
                Function to compute Exponential Moving Average (EMA)
                
                Args :
                    df_stock : Pandas DataFrame which contains the columns ['Date', 'Open', 'High', 'Low', 'Close', 'Volume']
                    base_column : The String that indicates the column name from which the EMA needs to be computed
                    target_column : The String that indicates the column name in which the computed data will be stored
                    period : An Integer that indicates the period of computation in terms of number of candles
                    alpha : Boolean Value- If True indicates to use the formula for computing EMA using alpha (default is False)
                    
                Returns :
                    df : Pandas DataFrame with new column added with name 'target'
                """
            
                con = pd.concat([df_stock[:period][base_column].rolling(window=period).mean(), df_stock[period:][base_column]])
                
                if (alpha == True): #  set alpha=True for calculation of ATR
                    df_stock[target_column] = con.ewm(alpha=1 / period, adjust=False).mean()
                else: # set alpha=False for Calculation of EMA
                    df_stock[target_column] = con.ewm(span=period, adjust=False).mean()
                
                df_stock[target_column].fillna(0, inplace=True)
                return df_stock
            

            Next is to call the functions from the manin program:
            df_stock = pd.read_csv(input_file,header='infer',infer_datetime_format=True)
            df_stock['Date'] = pd.to_datetime(df_stock['Date']) # convert date to datetime
            
            SMA(df_stock, 'Close', sma, period)
            
            EMA(df_stock, 'Close', ema, period, False)
            
            The EMA calculated using this method is in sync with the EMA retreived from brokers like Sharekhan and Zerodha. In case there is a variance, try and increase the number of datapoints. The trick is to to have more datapoints for accurate smoothing, preferrably 250 datapoints or more.

            The EMA calcuation using Python was taken from https://github.com/arkochhar/Technical-Indicators. Thank you arkochhar