Create Bootable Ubuntu USB Drive with Persistent Storage from Win 10

Scenario:

One of my colleagues has a Windows 10 based laptop that uses multiple tabs of Chrome at any given point of time. Primarily for the use of Google Sheets and Google Docs. Besides this she also uses:

• Hangouts for conferencing wherein she needs to share her screen 

• Discord for Team Collaboration

• Skype for messaging

• Open Broadcaster Software(OBS) for audio-video recording

Performance finetuning of Win 10 has helped to improve the overall performance of her windows based laptop. But not as good as she would have preferred it to be. Hardware upgradation for performance improvement was not a factor for consideration . 

Crack PDF Password using John the Ripper in Ubuntu 20.04

Scenario:

So I have this old PDF file which is password protected and  forgot the password. I tried various permutations and  combinations that came to my mind and they did not work. So what to do?

I started with PDFcrack and almost 3 days later, the program is still running !!!. So that's when I started looking at alternatives. Nothing wrong with PDFcrack per se, just that it is one of the limitations of brute force password cracking. In fact there is even a beautiful blogpost from Ruby Pdf Technologies on some of the intricacies of PDFcrack.

I remembered using something earlier when I was experimenting with Kali Linux. So I went back to the "gold standard" which is called John the Ripper password cracker.  

Types of Password's:

PDFs can be encrypted for confidentiality by requiring either a user password  or a owner password (as in case of DRM). PDFs encrypted with a user password can only be opened by providing this password. PDFs encrypted with a owner password can be opened without providing a password, but some restrictions will apply (for example, printing could be disabled).

Install Nvidia Drivers & CUDA in Ubuntu 20.04 (in UEFI mode with Secure Boot Enabled)

In the previous article, we have dealt with Installation of Ubuntu in Dual Boot Mode using UEFI. Now lets look at the installation of Nvidia Drivers for the GPU.

Basic Verification's:

First let us find out the graphics card that is presently used by the system. Click on Settings->About. Presently we have "NV138 / Mesa Intel® UHD Graphics 620 (KBL GT2)" installed.

 

Dual Boot Win 10 and Ubuntu for system with (SSD + HDD) using UEFI - GPT method

Scenario:

My laptop uses Unified Extensible Firmware Interface (UEFI) and not the good old BIOS. I have a hybrid drive system in my machine, a mix of 256 GB SSD and 1 TB HDD. In my 256 GB SSD drive, I have my Win 10 operating system installed and main programs like Office , Chrome  etc. installed. I use the 1 TB HDD primarily to store data.

I intend to use a Linux based VPS to host my trading strategies on AWS. So how do I create a dual boot in my existing Win 10 machine ?

Thread Synchronization using Event Object in Python - Interactive Brokers Part 4

This post uses the Interactive Brokers Python TWS API to explain Thread Synchronization using Event Object.

In many applications, sometimes, we need to pause the running of the program until some external condition occurs. You may need to wait until another thread finishes, or another callback is processed. In these situations and and many other similar situations you will need to figure out a way to make your script wait. Three common ways to achieve this are mentioned: 

1. This is sometimes achieved by using a "trial and error approach using the time.sleep() function in Python". While this may work, it is always a problematic issue in determining the amount of time we need to wait. And using the time.sleep() in such cases is not the right way of achieving the result. 
2. Another approach is to use flags to arrive at a better approximation in using the time.sleep() function.
3. Use the Event Object to achieve thread synchronization

Making blogger a little less painful

Table of Contents

• Formatting Source Code in Blogger
• Creating Table of Contents in Blogger

Recently google made an update to blogger for formatting code and inserting anchors in the Compose View itself(instead of HTML view). Yeah, it was just about the time when they came out with this fine separation between "blogger" and "legacy blogger".

The new changes they had incorporated was very good, focused as it was on simplicity . I actually used it to create one blog post which required the use of Table of Contents right from the "Compose View". I also used it to insert my Python code right from the "Compose View". Imagine the savings in time for somebody who makes a blog post daily ?

Suddenly a few days later all of it vanished as mysteriously as they appeared. Another Google+ ??? (oh yeah, that much touted alternative to Facebook)

Google seems to have done a veto on their recent updates to blogger. But yet I prefer google over some other blogging options like Medium becos in the near future google corporation don't need to worry about monetizing blogger , unlike the corporation run by Evan William's. ( He is the same guy who started blogger too , which was latter purchased by Google ).

So I began to think - How can I make blogging more easy on blogger instead of switching to another platform. Maybe google might bring those feature back, or maybe they wont. We don't know. So lets see what we can do for now.

I would like to cover the following issues which are of concern to me:

Formatting Source Code:

I have already dealt with formatting code in one of my previous posts. Check out one of my earlier post - Syntax Highlighter for Source Code on Blogs . This deals with code-prettify and unfortunately for us this is archived and is no longer maintained. So sooner than latter, we will have to look at alternatives.

The one I found most helpful was the use of gist. I have been using GitHub for a couple of years now and so it was the logical to continue with it.

Advanced Techniques in TWS API - Interactive Brokers Part 3

In the first part of the blog on the TWS API of Interactive Brokers, I have dealt with the Installation of TWS API on a Windows machine for Anaconda distro.

In the second part of the blog, titled Fundamentals of TWS API, I have dealt with the essentials to get started.

Now in this third part I seek to explore the concepts like using a scanner, retrieving the OHLCV data and writing it to a CSV file, Hedging a Futures Contract with Options and many others including a fully functional sample trading system.

Table of Contents :

• Optionable contracts in NSE Using Scanner

• Write OHLCV data to Pandas and CSV files

• Web Scrap IB site for Single Stock Futures (SSF)

• Creating a Stock Universe

• Write OHLCV Data of the Stock Universe to CSV file

• Intraday Trading System: OpenEQ

• Codebase on GitHub

Optionable Contracts in NSE using the Scanner

Let's get to the point straight - The TWS API is just an interface to the TWS. If you are having problems defining a scanner viz the API , always make sure you can create a similar scanner using the TWS or the Mosaic Market Scanner.

Fundamentals of TWS API - Interactive Brokers Part 2

Generally Interactive Brokers releases the installer for both the TWS API and IB Gateway simultaneously. For this blog I will use TWS API ( Version: API 9.79 Release Date: Feb 05 2020 ) and not IB Gateway. Check out my previous blog post for detailed installation guide of TWS API on Win 10 Machine that uses the Anaconda Distribution for Python.

 

IB also has a host of API's. A recent one that they added is a REST based API which they refer to as Client Portal WebAPI . However this blog post deals solely with the TWS API

                            Table of Contents :

• EClient and EWrapper Class
• Establishing an API Connectivity
• Choice of IDE
• Threads and Messages
• Contracts in TWS API
• Orders in TWS API
• Live Market Data & Historical Data in TWS API
• Codebase on GitHub

The general operation of the TWS API application is:

1. Establish connection with IB server.
2. Request information from IB server or execute an action
3. If a response is provided by the IB server, then receive the response and process the response
4. Repeat steps 2 & 3 until all the required information has been received and all the operations have been executed.
5. Terminate the Connection with IB server.

Installation of TWS API - Interactive Brokers Part 1

One advantage of Interactive Brokers (referred to as IBKR or just IB hereafter) over Kite Connect or Upstock API is that IB provide's a Paper Trading Account.

If you are an Individual, you can signup for the free trial ofInteractive Brokers . This basically gives you a "Paper Trading Account" with access to all the IBKR platforms for a couple of months including the TWS and Mosaic platforms. For the purpose of live testing our trading strategy , we have access to  Market Data ( Delayed by 10-15 minutes) and the API for retrieving data  (Not all Data) and placing orders. So yeah, the trial version of IB provides "Limited Functionality" for testing your strategies using the API. 

Some of the things that you can do with ApplicationProgramming Interface(API) from IBKR  :

• Automate your strategies
• Create a custom trading terminal
• Develop a Screener
• Experiment with your own Trading Indicator

Basically if you are reading this, you know why you have opted for IBKR.

Its is important to note that there are other third party Libraries in Python like ib_insync, IbPy and IBridgePy. But these have been developed well before the release of the Python Native API which is developed by IBKR . Only the API developed by IBKR is officially supported by IBKR.  For more information on the API, Check the YouTubelink for the official videos of TWS Python API

And besides the TWS API, there are also other API solutions that are offered by Interactive Brokers and check them in this PDF download.  Also one can check out groups for discussion on  API

It is always helpful to get a feel of the web interface of IB which they refer to as Client Portal and Trader Work Station(TWS) before we get started.

Calculation of Supertrend (ST)

The Supertrend is one of the most widely used Indicator among the Indian traders. And unfortunately there is very little documentation on this subject at Investopedia or the chart school of Stockcharts or even with Zerodha Varsity. So it required quite some investigation to arrive at the right method to calculate the Supertrend.

In its simplest form, when the Supertrend is overlayed over a candlestick chart, if the Supertrend crosses to below the Price we take a Long Position and when the Supertrend crosses to above the Price, we take a Short position. Here is a snapshot of the Supertrend.

This article from Economic Times is probably a good place to start with to get a qucik overview of Supertrend.

So let us dive straight into and get the calculation of Supertrend. Check this Google Sheet for the Calculation of Supertrend. 

Given below is the pseudo code for the calculation of Supertrend:

Basic UpperrBand = (High + Low) / 2 + Multiplier * ATR
Basic LowerBand =  (High + Low) / 2 - Multiplier * ATR

Final UpperBand = IF((Current BasicUpperband < Previous Final UpperBand) OR 
                    (Previous Close > Previous Final UpperBand))  THEN 
                    (Current Basic UpperBand) ELSE
                    (Previous FinalUpperBand)

Final LowerBand = IF((Current Basic LowerBand > Previous Final LowerBand) OR
                    (Previous Close < Previous Final LowerBand)) THEN
                    (Current Basic LowerBand) ELSE
                    (Previous Final LowerBand)

SuperTrend = IF((Previous SuperTrend = Previous Final UpperBand) AND 
               (Current Close <= Current Final UpperBand)) THEN 
               Current Final UpperBand
             ELSE
                IF((Previous SuperTrend = Previous Final UpperBand) AND
                  (Current Close > Current Final UpperBand)) THEN
                  Current Final LowerBand
                ELSE
                   IF((Previous SuperTrend = Previous Final LowerBand) AND
                     (Current Close >= Current Final LowerBand)) THEN
                     Current Final LowerBand
                   ELSE
                      IF((Previous SuperTrend = Previous Final LowerBand) AND
                        (Current Close < Current Final LowerBand)) THEN
                        Current Final UpperBand

Now lets see how this is calculated in Python. Lets first download the OHLCV data from the above google sheet into a CSV file. Name it as "test data". We can use this as our input file for the Python program.

Lets first take a look at the Imports and Global Declarations that we will use in this program:

#imports used in the program
import numpy as np
import pandas as pd

#Global Declarations
input_file = "test data.csv"
multiplier = 2 # An integer to indicate the value to multiply the ATR.
period = 14
atr = 'ATR' + '_' + str(period)
st = 'SuperTrend' + '_' + str(period) + '_' + str(multiplier)