21 November 2022

Send email from Gmail using Python

Configuration of Gmail

As of May 30, 2022, ​​Google no longer supports the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password. Here is the note on Google Policy on less secure apps.

Step 1:

Login to the Dashboard of Google Accounts:  https://myaccount.google.com

20 November 2022

Schedule Python Scripts with Windows Task Scheduler

Step 1

First let us identify the where python.exe is situated. Open the command prompt and type in "where python". 


16 November 2022

Webscraping of JavaScript Pages using a combination of Selenium & Beautiful Soup

Why Selenium?

Beautiful Soup makes web scraping easy by traversing the DOM (Document Object Model). But it handles only Static Scraping and is not capable of handling JavaScript. Essentially Beautiful Soup fetches webpage from the server without the help of a browser. We get what we see in the "view page source". If the data that we are looking for is available in the "view page source", then Beautiful Soup is sufficient for web scraping. But if we  need data that gets rendered only upon clicking a JavaScript link, then we need to use dynamic web scraping methods In such a situation we first use Selenium to automate the browser and click on the JavaScript link, wait for the elements to load and then use Beautiful Soup to extract the elements.

15 November 2022

Intro to Python logging module - An alternative to print() function

 Logging is generally used to debug our Python code. Many of us (like me) have a habit of using the print() function extensively during the development of a program. Nothing bad in that per se, just that when we want to promote this program to the production environment we must necessarily remove the print() statements. 

There are other reasons too why the print statement is not recommended and its better to use the logging module:

  • The print statement works only if we have access to the console
  • We have no mechanism to write the print statements to a text file for further perusal

08 May 2022

pdb - The Python Debugger / breakpoint

breakpoint versus pdb

Normally we import pdb and then to use the PDB in the program we have to use one of its method named set_trace(). This is how the code goes:
import pdb
pdb.set_trace()

An easier alternative (starting from Python version 3.7 onwards) is to use the built-in breakpoint() function. This helps us to do away with explicit import

breakpoint() # in python 3.7 and above

When we call the breakpoint() function, the default implementation of the breakpoint() function will call sys.breakpointhook(), which in turn calls the pdb.set_trace() function. This is why we do not need to import pdb and call pdb.set_trace() explicitly ourselves. 

27 March 2021

Create a Watchdog in Python

 What is Watchdog in Python

Watchdog is a python module that can be used to monitor file system changes. As the name suggests this module observes the given directory and can notify if a file is created or changed.

Step 1: Import some Stuff

import os
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler

Step 2: Create the event handler


if __name__ == "__main__":
    # patterns = "*" # the file patterns we want to handle
    patterns = "*.csv" # choose only csv files
    ignore_patterns = ""
    ignore_directories = True
    case_sensitive = True
    # Create the event handler
    my_event_handler = PatternMatchingEventHandler(patterns, ignore_patterns, ignore_directories, case_sensitive)

20 March 2021

Installation of WordPress on localhost

Why install wordpress.org locally on your computer?

We can do all our testing on our local computer , one that doesn’t affect our live WordPress site.

Note: If you install WordPress locally on Windows, then the only person who can see that site is you. If you want to make a website that is available to the public, then you need to have a domain name and web hosting.


What do I need for running wordpress.org locally on my computer?

For Windows users, WampServer is the best way to start off with installation of wordspress.org on your computer? WAMP, is a compilation of Apache web server, PHP and MySQL bundled for Windows computers.