Google Colab for Machine Learning

GPU was a problematic issue for many of us who want to experiment with  Machine Learning and Deep Learning modules.  Not Anymore !!!!!

Colaboratory is a Google research project and presently it supports free GPU.  It's a Jupyter notebook environment that requires no setup to use and runs entirely in the cloud. Colaboratory notebooks are stored in Google Drive and can be shared just as you would with Google Docs or Sheets. Colaboratory is free to use. For more information, see the Google Colab FAQ

Am I using GPU ?

import tensorflow as tf
tf.test.gpu_device_name()

If you are using GPU, you will receive a message like this:

If you are Not using GPU, you will receive a message like this:

Which GPU am I using ?

from tensorflow.python.client import device_lib
device_lib.list_local_devices()

What is my RAM info ?

!cat /proc/meminfo

What is my CPU info ?

!cat /proc/cpuinfo

How do I Read/Write files from/to Google Drive ?

Step 1: Installation of Drive FUSE Wrapper ?

Google Drive can be accessed using a FuseDrive Interface to access Google Drive files like Local Files

# Install a Drive FUSE wrapper.
# https://github.com/astrada/google-drive-ocamlfuse
!apt-get update -qq 2>&1 > /dev/null
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse

If everything goes through smoothly, you will get a message like this: 

If however you are running it for the second time,  you will get a warning like this: 

Warning: apt-key output should not be parsed (stdout is not a terminal)

Step 2: Generate Auth Token for Colab

# Generate auth tokens for Colab
from google.colab import auth
auth.authenticate_user()

This will give you to a link. Click on that and sign in to your google account. Now add the code and press Enter.

Step 3: Generate Credentials for the Drive FUSE library

# Generate creds for the Drive FUSE library.
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
# Work around misordering of STREAM and STDIN in Jupyter.
# https://github.com/jupyter/notebook/issues/3159
prompt = !google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass(prompt[0] + '\n\nEnter verification code: ')

!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}

Once again, Click on the link, get the verification code and enter it.


Step 4: Create a Directory and mount Google Drive using that directory

!mkdir -p MyDrive
!google-drive-ocamlfuse MyDrive

Step 5: Check files in the directory

#print 'Files in Drive:'
!ls MyDrive

Step 6: Create New File in the directory

!echo "This data will appear your newly created file list." > MyDrive/created.txt

Step 7: Check files in the directory once again
This will show up the newly created file

Step 8: Read the newly created file

f = open("MyDrive/created.txt")
print(f.read())

Step 9: Set the working directory
Note that at every step for reading or creating the file we have to add "MyDrive/" before defining each filename. To simplify this we can set the working directory.

import os
os.chdir("MyDrive/")

Step 10: Read the file without the fullpath

f = open("created.txt")
print(f.read())