How to create a Python based Machine Learning Development Environment

Machine Learning is one of the top skills these days and is capturing majority of the Software Industry day by day. The concept behind Machine Learning is to enable machines programmatically to learn and take smart decisions and keep on improving as the data size increases. The applications of Machine Learning can range from simple results predicting systems to complex online fraud detection applications.

So, If you are very much inspired in becoming  an Expert but unsure where to start from? This blog is for you. In this blog we will learn how to setup our own classroom, i.e. the fundamental development environment for Python based machine learning on your own machine.

Few Analogies to remember, these will be used  throughout this blog

Student = Machine

Teacher = You

Teaching/Training = Enabling machines to learn from data

Classroom = Development Environment

Medium of communication = Python

Chalkboard = IDE

Geometry box = Collection of installed packages

Let’s get started

Many people who are already working on Machine Learning will recommend to install complete Anaconda package on your machine. You can imagine Anaconda as a carefully curated toolset wrapped over Python with thousands of pre-installed packages, for details visit https://www.anaconda.com/download.

However, when I first installed Anaconda on my 16 GB RAM, Core i5 machine, the performance of my machine dropped  because Anaconda comes with a number of additional utilities which were more than I needed. This leads to uninstalling Anaconda, and install a smaller/compact version called Miniconda. Miniconda is a small toolset, which installs a customized version of Python, a package manager named ‘conda‘ and few other essential tools on your machine. You can download miniconda from https://conda.io/miniconda.html

Note: Python 3 is usually recommended for machine learning, so you need to download the setup which indicates to be using Python 3.

After you have downloaded miniconda, its time to install it. So continue with the installation by executing the downloaded file, and after installation, open command prompt with Administrative privileges (if you have installed miniconda for All Users). To check if miniconda has been installed successfully, hit the following commands.

> python --version
   Expected Output: Python 3.6.4 :: Anaconda, Inc.

> conda --version
   Expected Output: conda 4.5.0

Note: The versions may be different, depending on the version of miniconda you have installed

If these commands ran successfully, now is the time to install some of the very cool packages that will enable you to teach your machine to learn from data. We will install following packages

1. numpy: fundamental package for scientific computing with Python
2. pandas: package of easy-to-use data structures and data analysis tools
3. matplotlib: package of a cross-platform 2D plotting library
4. statsmodels: package that provides classes and functions for the estimation of many different statistical models, because a major portion of Machine learning is derived from Statistics
5. scikit-learn: package that provides simple and efficient tools for data mining and data analysis

Following command will do the work for you and install all above packages on your machine:

> conda install numpy pandas matplotlib statsmodels scikit-learn

Follow onscreen instructions, and enter ‘Y’ to proceed with the installation where asked to complete the installation. Once the installation is complete you will be able to create the elementary classroom (i.e. basic development environment).

At this point, you are very much capable of teaching your machine to learn many interesting things from data. But wait! what classroom is without a chalkboard? Yes, I know that you have some great text editors or IDEs already installed on your machine, but there is this very cool IDE called Spyder (visit: https://github.com/spyder-ide/spyder), that will help you teach your machine much faster. So to continue with the installation of this IDE, you can run the following command on the terminal.

> conda install spyder

Spyder has many dependencies, and conda will manage all of these dependencies for you. So what you need to do is enter ‘y’ to proceed with the installation. Once Spyder has been installed properly, you can open it and get familiar with it. Spyder comes with many cool features, few of them are:

1. Variables Explorer, this will help you visualize your data and variables in pretty formattable tables
2. Spyder installs IPython for you, which enables you to plot interactive graphs, right on your console.
3. You can execute the whole program or some selected code or even a single line
4. Embedded IPython console
5. Embedded Help box, where you can view the documentation of packages and methods

Congratulations, your classroom is ready to learn and teach your machine from the data.

Enjoy Machine Learning!

Fixing an issue with Spyder’s help

In case you observe the same issue that I observed with Spyder’s help panel, which made the help panel badly formatted and harder to understand.

For me, the python files I was using in Spyder were in different drive whereas Spyder was installed in a different drive. To fix this issue I downgraded the dependent package “sphinx” using following command:

> conda install sphinx=1.6.6

Follow this issue on GitHub (https://github.com/spyder-ide/spyder/issues/6549) for more details.

Blog Post