Python Basics: With Illustrations from the Financial Markets

By Learning, QuantInsti, Quantitative

Book Id: WPLBN0100302034
Format Type: PDF eBook:
File Size: 7.31 MB
Reproduction Date: 27/06/2019

Title:	Python Basics: With Illustrations from the Financial Markets
Author:	Learning, QuantInsti, Quantitative
Volume:
Language:	English
Subject:	Non Fiction, Technology, Programming
Collections:	Authors Community, Education
Historic Publication Date:	2019
Publisher:	QuantInsti Quantitative Learning Pvt. Ltd.

Member Page:	Rekhit Pachanekar
Citation APA MLA Chicago Quantitative Learning, Compile, B. Q. (2019). Python Basics. Retrieved from http://www.gutenberg.cc/

Description
The material presented here is a condensed introduction to Python and its data science related libraries such as NumPy, Pandas, and Matplotlib. The illustrative examples we use are associated with the financial markets. We think it should also be useful to - Anyone who wants a brief introduction to Python and the key components of its data science stack, and - Python programmers who want a quick refresher on using Python for data analysis. We do not expect any of our readers to have a formal background in computer science, although some familiarity with programming would be nice to have. The concepts and ideas here are covered with several examples to help connect theory to practice.

Summary
Why was this written? This book grew out of the EPAT lectures that we have been conducting in Python for a number of years. In the EPAT Python lectures, we attempt to cover topics as widely as possible and therefore do not dig as deep as we’d like to, given the time limitations. These notes give us the freedom to marinate on some concepts a little longer and fill in on gaps that arise in the lecture format. We hope that our writing is compact and adequate for you, the reader, to obtain a greater understanding of the language and some of its essential libraries. We expect that for the interested reader, this book would be among the first of many books or blogs that you would read on Python. There is also a list of references towards the end of the book which we trust you may find useful.

Excerpt
Installing Numpy NumPy is not a part of the Python Standard Library and hence, as with any other such library or module, it needs to be installed on a workstation before it can be used. Based on the Python distribution one uses, it can be installed via a command prompt, conda prompt, or terminal using the following command. pip install numpy One point to note is that if we use the Anaconda distribution to install Python, most of the libraries (like NumPy, pandas, scikit-learn, matplotlib, etc.) used in the scientific Python ecosystem come pre-installed. Note: If we use the Python or iPython console to install the NumPy library, the command to install it would be preceded by the character! Once installed we can use it by importing into our program by using the import statement. The de facto way of importing is shown below: import numpy as np Here, the NumPy library is imported with an alias of np so that any functionality within it can be used with convenience. We will be using this form of alias for all the examples in this section. NumPy Arrays A Python list is a pretty powerful sequential data structure with some nifty features like index sub-setting and traversal. But lists lack an important feature, carrying out operations over an entire collection of elements in an efficient manner. For example, consider a case where we calculate PCR (Put Call Ratio) for the previous 5 days. Say, we have put and call options volume (in Lacs) stored in lists call_vol and put_vol respectively. We then compute the PCR by dividing put volume by call volume as illustrated in the below script: put_vol = [52.89, 45.14, 63.84, 77.1, 74.6] # Put volume in lacs call_vol = [49.51, 50.45, 59.11, 80.49, 65.11] # Call volume in lacs # Computing Put Call Ratio (PCR) put_vol / call_vol Traceback (most recent call last): File "", line 1, in put_vol / call_vol TypeError: unsupported operand type(s) for /: 'list' and 'list' Unfortunately, Python threw an error while calculating PCR values as it has no idea on how to do calculations on lists. We can do this by iterating over each item in lists and calculating the PCR for each day separately. However, doing so is inefficient and tiresome too. A way more elegant solution is to use Python NumPy arrays, an alternative to the regular Python list. Let us execute the same operation using a Python NumPy array. To do this, we use array() function from the NumPy package and create the Python NumPy version of put_vol and call_vol lists. In[] # Importing NumPy library import numpy as np # Creating arrays n_put_vol = np.array(put_vol) n_call_vol = np.array(call_vol) n_put_vol Out[] array([52.89, 45.14, 63.84, 77.1 , 74.6 ]) In[] n_call_vol Out[] array([49.51, 50.45, 59.11, 80.49, 65.11]) Here, we have two arrays n_put_vol and n_call_vol which holds put and call volume respectively.

Table of Contents
Contents 1 Introduction . . . . . . . . . . . . . . .. . . . . . . . . . . 1 1.1 What is Python? . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Where is Python used? . . . . . . . . . . . . . . . . . . . . 2 1.3 Why Python? . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 History of Python . . . . . . . . . . . . . . . . . . . . . . 6 1.5 Python 3 versus Python 2 . . . . . . . . . . . . . . . . . . .7 1.6 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . .10 2 Getting Started with Python . . . . . . . . . . . . . . .. . . 11 2.1 Python as a Calculator . . . . . . . . . . . . . . . . . . . 11 2.1.1 Floating Point Expressions . . . . . . . . . . . . . . . . 14 2.2 Python Basics . . . . . . . . . . . . . . . . . . . . . . . .17 2.2.1 Literal Constants . . . . . . . . . . . . . . . . . . . . .17 2.2.2 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . .18 2.2.3 Strings . . . . . . . . . . . . . . . . . . . . . . . . . .18 2.2.4 Comments . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.5 print() function . . . . . . . . . . . . . . . . . . . . . 20 2.2.6 format() function . . . . . . . . . . . . . . . . . . . . .22 2.2.7 Escape Sequence . . . . . . . . . . . . . . . . . . . . . .23 2.2.8 Indentation . . . . . . . . . . . . . . . . . . . . . . . .24 2.3 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . .25 3 Variables and Data Types in Python . . . . . . . . . . . . . . 27 3.1 Variables . . . . . . . . . . . . . . . . . . . . . . . . . .27 3.1.1 Variable Declaration and Assignment . . . . . . . . . . . .27 3.1.2 Variable Naming Conventions . . . . . . . . . . . . . . . .28 3.2 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.1 Integer . . . . . . . . . . . . . . . . . . . . . . . . . .31 3.2.2 Float . . . . . . . . . . . . . . . . . . . . . . . . . . .32 3.2.3 Boolean . . . . . . . . . . . . . . . . . . . . . . . . . .34 3.2.4 String . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2.5 Operations on String . . . . . . . . . . . . . . . . . . . 38 3.2.6 type() function . . . . . . . . . . . . . . . . . . . . . .41 3.3 Type Conversion . . . . . . . . . . . . . . . . . . . . . . .42 3.4 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . .45 4 Modules, Packages and Libraries . . . . . . . . . . . . . . .. 47 4.1 Standard Modules . . . . . . . . . . . . . . . . . . . . . . 50 4.2 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.3 Installation of External Libraries . . . . . . . . . . . . . 53 4.3.1 Installing pip . . . . . . . . . . . . . . . . . . . . . . 54 4.3.2 Installing Libraries . . . . . . . . . . . . . . . . . . . 54 4.4 Importing modules . . . . . . . . . . . . . . . . . . . . . .56 4.4.1 import statement . . . . . . . . . . . . . . . . . . . . . 56 4.4.2 Selective imports . . . . . . . . . . . . . . . . . . . . .57 4.4.3 The Module Search Path . . . . . . . . . . . . . . . . . . 59 4.5 dir()function . . . . . . . . . . . . . . . . . . . . . . . .61 4.6 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . .63 5 Data Structures . . . . . . . . . . . . . . .. . . . . . . . . 65 5.1 Indexing and Slicing . . . . . . . . . . . . . . . . . . . . 65 5.2 Array . . . . . . . . . . . . . . . . . . . . . . . . . . . .67 5.2.1 Visualizing an Array . . . . . . . . . . . . . . . . . . . 67 5.2.2 Accessing Array Element . . . . . . . . . . . . . . . . . .68 5.2.3 Manipulating Arrays . . . . . . . . . . . . . . . . . . . .68 5.3 Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.3.1 Accessing tuple elements . . . . . . . . . . . . . . . . . 71 5.3.2 Immutability . . . . . . . . . . . . . . . . . . . . . . . 72 5.3.3 Concatenating Tuples . . . . . . . . . . . . . . . . . . . 72 5.3.4 Unpacking Tuples . . . . . . . . . . . . . . . . . . . . . 73 5.3.5 Tuple methods . . . . . . . . . . . . . . . . . . . . . . .73 5.4 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 5.4.1 Accessing List Items . . . . . . . . . . . . . . . . . . . 75 5.4.2 Updating Lists . . . . . . . . . . . . . . . . . . . . . . 75 5.4.3 List Manipulation . . . . . . . . . . . . . . . . . . . . .77 5.4.4 Stacks and Queues . . . . . . . . . . . . . . . . . . . . 80 5.5 Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . 82 5.5.1 Creating and accessing dictionaries . . . . . . . . . . . .82 5.5.2 Altering dictionaries . . . . . . . . . . . . . . . . . . .85 5.5.3 Dictionary Methods . . . . . . . . . . . . . . . . . . . . 86 5.6 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.7 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . .92 6 Keywords & Operators . . . . . . . . . . . . . . .. . . . . . .95 6.1 Python Keywords . . . . . . . . . . . . . . . . . . . . . . .95 6.2 Operators . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.2.1 Arithmetic operators . . . . . . . . . . . . . . . . . . .106 6.2.2 Comparison operators . . . . . . . . . . . . . . . . . . .107 6.2.3 Logical operators . . . . . . . . . . . . . . . . . . . . 109 6.2.4 Bitwise operator . . . . . . . . . . . . . . . . . . . . .110 6.2.5 Assignment operators . . . . . . . . . . . . . . . . . . .114 6.2.6 Membership operators . . . . . . . . . . . . . . . . . . .118 6.2.7 Identity operators . . . . . . . . . . . . . . . . . . . .118 6.2.8 Operator Precedence . . . . . . . . . . . . . . . . . . . 119 6.3 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . 121 7 Control Flow Statements . . . . . . . . . . . . . . .. . . . .123 7.1 Conditional Statements . . . . . . . . . . . . . . . . . . .123 7.1.1 The if statement . . . . . . . . . . . . . . . . . . . . .123 7.1.2 The elif clause . . . . . . . . . . . . . . . . . . . . . 125 7.1.3 The else clause . . . . . . . . . . . . . . . . . . . . . 125 7.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 7.2.1 The while statement . . . . . . . . . . . . . . . . . . . 126 7.2.2 The for statement . . . . . . . . . . . . . . . . . . . . 128 7.2.3 The range() function . . . . . . . . . . . . . . . . . . .128 7.2.4 Looping through lists . . . . . . . . . . . . . . . . . . 130 7.2.5 Looping through strings . . . . . . . . . . . . . . . . . 131 7.2.6 Looping through dictionaries . . . . . . . . . . . . . . .131 7.2.7 Nested loops . . . . . . . . . . . . . . . . . . . . . . .133 7.3 Loop control statements . . . . . . . . . . . . . . . . . . 134 7.3.1 The break keyword . . . . . . . . . . . . . . . . . . . . 134 7.3.2 The continue keyword . . . . . . . . . . . . . . . . . . .135 7.3.3 The pass keyword . . . . . . . . . . . . . . . . . . . . 136 7.4 List comprehensions . . . . . . . . . . . . . . . . . . . . 137 7.5 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . 140 8 Iterators & Generators . . . . . . . . . . . . . . .. . . . . 143 8.1 Iterators . . . . . . . . . . . . . . . . . . . . . . . . . 143 8.1.1 Iterables . . . . . . . . . . . . . . . . . . . . . . . . 143 8.1.2 enumerate() function . . . . . . . . . . . . . . . . . . 145 8.1.3 The zip()function . . . . . . . . . . . . . . . . . . . . 146 8.1.4 Creating a custom iterator . . . . . . . . . . . . . . . .147 8.2 Generators . . . . . . . . . . . . . . . . . . . . . . . . .149 8.3 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . 151 9 Functions in Python . . . . . . . . . . . . . . .. . . . . . .153 9.1 Recapping built-in functions . . . . . . . . . . . . . . . .154 9.2 User defined functions . . . . . . . . . . . . . . . . . . .155 9.2.1 Functions with a single argument . . . . . . . . . . . . .156 9.2.2 Functions with multiple arguments and a return statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 9.2.3 Functions with default arguments . . . . . . . . . . . . 159 9.2.4 Functions with variable length arguments . . . . . . . . .160 9.2.5 DocStrings . . . . . . . . . . . . . . . . . . . . . . . .162 9.2.6 Nested functions and non-local variable . . . . . . . . . 164 9.3 Variable Namespace and Scope . . . . . . . . . . . . . . . .166 9.3.1 Names in the Python world . . . . . . . . . . . . . . . . 167 9.3.2 Namespace . . . . . . . . . . . . . . . . . . . . . . . . 168 9.3.3 Scopes . . . . . . . . . . . . . . . . . . . . . . . . . .169 9.4 Lambda functions . . . . . . . . . . . . . . . . . . . . . .174 9.4.1 map() Function . . . . . . . . . . . . . . . . . . . . . .175 9.4.2 filter() Function . . . . . . . . . . . . . . . . . . . . 176 9.4.3 zip() Function . . . . . . . . . . . . . . . . . . . . . .177 9.5 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . . 179 10 NumPy Module . . . . . . . . . . . . . . .. . . . . . . . . .181 10.1 NumPy Arrays . . . . . . . . . . . . . . . . . . . . . . . 182 10.1.1 N-dimensional arrays . . . . . . . . . . . . . . . . . . 185 10.2 Array creation using built-in functions . . . . . . . . . .186 10.3 Random Sampling in NumPy . . . . . . . . . . . . . . . . . 188 10.4 Array Attributes and Methods . . . . . . . . . . . . . . . 192 10.5 Array Manipulation . . . . . . . . . . . . . . . . . . . . 198 10.6 Array Indexing and Iterating . . . . . . . . . . . . . . . 203 10.6.1 Indexing and Subsetting . . . . . . . . . . . . . . . . .203 10.6.2 Boolean Indexing . . . . . . . . . . . . . . . . . . . . 205 10.6.3 Iterating Over Arrays . . . . . . . . . . . . . . . . . .210 10.7 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . .212 11 Pandas Module . . . . . . . . . . . . . . .. . . . . . . . . 215 11.1 Pandas Installation . . . . . . . . . . . . . . . . . . . .215 11.1.1 Installing with pip . . . . . . . . . . . . . . . . . . .216 11.1.2 Installing with Conda environments . . . . . . . . . . . 216 11.1.3 Testing Pandas installation . . . . . . . . . . . . . . .216 11.2 What problem does Pandas solve? . . . . . . . . . . . . . .216 11.3 Pandas Series . . . . . . . . . . . . . . . . . . . . . . .217 11.3.1 Simple operations with Pandas Series . . . . . . . . . . 219 11.4 Pandas DataFrame . . . . . . . . . . . . . . . . . . . . . 223 11.5 Importing data in Pandas . . . . . . . . . . . . . . . . . 228 11.5.1 Importing data from CSV file . . . . . . . . . . . . . . 228 11.5.2 Customizing pandas import . . . . . . . . . . . . . . . 228 11.5.3 Importing data from Excel files . . . . . . . . . . . . .229 11.6 Indexing and Subsetting . . . . . . . . . . . . . . . . . .229 11.6.1 Selecting a single column . . . . . . . . . . . . . . . .230 11.6.2 Selecting multiple columns . . . . . . . . . . . . . . . 230 11.6.3 Selecting rows via [] . . . . . . . . . . . . . . . . . .231 11.6.4 Selecting via .loc[] (By label) . . . . . . . . . . . . .232 11.6.5 Selecting via .iloc[] (By position) . . . . . . . . . . .233 11.6.6 Boolean indexing . . . . . . . . . . . . . . . . . . . . 234 11.7 Manipulating a DataFrame . . . . . . . . . . . . . . . . . 235 11.7.1 Transpose using .T . . . . . . . . . . . . . . . . . . . 235 11.7.2 The .sort_index() method . . . . . . . . . . . . . . . . 236 11.7.3 The .sort_values() method . . . . . . . . . . . . . . . .236 11.7.4 The .reindex() function . . . . . . . . . . . . . . . . .237 11.7.5 Adding a new column . . . . . . . . . . . . . . . . . . 238 11.7.6 Delete an existing column . . . . . . . . . . . . . . . .239 11.7.7 The .at[] (By label) . . . . . . . . . . . . . . . . . . 241 11.7.8 The .iat[] (By position) . . . . . . . . . . . . . . . . 242 11.7.9 Conditional updating of values . . . . . . . . . . . . . 243 11.7.10 The .dropna() method . . . . . . . . . . . . . . . . . .244 11.7.11 The .fillna() method . . . . . . . . . . . . . . . . . .246 11.7.12 The .apply() method . . . . . . . . . . . . . . . . . . 247 11.7.13 The .shift() function . . . . . . . . . . . . . . . . . 248 11.8 Statistical Exploratory data analysis . . . . . . . . . . .250 11.8.1 The info() function . . . . . . . . . . . . . . . . . . .250 11.8.2 The describe() function . . . . . . . . . . . . . . . . 251 11.8.3 The value_counts() function . . . . . . . . . . . . . . 252 11.8.4 The mean() function . . . . . . . . . . . . . . . . . . .252 11.8.5 The std() function . . . . . . . . . . . . . . . . . . . 253 11.9 Filtering Pandas DataFrame . . . . . . . . . . . . . . . . 253 11.10Iterating Pandas DataFrame . . . . . . . . . . . . . . . . 255 11.11Merge, Append and Concat Pandas DataFrame . . . . . . . . 256 11.12TimeSeries in Pandas . . . . . . . . . . . . . . . . . . . 259 11.12.1 Indexing Pandas TimeSeries . . . . . . . . . . . . . . .259 11.12.2 Resampling Pandas TimeSeries . . . . . . . . . . . . . .262 11.12.3 Manipulating TimeSeries . . . . . . . . . . . . . . . . 263 11.13Key Takeaways . . . . . . . . . . . . . . . . . . . . . . .265 12 Data Visualization with Matplotlib . . . . . . . . . . . . . 267 12.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . 268 12.1.1 Axes . . . . . . . . . . . . . . . . . . . . . . . . . . 269 12.1.2 Axes method v/s pyplot . . . . . . . . . . . . . . . . . 272 12.1.3 Multiple Axes . . . . . . . . . . . . . . . . . . . . . .273 12.2 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . 275 12.2.1 Line Plot . . . . . . . . . . . . . . . . . . . . . . . .276 12.2.2 Scatter Plot . . . . . . . . . . . . . . . . . . . . . . 289 12.2.3 Histogram Plots . . . . . . . . . . . . . . . . . . . . .294 12.3 Customization . . . . . . . . . . . . . . . . . . . . . . .300 12.4 Key Takeaways . . . . . . . . . . . . . . . . . . . . . . .313