Python Commands Hugo Touchette Department of Mathematical Sciences Stellenbosch University Version: 27 March 2020 --------------------------------------------------------- Features - Dynamic types (variable type can change) - No type declaration - Clean syntax, no end of line symbol - Extensive libraries (numpy, scipy, scikit) - Methods oriented (mimicking postfix application) --------------------------------------------------------- Environments ipython --pylab Interactive python Basic libraries are included (numpy, Matplotlib, etc.) python: Bare python command line python3: python v3 Running scripts: python file.py python3 file.py #!/usr/bin/env python #!/usr/bin/env python3 Google notebook: colaboratory or colab Anaconda Python (with IDLE Spyder) Jupyter = old IPython jupyter notebook jupyter lab (for fancy version) --------------------------------------------------------- Version compatibility - enable v3 features in v2 from __future__ import print_function from __future__ import division --------------------------------------------------------- Packages import [package] as [acronym] All functions available in namespace package.function All .py files and functions from .py files can be imported from [package] import [function1], [function2] Function is then called without package name from package import * Import all with given names Important packages: math log, log10, exp, sin, cos, asin, sinh e, pi, degrees, radians, fabs, pow cmath Complex maths numpy (as np) median, average, mean, std, var Many others .fft .linalg (LINPACK) .random Functions better than in math/cmath because listable matplotlib.pyplot (as plt) scipy Root finders, optimization, curve fitting, etc. Bigger vectors and matrices pylab = numpy, scipy and matplotlib seaborn (as sns) Nicer plots pandas (as pd) scikit-learn statsmodels keras datetime networkx (as nx) --------------------------------------------------------- Package installation pip install package Should be v2 but doesn't work on my installation pip3 install package v3 python -m pip install package Force v2 install python3 -m pip install package v3 sudo -H System wide install --------------------------------------------------------- Magic commands %timeit operation %pwd %run file.py --------------------------------------------------------- General # Comments Pydoc comments: """ ... """ Hash-bang or shebang: #!/usr/bin/env python #!/usr/bin/python --------------------------------------------------------- Variables and types Assignments not dynamic: x=3, y=x, x=5 doesn't change y Multiple assignments: x = y = 3 Dummy (void) assignments: x, _ = 1, 2 Tuple assignment: x, y = 3, 4 Swap: x, y = y, x Variables must start with letter Names are case sensitive Global and local variables can be handled - to avoid! Integer max = 2^31-1 Long Integer: with trailing L Float Complex: x=0.5+1.2j (no space or * with j) Real part: a.real or np.real(a) Imaginary part: a.imag or np.imag(a) Strings: '...' or "..." Python automatically converts types as necessary Examples: 1/2 -> 0 (v2 not v3) 3/2 -> 1.5 (v3) 1.0/2.0 -> 0.5 1.0/2 -> 0.5 Boolean: True and False Type conversion: float() int() complex() bool() str() type(x) Returns type of x --------------------------------------------------------- Operations + - * Multiplication or concatenation for lists / Division; int division if used on ints // Floor division for floats (rounding) 3/2 = 1 in v2 3/2 = 1.5 in v3 3//2 = 1 in both versions ** Exponentiation x%y modulo x|y bitwise (binary) OR of two integers x&y bitwise (binary) AND of two integers = Assignment == Test equality != Not equal (<> deprecated version) += -= *= /= //= or Logical and and Logical or Operations on different types: Result inherits general type Note: no ++, --, etc. iterators --------------------------------------------------------- Tuples (a, b, ...) Immutable Numbering from 0 (1,) Single tuple. (1) is just number 1 Function returns: a, b, c = ... return a, b, c --------------------------------------------------------- Lists [a, b, ...] Mutable Lists don't behave like vectors! List of lists don't behave like matrices! x[0][1] Matrix indices x[i] = 3 Override [] Empty list []*N Empty list of size N l=l*2 Repetition (growing same entry) l*=2 l=l+[elem] Append/concatenate l+=[elem] l.extend('elem') l.append(elem) del l[i] Delete element l.index(elem) Where element elem in list Boolean presence if not a: Check empty list zip(l1,l2) Make pairs b=a Linked assignments. Changing one changes the other copy() Real copy of list --------------------------------------------------------- Strings "..." or '..' Immutable s[i] ith element len(s) Length str() Convert to string + Concatenate s=str(i).zfill(3) Padding s.upper() To uppercase s.lower() To lowercase .strip().split Split words r'\blabla' Raw string: \ treated as literal, not as escape u'blabla' Unicode string --------------------------------------------------------- Dictionaries {a, b, ...} a, b, ... String keys 'string':value String keys d['a'] Element by name d['c']=n Add element del x['a'] Delete key 'a' in d Check entry in d.keys() Extract keys as iterable d.values() Extract values as iterable d.items() Extract both as iterable sorted(d) Sorted by keys Also use: keys=(lambda x: ...) If assigned, only linked. For copy, use copy() Examples: for x in dict: print(x) for x in dict.keys(): for x in dict.values(): for x in dict.items(): % For both for substance in sorted(d): print(substance, d[substance]) --------------------------------------------------------- Sets S={a, b, c, ...} Dictionary without values. Immoral dictionary. Not ordered. Useful to delete repetitions. S={} Empty set s.add() Append value set(l) Transform to set Can also use s={*list} Functions: intersection, union, difference --------------------------------------------------------- Slicing x[start:end:step] end not included x[0] First x[-1] Last x[:2] Same as x[0:2] x[:] All elements x[::-1] Reversed x[2:] After 2, 2 included x[2:-1] From two, last excluded (!) x[i is boolean] x[mask] Boolean element selection mask = vector of true/false --------------------------------------------------------- List unpacking *a = list From a in list *a, b = list From a to last b a, *b = list First a, rest from b a, *b, c = list First a, rest, last c --------------------------------------------------------- Ranges range(n) List from 0 to n-1 range(a,b) List from a to b-1 range(a,b,d) List from a to b in step d; b not included r.append(n) Append element n r.pop() Remove last element r.pop(n) Remove n-1 element r.insert(n,'...') r.reverse() r=[] Clear --------------------------------------------------------- Iterator map(f,r) Elementwise f on list r v2 list(map(f,r)) Change iterator to list v3 Can apply functions directly on lists or arrays to produce arrays --------------------------------------------------------- Truth values False, 0, 0L, 0.0, empty strings, lists, etc. True 1, and \neq 0 number and non-empty strings, lists, etc. x is x x is not x elem in xlist elem not in xlist Comparisons: <, >, <-, >= == != (<> deprecated) Booleans: and, or, not is Check if same object (address), not same value Example: y=0 if y: x/y else: print('Division not defined') --------------------------------------------------------- Conditional statement if [condition]: ... if [cond1]: ... elif [cond2]: ... elif [cond3]: ... else: ... Conditional assignment: exp = ... if ... else ... Chain conditions accepted: if 0 < x < 2: Example: print('horse' if x>=10 else 'duck' if 11=18 else False --------------------------------------------------------- While while [condition]: ... To be added: continue, break, pass Example: Fibonacci numbers f1, f2 = 1, 1 while f1<= 1000: f1, f2 = f2, f1+f2 --------------------------------------------------------- For loops for [item] in [sequence/iterable]: ... Example: for i in range(n): for key, val in d.item(): for x in list: Note: n not included in range (from 0 to n-1) Mute iterator: for _ in ...: --------------------------------------------------------- List comprehensions Don't loop on indices if you don't have to! Iterators: [expression for i in range(n)] [expression for i in list] [expression for i in list if condition] Same as for i in list: if condition: expression Examples: [i*i for i in range(5)] [i for i in list if boolean else ...] [x for x in X if P(x)] [f(x) for x in X if P(f(x))] [x**3 for x in range(n) if prime(x**3)] Pythonic filter(prime, [x**3 for x in range(n)] Non-pythonic print(list(filter(is_odd, aList))) Semi pythonic print([x for x in aList if x%2 == 1]) Pythonic List comprehensions can be nested. Generators: x=(expression for i in list) Not evaluated until prompted with x.next() Same behavior as 'yield' in loop --------------------------------------------------------- Functions def name(inputs): """ Name of function Description """ body return ... Default values: def name(x=2): One liner: def f(x): return 2*x yield Same as return for generators Notes: Can pass function heads as input/argument to functions. Multiple outputs: return a, b Functions can be recursive. Main program def main(): Function stub def name(...): pass No output: return None fcthead2=fcthead Renaming _, a, _ = fct() Dummy assignment --------------------------------------------------------- Lambda functions Examples: mult=lambda x, y: x*y mult(2,3) filter((lambda x: x>0), list) or directly on list --------------------------------------------------------- Creating packages Save all functions in mypackage.py from mypackage import [fct] --------------------------------------------------------- Built-in functions type() Return object type isinstance(object,type) print() Or just put object variable and press shift-enter sum(list) int() complex() float() str() min(list) max(list) len(list) map(f,list) list(iterator) append() Method pop() Method range() type() complex(a,b) a+bj tuple(list) Tuple to list enumerate(list) filter(cond,list) zip(l1,l2) reduce() set() Transform to set ravel() Flattens pass Null operation --------------------------------------------------------- Plots with pylab from pylab import plot, show plot(list) plot(x,y) show() Show terminal figure Example: Plotting a function x=linspace(a,b,n) y=sin(x) plot(x,y) Options: ylim(a,b) xlim(a,b) xlabel(str) ylabel(str) Scatter plot: plot(x,y) or scatter(x,y) Labels: o Circles x Crosses ^ Triangles s Squares D Diamond - Line -- Dashed _ Dashed line : Dotted line -. Dash dot Colors: r Red g Green b Blue c Cyan m Magenta y Yellow k Purple w White --------------------------------------------------------- Plots with matplotlib import matplotlib.pyplot as plt plot(x) plot(x,y) show() title(str) xlabel(str) ylabel(str) xlim(a,b) ylim(a,b) legend(['a','b'],loc='upper right') or label=str in plot figure() Re-initialize figure subplot(nmx) As in Matlab errorbar(x,y,xerr=...,yerr=...,fmt='...') Bar plots: plt.bar(bin_boundaries, bin_heights, width=) f = plt.figure() f.savefig("name.pdf", bbox_inches='tight') --------------------------------------------------------- Histograms Plots: plt.hist(data,binspec) plt.hist(data,bins=binspec) Bin specification: arange(a,b,d) Only bincounts Normalize as pdf: normed=1 Histogram counts: binheight, binboundary = np.histogram(data, bins=nbin) Options: bins=number or 'auto' range=(a,b) normed=True or False but deprecated density=True or False for pdf normalization --------------------------------------------------------- In and out a, b, ... = loadtxt('file.txt', unpack=True) From numpy Unpack: return columns in separate arrays Function ignores lines with # Assumes columns separated by tabs name = raw_input("Text...") Assign raw string from keyboard y = input("...") Assign numerical value from keyboard print "text" (v2) print a, b, c (v2) print(...) (v3) Space added automatically print(x,z,sep="...") for separator print(x,z,sep="") for no separator Formatting "Text: %d" % x "Text: %d ... %2.2f ... %d" % (a, b, c) %d integer %xd integer with x width %x.yf floating with x integer and y decimals %x.ye scientific e notation %+... forces sign specification \n newline \t tab \# # \' \" FileHandle = open('File', mode) r Read w Write a Append FileHandle.read() Read all FileHandle.readline() Read line by line FileHandle.readlines() Read/parse all lines FileHandle.close() Close file FileHandle.write(string) numpy.loadtxt('file', delimiter=',') pandas.read_csv pandas.read_table --------------------------------------------------------- Classes and objects Class with fixed attributes: class ClassName: var1 = ... var2 = ... def function(self): self. ... return ... Example: myvar = ClassName() myvar.var1 myvar.function() Notes: 'self' means anything that is in the class 'ClassName' 'self' = current instance of the class 'self' can be renamed to anything Inside functions: self.var1, self.var2, ... Functions inside classes are called methods Class with initialised attributes: class ClassName: def __init__(self, var1, var2): self.var1 = var1 self.age = var2 Example: myvar = ClassName(a, b) --------------------------------------------------------- Numpy pi e Functions are listable. array(list) Array out of list array(list,type) Specify type int or float x[a:b] a+1 to b or a to b-1 starting from 0 x[a:] a+1 to end x[:b] Beginning to b-1 x[:] All matrix( [ [..,..,..], [..,..,..], [..,..,..] ]) a[2:4,3:6] a[2,3:6] a[2,:] matrix(' a b; c d ') Matlab notation arange(a,b,s) Array range linspace(a,b,n) logspace(a,b,n) reshape(n,m) Reshape array in nxm matrix Method (comes after) zeros(n,type) Array of 0s. Type optional zeros([n,m],type) Matrix of 0s ones(n,type) Array of 1s ones([n,m],type) Matrix of 1s empty(n,type) Empty list empty([n,m],type) Empty matrix x.append(value) Append to x x.extend('hello') Append multiple elements x.size x.shape Dimensions x.shape = (1,2) Reshape x.dim x.dtype Data type np.clip where(array > 5) Find locations x.sort() Sort x Same as values=sort(values) key=len for sorting strings axis=0 or 1 for 2D arrays 2*a Listable a+b Array element add a+1 Listable add a*b Elementwise product dot(A,B) Dot product or matrix product cross(A,B) Cross product determinant(A) Determinant inv(A) Inverse matmul Matrix multiplication inner Inner product solve Solve linear system sum(axis=0) Column sums sum(axis=1) Row sums cusum Cumulative sum (also with axis) transpose or myarray.T inv or .I det eig linalg.solve np.zeros_like np.ones_like array_equal(a,b) concatenate stack hstack vstack median, mean, average std, var (normalized by N, not N-1) floor, ceil argmin, argmax intersect Notes: numpy uses LINPACK Explicit listable operation: b = array(map(sqrt,a),float) A=B are linked, not copied. Use copy() for copy --------------------------------------------------------- Random numbers From numpy random.random() Uniform [0,1] random.random(size) Array with uniform RVs random.uniform(a,b) Random real in [a,b) random.choice(list) Random element from list random.randn(size) Normal array random.normal(mu,sigma) Gaussian random.randint(a,b) Random integer in [a,b] random.shuffle(list) Shuffle a list --------------------------------------------------------- Funcy package Functions for list manipulation merge join walk (like map) flip where remove concat select --------------------------------------------------------- String manipulation str=str.strip().split() Read from right to left str.str.join(a) + Concatenation len() Length of string zfill(n) Padding with n a[0:5] Slicing str(i) To string i --------------------------------------------------------- Curve fitting Linear least square: np.polyfit(x, y, degree) Fit polynomial np.polyval([m, c], xvec) Define polynomial Ref: https://mmas.github.io/least-squares-fitting-numpy-scipy Nonlinear least square: scipy.optimize.curve_fit popt, pcov = curve_fit(fct_fit, xdata, ydata, guess_params, dy_optional) --------------------------------------------------------- Standard data classes and methods .data All data (X) .data.shape Data size .feature_names Data labels .target Class labels (Y) .target.shape Label size .target_names Label names .astype(type) Type conversion --------------------------------------------------------- Panda DataFrame Matrix with entries of multiple types --------------------------------------------------------- Jupyter shortcuts Esc m Text mode with latex enabled Esc y Code mode #, ##, ... Heading levels Esc B or b Insert cell below Esc A or a Insert cell above Esc D or d Delete cell - Bullet items 1. Enumerate items *text* Italics **text** Bold > text Quote $...$ Latex $$..$$ Displayed equation \\( .. )\\ Displayed equation %%latex More latex magic - can use anything like \[ \] Option + letter for greeks %run notebook.py Run external notebook %load file.py Load external script %%time Time cell %config InlineBackend.figure_format = 'retina' !ls Shell command jupyter nbconvert --to pdf jupyter nbconvert --to latex Insert image --------------------------------------------------------- Style tips Space after commas, not before Space around operators (=, +, etc.), except with setting values in functions No space in slicing: a[1:3] References: https://www.python.org/dev/peps/pep-0008/ https://www.python.org/dev/peps/pep-0020/ The Zen of Python https://barry.warsaw.us/software/STYLEGUIDE.txt import this --------------------------------------------------------- Jupyter environments jupyter --paths jupyter kernelspec list jupyter kernelspec uninstall unwanted-kernel --------------------------------------------------------- Jupyter extensions pip show jupyter_contrib_nbextensions jupyter nbextension list System wide: jupyter contrib nbextension install User directory: jupyter contrib nbextension install --user --------------------------------------------------------- Matlab engine in python (v2 not v3) import matlab.engine eng = matlab.engine.start_matlab() Start session (slow at first) eng.matlab_command Command use --------------------------------------------------------- Links General python.org ipython.org/notebook.html Plotting matplotlib.org/gallery.html python-graph-gallery.com Graphs/networks plot.ly/python/network-graphs/ networkx.github.io