In this semester I’m taking machine learning course, in which I do my assignments with Python. Currently the major package I’ve used is Python, and I decide to write this post to note something I’ve made mistakes on and learnt about.
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Examples:


According to this StackOverflow post, when use and
to evaluate 2 boolean arrays (and
still works for 2 boolean values), there is no fixed way to do this. So Numpy chooses to throw an ValueError
in this case.
While for &
and 
, they are defined to be bitwise operators on two arrays, so they would work as expected in this case.
And Numpy also provides another way to realize this feature, which is elementwise function like logical_and (compute the truth value of x1 AND x2 elementwise) and logical_or (compute the truth value of x1 OR x2 elementwise).
Another set of functions similar to logical_and
and logical_or
is elementwise bitwise function like bitwise_and (compute the bitwise AND of two arrays elementwise, and computes the bitwise AND of the underlying binary representation of the integers in the input arrays.) and bitwise_or.
So for boolean arrays, the effects of logical_and
and bitwise_and
are same, while bitwise_and
could also be applied to integer arrays.
Boolean Indexing
An example to predict yhat
in classification problems:


An example to calculate confusion matrix:


The results of yhat <= 0
and yhat > 0
are arrays of boolean values.
From manual:
Boolean arrays used as indices are treated in a different manner entirely than index arrays. Boolean arrays must be of the same shape as the initial dimensions of the array being indexed.
The result is a 1D array containing all the elements in the indexed array corresponding to all the true elements in the boolean array. The elements in the indexed array are always iterated and returned in rowmajor (Cstyle) order.
The result will be multidimensional if y has more dimensions than b. When the boolean array has fewer dimensions than the array being indexed, this is equivalent to y[b, …], which means y is indexed by b followed by as many : as are needed to fill out the rank of y. Thus the shape of the result is one dimension containing the number of True elements of the boolean array, followed by the remaining dimensions of the array being indexed.
numpy.dot()
numpy.dot() performs dot product of 2 numpy arrays.
If the two arrays are 1D, it is dot product. Two input arrays should have same dimensions. The output is a number.


If the two arrays are 2D, it is matrix multiplication. Dimension 1 of array 1 should be the same as dimension 0 of array 0.


Python function
return multiple values and whatever values
pass parameter by reference
Python pass parameter
To calculate ROC, I need to move the initial weight vector w
in parallel (by adding different offsets to it). At first I found after I ran the function, the w
vector passed in was also changed. So later I modified the code to following and solved the problem:


numpy way in Winnow algorithm
The running speeds of the following two code snippets with the same function are very different:




The function numpy.multiply() multiplies 2 input arrays elementwise.
The function numpy.exp() calculates the exponential of all input elements (also elementwise).
numpy 1D and 2D arrays


Both a and b seem exactly same (both 51 vector) in terms of real world linear algebra. But for Numpy they are actually *DIFFERENT.
And the difference in shape can lead to corresponding difference in calculation results. Whenever we want to define “vector” in Numpy, we need to think carefully about its dimension and shape.
One bug that I encountered related to this issue is to compute model prediction error (mean squared error). And it took me really long time to fix this bug!!






numpy bootstrap sampling
Definition of bootstrap sampling: random sampling with replacement


numpy OOB permuted error


The np.random.shuffle(x) function modifies a sequence inplace by shuffling its contents. It doesn’t have return values.
This function only shuffles the array along the first axis of a multidimensional array. The order of subarrays is changed but their contents remains the same.