69.numpy-applications-2.md

NumPy Applications - 2

Methods of Array Objects

Getting descriptive statistics

Descriptive statistics mainly include central tendency, degree of dispersion, frequency analysis, and so on. For central tendency, we mainly look at the mean and median. For dispersion, we can look at extreme values, variance, standard deviation, and so on.

array1 = np.random.randint(1, 100, 10)
array1

Output:

array([46, 51, 15, 42, 53, 71, 20, 62,  6, 94])

Calculate the sum, mean, and median.

print(array1.sum())
print(np.sum(array1))
print(array1.mean())
print(np.mean(array1))
print(np.median(array1))
print(np.quantile(array1, 0.5))

Note: In the code above, mean, median, and quantile are functions in NumPy for calculating arithmetic mean, median, and quantiles. Setting the second argument of quantile to 0.5 means calculating the 50% quantile, which is the median.

Output:

460
460
46.0
46.0
48.5
48.5

Extreme values, range, and interquartile range.

print(array1.max())
print(np.amax(array1))
print(array1.min())
print(np.amin(array1))
print(np.ptp(array1))
print(np.ptp(array1))
q1, q3 = np.quantile(array1, [0.25, 0.75])
print(q3 - q1)

Output:

94
94
6
6
88
88
34.25

Variance, standard deviation, and coefficient of variation.

print(array1.var())
print(np.var(array1))
print(array1.std())
print(np.std(array1))
print(array1.std() / array1.mean())

Output:

651.2
651.2
25.51862065237853
25.51862065237853
0.5547526228777941

Draw a box plot.

A box plot, also called a box-and-whisker plot, is a statistical chart that shows the dispersion of a set of data. It gets its name because it looks like a box. It is mainly used to reflect the distribution characteristics of original data, and it can also compare the distribution characteristics of multiple groups of data.

plt.boxplot(array1, showmeans=True)
plt.ylim([-20, 120])
plt.show()

It is worth noting that for a two-dimensional or higher-dimensional array, when descriptive statistics are calculated, we can specify along which axis operations such as mean and variance are performed by using the parameter named axis. Different values of the axis parameter may lead to very different results, as shown below.

array2 = np.random.randint(60, 101, (5, 3))
array2

Output:

array([[72, 64, 73],
       [61, 73, 61],
       [76, 85, 77],
       [97, 88, 90],
       [63, 93, 82]])

array2.mean()

Output:

77.0

array2.mean(axis=0)

Output:

array([73.8, 80.6, 76.6])

array2.mean(axis=1)

Output:

array([69.66666667, 65.        , 79.33333333, 91.66666667, 79.33333333])

array2.max(axis=0)

Output:

array([97, 93, 90])

array2.max(axis=1)

Output:

array([73, 73, 85, 97, 93])

If we draw a box plot for a two-dimensional array, each column will produce one statistical graphic, as shown below.

plt.boxplot(array2, showmeans=True)
plt.ylim([-20, 120])
plt.show()

Note: The small circles in a box plot represent outliers, that is, values greater than $\small{Q_3 + 1.5 \times IQR}$ or smaller than $\small{Q_1 - 1.5 \times IQR}$. The constant 1.5 in the formula can be modified by the whis parameter of the boxplot function. Common values are 1.5 and 3. Setting it to 3 is usually done to identify extreme outliers.

It should also be said that NumPy array objects do not provide methods for calculating geometric mean, harmonic mean, trimmed mean, and so on. If you need these, you can use the third-party library scipy, whose stats module provides these functions. In addition, that module also provides functions for calculating mode, coefficient of variation, skewness, and kurtosis, as shown below.

from scipy import stats

print(np.mean(array1))                # arithmetic mean
print(stats.gmean(array1))            # geometric mean
print(stats.hmean(array1))            # harmonic mean
print(stats.tmean(array1, [10, 90]))  # trimmed mean
print(stats.variation(array1))        # coefficient of variation
print(stats.skew(array1))             # skewness coefficient
print(stats.kurtosis(array1))         # kurtosis coefficient

Output:

46.0
36.22349548825599
24.497219530825497
45.0
0.5547526228777941
0.11644192634527782
-0.7106251396024126

Brief overview of other related methods

1. all() / any(): check whether all elements in the array are True, or whether there is any element that is True.

2. astype(): copy the array and convert the elements in it to a specified type.

3. reshape(): adjust the shape of an array object.

4. dump(): save an array into a binary file, and then load it back with the load() function in NumPy.

   array1.dump('array1-data')
   array3 = np.load('array1-data', allow_pickle=True)
   array3

   Output:

   array([46, 51, 15, 42, 53, 71, 20, 62,  6, 94])
   

5. tofile(): write an array object into a file.

   array1.tofile('array.txt', sep=',')

6. fill(): fill the array with a specified element.

7. flatten(): flatten a multidimensional array into a one-dimensional array.

   array2.flatten()

   Output:

   array([1, 2, 3, 4, 5, 6, 7, 8, 9])
   

8. nonzero(): return the indexes of non-zero elements.

9. round(): round the elements in the array.

10. sort(): sort the array in place.

array1.sort()
array1

Output:

array([ 6, 15, 20, 42, 46, 51, 53, 62, 71, 94])

11. swapaxes() and transpose(): swap specified axes of an array and transpose an array.

array2.swapaxes(0, 1)

Output:

array([[1, 4, 7],
       [2, 5, 8],
       [3, 6, 9]])

array2.transpose()

Output:

array([[1, 4, 7],
       [2, 5, 8],
       [3, 6, 9]])

12. tolist(): convert an array into a Python list.

print(array2.tolist())
print(type(array2.tolist()))

Output:

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
<class 'list'>