Unit 8: NumPy functions

I. Orientation — Array-Wise Computation

NumPy functions operate efficiently on homogeneous, multidimensional arrays (ndarray). Most functions are vectorized: they process entire arrays in compiled code instead of requiring explicit Python loops.

• Core object: An ndarray has a fixed shape, a data type (dtype), and one or more dimensions called axes.
  • For shape (2, 3), axis 0 runs down the two rows and axis 1 runs across the three columns.
• Vectorization: An expression such as np.sqrt(a) applies square root to every element of a.
• Universal functions: A universal function, or ufunc, performs element-wise operations and commonly supports out, where, and type-related arguments.
• Broadcasting: NumPy can combine compatible shapes without manually copying data; for example, shape (3,) can broadcast across the rows of shape (2, 3).
• Axis convention: Reduction functions remove or reduce dimensions.
  • axis=0 computes down rows, producing one result per column.
  • axis=1 computes across columns, producing one result per row.
  • axis=None, usually the default, processes the flattened array.
• Dimension preservation: keepdims=True retains reduced axes with size 1, which can simplify later broadcasting.
• Data types: The input dtype affects precision, range, and output. Integer operations may overflow, while floating-point calculations can produce nan or inf.
• NaN convention: Ordinary statistical reductions usually propagate nan; specialized functions such as np.nanmean ignore it.

import numpy as np

a = np.array([[1, 2, 3],
              [4, 5, 6]])

print(a.shape)          # (2, 3)
print(np.sum(a, axis=0))  # [5 7 9]
print(np.sum(a, axis=1))  # [ 6 15]

II. Mathematical Functions — Vectorized Numerical Operations

A. Mathematical functions

Mathematical functions transform array elements or reduce them through arithmetic operations.

• Basic arithmetic: np.add, np.subtract, np.multiply, np.divide, and np.power are functional forms of +, -, *, /, and **.
  • np.floor_divide(a, b) performs floor division.
  • np.mod(a, b) or np.remainder(a, b) returns remainders.
• Powers and roots: np.square(x), np.sqrt(x), and np.cbrt(x) calculate common powers and roots element by element.
• Exponential functions: np.exp(x) computes (e^x), while np.exp2(x) computes (2^x).
• Logarithmic functions: np.log(x), np.log2(x), and np.log10(x) use bases (e), (2), and (10).
  • The natural logarithm requires (x>0) for finite real output.
• Trigonometric functions: np.sin, np.cos, and np.tan interpret angles in radians.
  • Convert degrees with np.deg2rad; convert radians with np.rad2deg.
  • Inverse functions include np.arcsin, np.arccos, and np.arctan.
• Rounding functions: np.round, np.floor, np.ceil, and np.trunc implement distinct rounding rules.
  • floor(2.8)=2, ceil(2.2)=3, and trunc(-2.8)=-2.
• Absolute value and sign: np.abs(x) returns magnitude, while np.sign(x) returns -1, 0, or 1.
• Aggregate arithmetic: np.sum adds elements, np.prod multiplies them, and np.cumsum and np.cumprod produce cumulative results.

For (x_i) denoting the element at position (i), cumulative sum is:

s_k = x_1 + x_2 + ... + x_k

Here, (s_k) is the cumulative total through position (k).

x = np.array([1, 4, 9, 16])

print(np.sqrt(x))       # [1. 2. 3. 4.]
print(np.cumsum(x))     # [ 1  5 14 30]
print(np.log2(x))       # [0. 2. 3.169925 4.]

B. Applications and limitations

Mathematical functions provide concise numerical code, but their domains and floating-point behavior must be respected.

• Conditional calculation: A ufunc can restrict computation with where; for example, np.sqrt(x, where=x >= 0) selects nonnegative positions.
• Output reuse: The out parameter can place results into an existing compatible array, reducing temporary allocations.
• Invalid domains: np.sqrt(-1.0) and np.log(-1.0) produce nan in real-valued arrays and issue runtime warnings.
• Floating-point comparison: Calculated decimals should generally be compared with np.isclose or np.allclose, not exact equality.

III. Statistical Functions — Describing Array Distributions

A. Statistical functions

Statistical functions summarize central tendency, dispersion, position, and relationships within numerical data.

• Mean: np.mean(x) calculates the arithmetic average:

mean = (x_1 + x_2 + ... + x_n) / n

Here, (x_i) is observation (i), and (n) is the number of observations.

• Median: np.median(x) returns the middle ordered value, or the average of the two middle values when (n) is even.
• Minimum and maximum: np.min and np.max return extreme values; np.argmin and np.argmax return their first indices.
• Range helper: np.ptp(x) computes peak-to-peak spread, equal to np.max(x) - np.min(x).
• Variance: np.var(x) measures average squared deviation from the mean:

variance = Σ(x_i - mean)² / (n - ddof)

ddof is the delta degrees of freedom. NumPy uses ddof=0 by default; sample variance commonly uses ddof=1.

• Standard deviation: np.std(x) is the square root of variance and has the same measurement unit as the observations.
• Percentiles and quantiles: np.percentile(x, 25) gives the 25th percentile; np.quantile(x, 0.25) expresses the same position on a 0–1 scale.
• Correlation and covariance: np.corrcoef returns normalized linear relationships, while np.cov measures joint variation.

scores = np.array([60, 70, 70, 80, 90])

print(np.mean(scores))          # 74.0
print(np.median(scores))        # 70.0
print(np.std(scores))           # 10.198...
print(np.percentile(scores, 75))  # 80.0

B. Applications and limitations

Statistical results depend on the selected axis, treatment of missing values, and interpretation of the population.

• Axis-based summaries: For a matrix of students by subjects, np.mean(scores, axis=0) computes each subject’s mean.
• Missing values: np.nanmean, np.nanmedian, np.nanstd, and related functions omit nan values rather than propagating them.
• Empty input: Statistics such as the mean of an empty slice are undefined and generally produce nan with a warning.
• Interpretation: Correlation measures linear association, not causation; a coefficient near zero does not rule out a nonlinear relationship.

IV. Sort Functions — Ordering Values and Indices

A. Sort functions

Sort functions arrange values or return the index order needed to arrange them.

1. Value sorting

   • np.sort(a): Returns a sorted copy, leaving a unchanged.
   • a.sort(): Sorts the array in place and returns None.
   • Axis control: The default axis=-1 sorts along the last axis; axis=None sorts the flattened values.

2. Index and partial sorting

   • np.argsort(a): Returns indices that would sort a; a[np.argsort(a)] produces the ordered values for a one-dimensional array.
   • np.partition(a, k): Places the element that belongs at index k into its sorted position, with smaller elements before it and larger elements after it; each partition is not fully sorted.
   • np.argpartition(a, k): Returns indices for the equivalent partial ordering.
   • np.lexsort(keys): Performs an indirect stable sort using multiple keys, with the last key acting as the primary key.

a = np.array([40, 10, 30, 20])

order = np.argsort(a)
print(order)             # [1 3 2 0]
print(a[order])          # [10 20 30 40]
print(np.partition(a, 1))  # element at index 1 is 20

B. Applications and limitations

The appropriate sorting function depends on whether ordered values, original positions, or only extreme elements are required.

• Rank preservation: argsort is preferable when another aligned array must be rearranged in the same order.
• Efficiency: partition is useful for finding the smallest (k) elements without paying for a complete sort.
• Stability: A stable sort preserves the original order of equal elements; this matters in multi-stage sorting.
• NaN placement: In NumPy sorting, nan values are placed after finite real numbers.

V. Search Functions — Locating Values and Conditions

A. Search functions

Search functions return positions where values satisfy conditions or where values should be inserted.

• Conditional selection: np.where(condition) returns index arrays for positions where the condition is true.
• Conditional construction: np.where(condition, x, y) chooses elements from x where true and from y where false.
• Nonzero positions: np.nonzero(a) returns indices of nonzero elements; np.flatnonzero(a) returns flattened indices.
• Coordinate output: np.argwhere(condition) returns matching coordinates as rows, making it convenient for listing locations in multidimensional arrays.
• Insertion search: np.searchsorted(a, v) finds insertion indices for v in an already sorted one-dimensional array.
  • side='left' returns the first valid insertion point.
  • side='right' returns the position after existing equal values.
• Membership testing: np.isin(a, values) creates a Boolean array indicating whether each element of a occurs in values.

a = np.array([10, 20, 30, 40, 50])

print(np.where(a > 25)[0])       # [2 3 4]
print(np.searchsorted(a, 35))    # 3
print(np.isin(a, [20, 50]))      # [False True False False True]

B. Applications and limitations

Search results must be interpreted according to their return structure and ordering assumptions.

• Tuple output: For multidimensional input, np.where(condition) returns one index array per dimension, not one table of coordinates.
• Sorted-input requirement: searchsorted assumes the search array is sorted; unsorted input can produce meaningless insertion positions.
• Exact versus approximate: Equality searches on computed floating-point values should use np.isclose(a, target) as the condition.
• First occurrence: np.argmax(condition) can locate the first True, but it returns 0 when all values are false, so np.any(condition) must be checked first.

VI. Counting Functions — Measuring Occurrences and Frequencies

A. Counting functions

Counting functions determine how many elements satisfy a condition or how frequently distinct values occur.

• Nonzero count: np.count_nonzero(a) counts elements not equal to zero.
  • With a Boolean array, it counts True values.
  • Its axis argument produces counts along selected dimensions.
• Boolean summation: np.sum(condition) counts true comparisons because True behaves as 1 and False as 0.
• Distinct-value frequency: np.unique(a, return_counts=True) returns sorted unique values and their occurrence counts.
• Nonnegative integer frequency: np.bincount(a) returns an array in which index (i) stores the number of occurrences of integer (i).
  • Inputs must be one-dimensional arrays of nonnegative integers.
• Interval frequency: np.histogram(a, bins) counts observations within numerical intervals.
  • Except for the final bin, intervals are half-open: the left edge is included and the right edge excluded.

a = np.array([1, 2, 2, 3, 3, 3])

values, counts = np.unique(a, return_counts=True)
print(values)                 # [1 2 3]
print(counts)                 # [1 2 3]
print(np.count_nonzero(a > 1))  # 5
print(np.bincount(a))         # [0 1 2 3]

B. Applications and limitations

The choice of counting function depends on whether the task concerns conditions, categories, integer labels, or continuous intervals.

• Condition counts: Use count_nonzero(a >= 50) for threshold-based frequencies.
• Category counts: Use unique(..., return_counts=True) for arbitrary sortable values such as strings or integer labels.
• Dense integer labels: bincount is efficient for compact nonnegative integer ranges, but can allocate a large result when the maximum label is very high.
• Histogram interpretation: Counts depend on bin boundaries; changing the bins can substantially change the apparent distribution.