numpy_ext¶

An extension library for NumPy that implements common array operations not present in NumPy.

Installation¶

Regular installation:

pip install numpy_ext

Installation for development:

git clone https://github.com/3jane/numpy_ext.git
cd numpy_ext
pip install -e .[dev]  # note: make sure you are using pip>=20

Window operations¶

numpy_ext.expanding()
numpy_ext.expanding_apply()
numpy_ext.rolling()
numpy_ext.rolling_apply()

Operations with nans¶

numpy_ext.nans()
numpy_ext.drop_na()
numpy_ext.fill_na()
numpy_ext.fill_not_finite()
numpy_ext.prepend_na()

Functions¶

numpy_ext.apply_map(func: Callable[[Any], Any], array: Union[List, numpy.ndarray]) → numpy.ndarray[source]¶

Apply a function element-wise to an array.

Parameters:	funcCallable[[Any], Any] Function that accepts one argument and returns a single value. arrayUnion[List, np.ndarray] Input array or a list. Any lists will be converted to np.ndarray first.
Returns:	np.ndarray Resulting array.

Examples

>>> apply_map(lambda x: 0 if x < 3 else 1, [[2, 2], [3, 3]])
array([[0, 0],
       [1, 1]])

numpy_ext.drop_na(array: numpy.ndarray) → numpy.ndarray[source]¶

Return a given array flattened and with nans dropped.

Parameters:	arraynp.ndarray Input array.
Returns:	np.ndarray New array without nans.

Examples

>>> drop_na(np.array([np.nan, 1, 2]))
array([1., 2.])

numpy_ext.expanding(array: numpy.ndarray, min_periods: int = 1, skip_na: bool = True, as_array: bool = False) → Union[Generator[numpy.ndarray, NoneType, NoneType], numpy.ndarray][source]¶

Roll an expanding window over an array. The window size starts at min_periods and gets incremented by 1 on each iteration. The result is either a 2-D array or a generator of slices, controlled by as_array parameter.

Parameters:	arraynp.ndarray Input array. min_periodsint, optional Minimum size of the window. Default is 1. skip_nabool, optional If False, the windows of size less than min_periods are filled with nans. If True, they’re dropped. Default is True. as_arraybool, optional If True, return a 2-D array. Otherwise, return a generator of slices. Default is False.
Returns:	np.ndarray or Generator[np.ndarray, None, None]

Examples

>>> expanding(np.array([1, 2, 3, 4, 5]), 3, as_array=True)
array([array([1, 2, 3]), array([1, 2, 3, 4]), array([1, 2, 3, 4, 5])],
      dtype=object)

numpy_ext.expanding_apply(func: Callable, min_periods: int, *arrays: numpy.ndarray, prepend_nans: bool = True, n_jobs: int = 1, **kwargs) → numpy.ndarray[source]¶

Roll an expanding window over an array or a group of arrays producing slices. The window size starts at min_periods and gets incremented by 1 on each iteration. Apply a function to each slice / group of slices, transforming them into a value. Perform computations in parallel, optionally. Return a new np.ndarray with the resulting values.

Parameters:

Parameters:	funcCallable The function to apply to each slice or a group of slices. min_periodsint Minimal size of expanding window. arrayslist List of input arrays. prepend_nansbool Specifies if nans should be prepended to the resulting array n_jobsint, optional Parallel tasks count for joblib. If 1, joblib won’t be used. Default is 1. *kwargsdict Input parameters (passed to func, must be named).
Returns:	np.ndarray

funcCallable: The function to apply to each slice or a group of slices.
min_periodsint: Minimal size of expanding window.
*arrayslist: List of input arrays.
prepend_nansbool: Specifies if nans should be prepended to the resulting array
n_jobsint, optional: Parallel tasks count for joblib. If 1, joblib won’t be used. Default is 1.
**kwargsdict: Input parameters (passed to func, must be named).

Returns:

np.ndarray

Examples

>>> arr = np.array([1, 2, 3, 4, 5])
>>> expanding_apply(sum, 2, arr)
array([nan,  3.,  6., 10., 15.])
>>> arr2 = np.array([1.5, 2.5, 3.5, 4.5, 5.5])
>>> func = lambda a1, a2, k: (sum(a1) + max(a2)) * k
>>> expanding_apply(func, 2, arr, arr2, k=-1)
array([  nan,  -5.5,  -9.5, -14.5, -20.5])

numpy_ext.expstep_range(start: Union[int, float], end: Union[int, float], min_step: Union[int, float] = 1, step_mult: Union[int, float] = 1, round_func: Callable = None) → numpy.ndarray[source]¶

Return spaced values within a given interval. Step is increased by a multiplier on each iteration.

Parameters:

Parameters:	startint or float Start of interval, inclusive endint or float End of interval, exclusive min_stepint or float, optional Minimal step between values. Must be bigger than 0. Default is 1. step_multint or float, optional Multiplier by which to increase the step on each iteration. Must be bigger than 0. Default is 1. round_func: Callable, optional Vectorized rounding function, e.g. np.ceil, np.floor, etc. Default is None.
Returns:	np.ndarray Array of exponentially spaced values.

startint or float: Start of interval, inclusive
endint or float: End of interval, exclusive
min_stepint or float, optional: Minimal step between values. Must be bigger than 0. Default is 1.
step_multint or float, optional: Multiplier by which to increase the step on each iteration. Must be bigger than 0. Default is 1.
round_func: Callable, optional: Vectorized rounding function, e.g. np.ceil, np.floor, etc. Default is None.

Returns:

np.ndarray: Array of exponentially spaced values.

Examples

>>> expstep_range(1, 100, min_step=1, step_mult=1.5)
array([ 1.        ,  2.        ,  3.5       ,  5.75      ,  9.125     ,
       14.1875    , 21.78125   , 33.171875  , 50.2578125 , 75.88671875])
>>> expstep_range(1, 100, min_step=1, step_mult=1.5, round_func=np.ceil)
array([ 1.,  2.,  4.,  6., 10., 15., 22., 34., 51., 76.])
>>> expstep_range(start=-1, end=-100, min_step=1, step_mult=1.5)
array([ -1.        ,  -2.        ,  -3.5       ,  -5.75      ,
        -9.125     , -14.1875    , -21.78125   , -33.171875  ,
       -50.2578125 , -75.88671875])

Generate array of ints

>>> expstep_range(start=100, end=1, min_step=1, step_mult=1.5).astype(int)
array([100,  99,  97,  95,  91,  86,  79,  67,  50,  25])

numpy_ext.fill_na(array: numpy.ndarray, value: Any) → numpy.ndarray[source]¶

Return a copy of array with nans replaced with a given value.

Parameters:	arraynp.ndarray Input array. valueAny Value to replace nans with.
Returns:	np.ndarray A copy of array with nans replaced with the given value.

Examples

>>> fill_na(np.array([np.nan, 1, 2]), -1)
array([-1.,  1.,  2.])

numpy_ext.fill_not_finite(array: numpy.ndarray, value: Any = 0) → numpy.ndarray[source]¶

Return a copy of array with nans and infs replaced with a given value.

Parameters:	arraynp.ndarray Input array. valueAny, optional Value to replace nans and infs with. Default is 0.
Returns:	np.ndarray A copy of array with nans and infs replaced with the given value.

Examples

>>> fill_not_finite(np.array([np.nan, np.inf, 1, 2]), 99)
array([99., 99.,  1.,  2.])

numpy_ext.nans(shape: Union[int, Tuple[int, ...]], dtype=<class 'numpy.float64'>) → numpy.ndarray[source]¶

Return a new array of a given shape and type, filled with np.nan values.

Parameters:	shapeint or tuple of ints Shape of the new array, e.g., (2, 3) or 2. dtype: data-type, optional
Returns:	np.ndarray Array of np.nans of the given shape.

Examples

>>> nans(3)
array([nan, nan, nan])
>>> nans((2, 2))
array([[nan, nan],
       [nan, nan]])
>>> nans(2, np.datetime64)
array(['NaT', 'NaT'], dtype=datetime64)

numpy_ext.prepend_na(array: numpy.ndarray, n: int) → numpy.ndarray[source]¶

Return a copy of array with nans inserted at the beginning.

Parameters:	arraynp.ndarray Input array. nint Number of elements to insert.
Returns:	np.ndarray New array with nans added at the beginning.

Examples

>>> prepend_na(np.array([1, 2]), 2)
array([nan, nan,  1.,  2.])

numpy_ext.rolling(array: numpy.ndarray, window: int, skip_na: bool = False, as_array: bool = False) → Union[Generator[numpy.ndarray, NoneType, NoneType], numpy.ndarray][source]¶

Roll a fixed-width window over an array. The result is either a 2-D array or a generator of slices, controlled by as_array parameter.

Parameters:	arraynp.ndarray Input array. windowint Size of the rolling window. skip_nabool, optional If False, the sequence starts with (window-1) windows filled with nans. If True, those are omitted. Default is False. as_arraybool, optional If True, return a 2-D array. Otherwise, return a generator of slices. Default is False.
Returns:	np.ndarray or Generator[np.ndarray, None, None] Rolling window matrix or generator

Examples

>>> rolling(np.array([1, 2, 3, 4, 5]), 2, as_array=True)
array([[nan,  1.],
       [ 1.,  2.],
       [ 2.,  3.],
       [ 3.,  4.],
       [ 4.,  5.]])

Usage with numpy functions

>>> arr = rolling(np.array([1, 2, 3, 4, 5]), 2, as_array=True)
>>> np.sum(arr, axis=1)
array([nan,  3.,  5.,  7.,  9.])

numpy_ext.rolling_apply(func: Callable, window: int, *arrays: numpy.ndarray, prepend_nans: bool = True, n_jobs: int = 1, **kwargs) → numpy.ndarray[source]¶

Roll a fixed-width window over an array or a group of arrays, producing slices. Apply a function to each slice / group of slices, transforming them into a value. Perform computations in parallel, optionally. Return a new np.ndarray with the resulting values.

Parameters:	funcCallable The function to apply to each slice or a group of slices. windowint Window size. arrayslist List of input arrays. prepend_nansbool Specifies if nans should be prepended to the resulting array n_jobsint, optional Parallel tasks count for joblib. If 1, joblib won’t be used. Default is 1. *kwargsdict Input parameters (passed to func, must be named).
Returns:	np.ndarray

Examples

>>> arr = np.array([1, 2, 3, 4, 5])
>>> rolling_apply(sum, 2, arr)
array([nan,  3.,  5.,  7.,  9.])
>>> arr2 = np.array([1.5, 2.5, 3.5, 4.5, 5.5])
>>> func = lambda a1, a2, k: (sum(a1) + max(a2)) * k
>>> rolling_apply(func, 2, arr, arr2, k=-1)
array([  nan,  -5.5,  -8.5, -11.5, -14.5])

numpy_ext

This Page

Show Source

numpy_ext¶

Installation¶

Window operations¶

Operations with nans¶

Others¶

Functions¶

Table of Contents

This Page