NumPy broadcasting (广播) 的计算过程

NumPy broadcasting (广播) 的计算过程

NumPy documentation
https://numpy.org/doc/stable/index.html

NumPy 中文
https://www.numpy.org.cn/

Python for Data Analysis, 3E
https://wesmckinney.com/book/

1. Broadcasting (广播)

https://numpy.org/doc/stable/user/basics.broadcasting.html
https://www.numpy.org.cn/user/basics/broadcasting.html

The term broadcasting describes how NumPy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, the smaller array is “broadcast” across the larger array so that they have compatible shapes. Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python. It does this without making needless copies of data and usually leads to efficient algorithm implementations. There are, however, cases where broadcasting is a bad idea because it leads to inefficient use of memory that slows computation.
术语广播 (broadcasting) 描述了 NumPy 如何在算术运算期间处理具有不同形状的数组。受某些约束的影响，较小的数组沿着较大的数组广播，以便它们具有兼容的形状。广播提供了一种向量化数组操作的方法，因此循环发生在 C 而不是 Python 中。它在不执行不必要的数据复制的情况下执行此操作，并且通常会导致高效的算法实现。然而，在某些情况下，广播不是一个好主意，因为它会导致内存使用效率低下，从而减慢计算速度。

NumPy 执行 broadcasting 的前提是，两个 arrays 执行的是 element-wise 的运算。

NumPy operations are usually done on pairs of arrays on an element-by-element basis. In the simplest case, the two arrays must have exactly the same shape, as in the following example:
NumPy 操作通常在逐个元素的基础上对数组对进行。在最简单的情况下，两个数组必须具有完全相同的形状，如下例所示：

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.array([1.0, 2.0, 3.0])
>>> a
array([1., 2., 3.])
>>> a.shape
(3,)
>>>
>>> b = np.array([2.0, 2.0, 2.0])
>>> b
array([2., 2., 2.])
>>> b.shape
(3,)
>>>
>>> c = a * b
>>> c
array([2., 4., 6.])
>>> c.shape
(3,)
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

 

  
 

arithmetic [.ærɪθ'metɪk]：n. 算术，算术运算，四则运算

NumPy’s broadcasting rule relaxes this constraint when the arrays’ shapes meet certain constraints. The simplest broadcasting example occurs when an array and a scalar value are combined in an operation:
当数组的形状满足某些约束时，NumPy 的广播规则放宽了这个约束。最简单的广播示例发生在数组和标量值组合出现在一个操作中：

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.array([1.0, 2.0, 3.0])
>>> b = 2.0
>>>
>>> c = a * b
>>> c
array([2., 4., 6.])
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

We can think of the scalar b being stretched during the arithmetic operation into an array with the same shape as a. The new elements in b, as shown in Figure 1, are simply copies of the original scalar. The stretching analogy is only conceptual. NumPy is smart enough to use the original scalar value without actually making copies so that broadcasting operations are as memory and computationally efficient as possible.
我们可以认为标量 b 在算术运算期间被拉伸成一个与 a 形状相同的数组。b 中的新元素，如图 1 所示，只是原始标量的副本。拉伸类比只是概念上的。NumPy 足够聪明，可以使用原始标量值而无需实际制作副本，因此广播操作的内存和计算效率尽可能高。

stretch [stretʃ]：v. 伸展，延伸，伸出，舒展 n. 伸展，弹性，舒展，一片 adj. 有弹力的
analogy [ə'nælədʒi]：n. 类比，类推，比拟，相似之处
conceptual [kən'septʃuəl]：adj. 概念 (上) 的，观念 (上) 的

Figure 1 In the simplest example of broadcasting, the scalar b is stretched to become an array of same shape as a so the shapes are compatible for element-by-element multiplication.

The code in the second example is more efficient than that in the first because broadcasting moves less memory around during the multiplication (b is a scalar rather than an array).
第二个示例中的代码比第一个示例中的代码更有效，因为广播在乘法期间移动的内存较少 (b 是标量而不是数组)。

1.1 General Broadcasting Rules

When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimensions and works its way left. Two dimensions are compatible when

• they are equal, or
• one of them is 1
  当对两个数组进行操作时，NumPy 按元素比较它们的形状。它从尾部 (即最右边) 的维度开始，并向左移动。当他们是相等的，或者其中一个是 1 时，两个维度兼容。

从末尾开始对比两个数组的维度，必须相同或者为 1。

1. All input arrays with ndim smaller than the input array of largest ndim, have 1’s prepended to their shapes.
   所有输入数组的轴数量都向其中轴数量最多的输入数组看齐，轴数量不足的部分都通过在前面 (左侧) 用 1 补齐。轴数量是 the number of dimensions。a.shape = (2, 2, 3)、b.shape = (2, 3)，计算 a + b 时，数组 b 将被补齐为 b.shape=(1, 2, 3)。

2. The size in each dimension of the output shape is the maximum of all the input sizes in that dimension.
   输出数组的维度是所有输入数组的维度进行右对齐，每个位置上维度的最大值。

3. An input can be used in the calculation if its size in a particular dimension either matches the output size in that dimension, or has value exactly 1.
   如果输入数组的某个轴和输出数组的对应轴的长度相同或者其长度为 1 时，这个数组能够用来计算，否则出错。

4. If an input has a dimension size of 1 in its shape, the first data entry in that dimension will be used for all calculations along that dimension. In other words, the stepping machinery of the ufunc will simply not step along that dimension (the stride will be 0 for that dimension).
   如果输入数组的某一个维度大小为 1，则该维度中的第一个数据条目将用于沿该维度的所有计算。换句话说，ufunc 的步进机制将不会沿着该维度步进 (该维度的步幅将为 0)。

prepend：预先考虑，预置，前置，预追加，预先准备
machinery [məˈʃiːnəri]：n. 机构，机器，大型机器，机器的运转部分

If these conditions are not met, a ValueError: operands could not be broadcast together exception is thrown, indicating that the arrays have incompatible shapes. The size of the resulting array is the size that is not 1 along each axis of the inputs.
如果不满足这些条件，则抛出 ValueError: operands could not be broadcast together 异常，指示数组具有不兼容的形状。结果数组的大小是沿输入的每个轴不为 1 的大小。

Arrays do not need to have the same number of dimensions. For example, if you have a 256x256x3 array of RGB values, and you want to scale each color in the image by a different value, you can multiply the image by a one-dimensional array with 3 values. Lining up the sizes of the trailing axes of these arrays according to the broadcast rules, shows that they are compatible:
数组不需要具有相同数量的维度。例如，如果您有一个 256x256x3 RGB 值数组，并且希望将图像中的每种颜色缩放不同的值，则可以将图像乘以具有 3 个值的一维数组。根据广播规则排列这些数组的尾轴的大小，表明它们是兼容的：

Image  (3d array): 256 x 256 x 3
Scale  (1d array):             3
Result (3d array): 256 x 256 x 3

When either of the dimensions compared is one, the other is used. In other words, dimensions with size 1 are stretched or “copied” to match the other.
当比较的任何一个维度为 1 时，使用另一个维度。换句话说，大小为 1 的维度被拉伸或复制以匹配另一个。

rightmost ['raɪtməʊst]：adj. 最右边的，最右面的

In the following example, both the A and B arrays have axes with length one that are expanded to a larger size during the broadcast operation:
在以下示例中，A 和 B 数组都具有长度为 1 的轴，在广播操作期间会扩展为更大的大小：

A      (4d array):  8 x 1 x 6 x 1
B      (3d array):      7 x 1 x 5
Result (4d array):  8 x 7 x 6 x 5

1.2 Broadcastable arrays

Two arrays are compatible for broadcasting if for each trailing dimension (that is, starting from the end), the axis lengths match or if either of the lengths is 1. Broadcasting is then performed over the missing and / or length 1 dimensions.
如果对于每个 trailing dimension (即，从末尾开始)，轴长度相等，或者如果任一长度为 1，则两个数组时兼容广播。然后在缺失和/或长度为 1 的维度上执行广播。

For example, if a.shape is (5,1), b.shape is (1,6), c.shape is (6,) and d.shape is () so that d is a scalar, then a, b, c, and d are all broadcastable to dimension (5,6); and

• a acts like a (5,6) array where a[:,0] is broadcast to the other columns,
• b acts like a (5,6) array where b[0,:] is broadcast to the other rows,
• c acts like a (1,6) array and therefore like a (5,6) array where c[:] is broadcast to every row, and finally,
• d acts like a (5,6) array where the single value is repeated.

A      (2d array):  5 x 4
B      (1d array):      1
Result (2d array):  5 x 4

A      (2d array):  5 x 4
B      (1d array):      4
Result (2d array):  5 x 4

A      (3d array):  15 x 3 x 5
B      (3d array):  15 x 1 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 1
Result (3d array):  15 x 3 x 5

 

  
 

Here are examples of shapes that do not broadcast:

A      (1d array):  3
B      (1d array):  4 # trailing dimensions do not match

A      (2d array):      2 x 1
B      (3d array):  8 x 4 x 3 # second from last dimensions mismatched

An example of broadcasting when a 1-d array is added to a 2-d array:

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.array([[ 0.0,  0.0,  0.0],
...               [10.0, 10.0, 10.0],
...               [20.0, 20.0, 20.0],
...               [30.0, 30.0, 30.0]])
>>>
>>> b = np.array([1.0, 2.0, 3.0])
>>>
>>> a + b
array([[ 1.,  2.,  3.],
       [11., 12., 13.],
       [21., 22., 23.],
       [31., 32., 33.]])
>>>
>>> b = np.array([1.0, 2.0, 3.0, 4.0])
>>> a + b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: operands could not be broadcast together with shapes (4,3) (4,)
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

 

  
 

As shown in Figure 2, b is added to each row of a. In Figure 3, an exception is raised because of the incompatible shapes.

Figure 2 A one dimensional array added to a two dimensional array results in broadcasting if number of 1-d array elements matches the number of 2-d array columns.

Figure 3 When the trailing dimensions of the arrays are unequal, broadcasting fails because it is impossible to align the values in the rows of the 1st array with the elements of the 2nd arrays for element-by-element addition.
当数组的尾部维度不相等时，广播将失败，因为不可能将第一个数组的行中的值与第二个数组的元素对齐以进行逐元素添加。

Broadcasting provides a convenient way of taking the outer product (or any other outer operation) of two arrays. The following example shows an outer addition operation of two 1-d arrays:

Figure 4 In some cases, broadcasting stretches both arrays to form an output array larger than either of the initial arrays.
在某些情况下，广播会拉伸两个数组以形成比任一初始数组都大的输出数组。

Here the newaxis index operator inserts a new axis into a, making it a two-dimensional 4x1 array. Combining the 4x1 array with b, which has shape (3,), yields a 4x3 array.
这里 newaxis 索引操作符插入一个新轴 a ，使其成为一个二维 4x1 数组。将 4x1 数组与形状为 (3,) 的 b 组合，产生一个 4x3 数组。

Broadcasting is a powerful tool for writing short and usually intuitive code that does its computations very efficiently in C. However, there are cases when broadcasting uses unnecessarily large amounts of memory for a particular algorithm. In these cases, it is better to write the algorithm’s outer loop in Python. This may also produce more readable code, as algorithms that use broadcasting tend to become more difficult to interpret as the number of dimensions in the broadcast increases.
广播是一种强大的工具，用于编写简短且通常直观的代码，这些代码可以在 C 中非常有效地进行计算。但是，在某些情况下，广播会为特定算法使用不必要的大量内存。在这些情况下，最好用 Python 编写算法的外层循环。这也可能会产生更易读的代码，因为随着广播中维数的增加，使用广播的算法往往会变得更难解释。

intuitive [ɪnˈtjuːətɪv]：adj. 直觉的，直观的

2. NumPy three-dimensional broadcasting

In the three-dimensional case, broadcasting over any of the three dimensions is only a matter of reshaping the data to be shape compatible.

Figure 5 Broadcasting over axis 0 of a 3D array.

Figure 6 Compatible 2D array shapes for broadcasting over a 3D array.

Figure 7 NumPy Broadcasting.

References

https://yongqiang.blog.csdn.net/
https://data-flair.training/blogs/numpy-broadcasting/

转载：https://blog.csdn.net/chengyq116/article/details/128325485