Convolution 2D

z^l_{i,j} = \sum_m \sum_n{a^{l-1}_{i+m,j+n} (\omega')^l_{m,n} + b^l_{i,j}}

A^l = \sigma (Z^l) = \sigma ( A^{l-1} * W^l + B^l )

Convolution

K, K' \isin \R^{M \times N} \quad K(m, n) = K'(M-1-m, N-1-n)

\begin{aligned} ( I * K )_{ij} &= \sum_m \sum_n {I(i+m, j+n)K(M-1-m, N-1-n)} \\ &= \sum_m \sum_n {I(i+m, j+n)K'(m, n)} &= (I \otimes K')_{ij} \end{aligned}

info

CNN 설명에서, 대부분의 그림은 cross-correlation $K'$ 을 나타내는 것 같습니다.

K' = \begin{bmatrix}9 & 8 & 7 \\ 6 & 5 & 4 \\ 3 & 2 & 1\end{bmatrix} \quad K = \begin{bmatrix}1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9\end{bmatrix}

in_channels=1,
out_channels(filters)=1,
kernel_size=3,
stride=1,
padding=1,
dilation=1,
bias=False

in_channels=1,
out_channels(filters)=1,
kernel_size=3,
stride=2,
padding=0,
dilation=1,
bias=False

in_channels=1,
out_channels(filters)=1,
kernel_size=3,
stride=1,
padding=0,
dilation=2,
bias=False

in_channels=2,
out_channels(filters)=3,
kernel_size=3,
stride=1,
padding=0,
dilation=1,
bias=True

\delta^l_{i,j} \equiv \frac{\partial Loss}{\partial z^l_{i,j}}

\begin{aligned} \delta^l_{i,j} = \frac{\partial Loss}{\partial z^l_{i,j}} &= \sum_x \sum_y { \frac { \partial Loss } { \partial z^{l+1}_{x,y} } \frac { \partial z^{l+1}_{x,y} } { \partial z^l_{i,j} } } \\ &= \sum_x \sum_y { \delta^{l+1}_{x,y} (\omega')^{l+1}_{i-x,j-y} \sigma' (z^l_{i,j}) } \\ &= \sum_m \sum_n { \delta^{l+1}_{i-m,j-n} (\omega')^{l+1}_{m,n} \sigma' (z^l_{i,j}) } \end{aligned}

\begin{aligned} \frac {\partial Loss} {\partial (\omega')^l_{m,n}} &= \sum_i \sum_j { \frac { \partial Loss } { \partial z^{l}_{i,j} } \frac { \partial z^{l}_{i,j} } { \partial ( \omega' )^l_{m,n} } } \\ &= \sum_i \sum_j { \delta^l_{i,j} a^{l-1}_{i+m, j+n} } \end{aligned}

\begin{aligned} \frac {\partial Loss} {\partial b^l_{i,j}} &= \sum_x \sum_y { \frac { \partial Loss } { \partial z^{l}_{x,y} } \frac { \partial z^{l}_{x,y} } { \partial b^l_{i,j} } } \\ & = \delta^l_{i,j} \end{aligned}