- LSTM
\[I_t=\sigma(X_tW_{xi}+H_{t-1}W_{hi}+b_i)\\
F_t=\sigma(X_tW_{xf}+H_{t-1}W_{hf}+b_f)\\
O_t=\sigma(X_tW_{xo}+H_{t-1}W_{ho}+b_o)\\
\bar{C_t}=tanh(X_tW_{xc}+H_{t-1}W_{hc}+b_c)\\
C_t=F_t\odot{C_{t-1}}+I_t\odot{\bar{C_t}}\\
H_t=O_t\odot{tanh(C_t)}
\]
- GRU
\[R_t=\sigma(X_tW_{xr}+H_{t-1}W_{hr}+b_r)\\
Z_t=\sigma(X_tW_{xz}+H_{t-1}W_{hz}+b_z)\\
\bar{H_{t}}=tanh(X_tW_{xh}+(R_t\odot{H_{t-1}})W_{hh}+b_h)\\
H_t=Z_t\odot{H_{t-1}}+(1-Z_t)\odot{\bar{H_t}}
\]