Gated Recurrent Units

$$$$r_t == sigma(W**r(x_t, h_(t - 1)) + b**r)

z_t == sigma(W**z(x_t, h_(t - 1)) + b**z)

0.3cm h__deriv_t == tanh(W**h(x_t, r_t odot h_(t - 1)) + b**h)

h_t == z_t odot h_(t - 1) +(1 - z_t) odot h__deriv_t

Gated Recurrent Units

$$$$h_t == GRU

Gated Recurrent Units

$$$$r_t == sigma(W**r(x_t, h_(t - 1)) + b**r)

z_t == sigma(W**z(x_t, h_(t - 1)) + b**z)

0.3cm h__deriv_t == tanh(W**h(x_t, r_t odot h_(t - 1)) + b**h)

h_t == z_t odot h_(t - 1) +(1 - z_t) odot h__deriv_t

Gated Recurrent Units

$$$$r_t == sigma(W**r(x_t, h_(t - 1)) + b**r)

z_t == sigma(W**z(x_t, h_(t - 1)) + b**z)

0.3cm h__deriv_t == tanh(W**h(x_t, r_t odot h_(t - 1)) + b**h)

h_t == z_t odot h_(t - 1) +(1 - z_t) odot h__deriv_t

Gated Recurrent Units

$$$$h_t == GRU

Gated Recurrent Units

$$$$r_t == sigma(W**r(x_t, h_(t - 1)) + b**r)

z_t == sigma(W**z(x_t, h_(t - 1)) + b**z)

0.3cm h__deriv_t == tanh(W**h(x_t, r_t odot h_(t - 1)) + b**h)

h_t == z_t odot h_(t - 1) +(1 - z_t) odot h__deriv_t