Protein secondary-structure forward pass — toy vanilla RNN
Pred:
Reference:
Helix (H)
Strand (E)
Coil (C)
vanilla RNN cell
h
t-1
W_h
recurrent mix
e_t = φ(aa_t)
W_x
input mix
+
z_t = W_h h_(t-1) + W_x e_t + b_h
tanh
h_t
W_y
output head
logits_t = W_y h_t + b_y
softmax(logits_t)
H
E
C
argmax
Hidden state h_t (d_h = 16):
input encoding e_t / aa_t:
[0,0,0,...,0]
phi(aa_t): [...]
h_(t-1) (16-dim):
[0.00, 0.00, 0.00, 0.00,
0.00, 0.00, 0.00, 0.00,
0.00, 0.00, ...0.00]
h_t (16-dim):
[0.00, 0.00, 0.00, 0.00,
0.00, 0.00, 0.00, 0.00,
0.00, 0.00, ...0.00]
logits / probs:
[0.00, 0.00, 0.00]
softmax: [.33,.33,.33]
Amino-acid sequence: