Skip to content

Commit

Permalink
Update nnbp.m
Browse files Browse the repository at this point in the history 
added rectified linear and softplus activation functions
  • Loading branch information
@ericstrobl
ericstrobl committed Mar 4, 2014
1 parent c083531 commit 6133e2f
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions NN/nnbp.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@
d_act = nn.a{i} .* (1 - nn.a{i});
case 'tanh_opt'
d_act = 1.7159 * 2/3 * (1 - 1/(1.7159)^2 * nn.a{i}.^2);
case 'relu'
d_act = nn.a{i};
d_act(find(d_act>0))=1; d_act(find(d_act<=0))=0;
case 'softp'
d_act = 1./(1+exp(-nn.a{i}));
end

if(nn.nonSparsityPenalty>0)
Expand Down

1 comment on commit 6133e2f

@happynear
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in relu's bp, just set d_act = (nn.a{i} > 0);

Please sign in to comment.