Hmmm...I am not sure I follow...
the error is simply (- target actual). Then you take the square of that
difference (2 reasons for that) and there you have the error per neuron
at the output layer. Then you simply add them up to get the entire
network's error. so far so good...
the tricky bit comes now when you need to find out how much the error
depends on the output, inputs and weights so you can feed that in to the
gradient descent formula via which the weights will be adjusted.
Basically gradient descent says this:
"the adjustment of each weight will be the negative of a constant theta
multiplied by the dependence of the previous weight on the error of the
network, which is the derivative of that error with respect to that weight."
so essentially you're only looking to find the derivative of E (total
error) with respect to Wij , but in order to find that you will need to know
* how much the error depends on the output
* how much the output depends on the activation (which depends on the
weights)
It is not straight forward to type formulas here but I've put a relevant
document that describes the procedure exactly and the encog book which i
cannot guarantee it goes that deep, in my public dropbox for you. the
link is :https://dl.dropbox.com/u/45723414/for_Tim.tar.gz ...I'll leave
it up until tomorrow...
hope that helps...
Jim
ps: the encog source did not help you?
On 01/10/12 19:41, Timothy Washington wrote:
Hey, thanks for responding. See responses inlined.
On Mon, Oct 1, 2012 at 4:24 AM, Jim - FooBar(); <jimpil1...@gmail.com
<mailto:jimpil1...@gmail.com>> wrote:
I've always found this page very good :
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
<http://www.doc.ic.ac.uk/%7End/surprise_96/journal/vol4/cs11/report.html>
generally, the weight of a connection is adjusted by an amount
proportional to the product of an error signal δ, on the unit k
receiving the input and the output of the unit j sending this
signal along the connection. You use the chain rule to calculate
partial derivatives.
also this looks quite detailed with regards to what you're asking:
https://docs.google.com/viewer?a=v&q=cache:9ahoWLbap8AJ:www.cedar.buffalo.edu/~srihari/CSE574/Chap5/Chap5.3-BackProp.pdf+the+partial+derivative+of+the+error+neural+nets&hl=en&gl=uk&pid=bl&srcid=ADGEEShVr8tHNIryQDGp0jJ2xv6JM40Ja4UFbcr2-QDgU1tK4Di_4mUTVgWsHdjHphbS7MaHmSn8VwB3dEnAQz-w4J-iyF1VJFklvLoDY5fxJbGaHsU0mio7XQYmGom0_cd7aZkUzhq5&sig=AHIEtbTyaLysuBdK7Bn_-cMzE88tqOzlag
<https://docs.google.com/viewer?a=v&q=cache:9ahoWLbap8AJ:www.cedar.buffalo.edu/%7Esrihari/CSE574/Chap5/Chap5.3-BackProp.pdf+the+partial+derivative+of+the+error+neural+nets&hl=en&gl=uk&pid=bl&srcid=ADGEEShVr8tHNIryQDGp0jJ2xv6JM40Ja4UFbcr2-QDgU1tK4Di_4mUTVgWsHdjHphbS7MaHmSn8VwB3dEnAQz-w4J-iyF1VJFklvLoDY5fxJbGaHsU0mio7XQYmGom0_cd7aZkUzhq5&sig=AHIEtbTyaLysuBdK7Bn_-cMzE88tqOzlag>
Looking at page 9 of this document, I take the error derivative
calculation to be:
Thus required derivative */∂En/∂w ji/* is obtained by...
1. Multiplying value of δ ( for the unit at output end of weight )
2. by value of z ( for unit at input end of weight )
So take this output neuron and it's input weights. Let's say the total
neural network error is *-0.3963277746392987*. I just multiply that
total error by each weight, to get the bolded, green error values. So
what you're saying is that the error derivative */∂En/∂w ji /*( or
weight change) for each input is that same error value, below in
green. Is this the case?
:output-layer
({:calculated-error -1.1139741279964241,
:calculated-value 0.9275622253607013,
:inputs
({:error */-0.2016795955938916/*,
:calculated 0.48962608882549025,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ffb",
:weight 0.5088707087900713,
:bias 0}
{:error */-0.15359996014735702/*,
:calculated 0.3095962076691644,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ffa",
:weight 0.38755790024342773,
:bias 0}
{:error */-0.11659507401745359/*
:calculated 0.23938733624830652,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff9",
:weight 0.2941885012312543,
:bias 0}
{:error */-0.2784739949663631/*,
:calculated 0.6681581686752845,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff8",
:weight 0.7026355778870271,
:bias 0}
{:error */-0.36362550327135884/*,
:calculated 0.8430641676611533,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff7",
:weight 0.9174868039523537,
:bias 0}),
:id "583c10bfdbd326ba525bda5d13a0a894b947ff6"})
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient
with your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
