Short/long term memory
January 18, 2007 10:37am CST
Why some things stay in long term memory & some in short? whats the reason?
19 Jan 07
i thought it was something to do with the brain , there are two parts to your brain and often if someone has a head injury after an accident they damage the part of the brain which controlls the short term memory or the part of brain which controls long term memory
18 Jan 07
In a recurrent network, information is stored in two distinct ways. The activations of the units are a function of the recent history of the model, and so form a short-term memory. The weights too form a memory, as they are modified based on experience, but the timescale of the weight change is much slower than that of the activations. We call those a long-term memory. The Long Short-Term Memory model  is an attempt to allow the unit activations to retain important information over a much longer period of time than the 10 to 12 time steps which is the limit of RTRL or BPTT models.