tutorial.recurrent_neural_networks.elman_net

Posted by Jek Stikels

Copy the project to Neural Network Console Cloud

Classify 4, 9 digits using elman net with attention.

Treat Y axis of image as time series.

dataset-require=MNIST

Dataset : Training

  • Number of data : 1,500
  • Variable : x (image)
  • Type : Image
  • Shape : 1, 28, 28
  • Variable : y (9)
  • Type : Scalar

Examples of variable x in “Training”

Dataset : Validation

  • Number of data : 500
  • Variable : x (image)
  • Type : Image
  • Shape : 1, 28, 28
  • Variable : y (9)
  • Type : Scalar

Examples of variable x in “Validation”

Network Architecture : Main

Type Value
Output 6,303
CostParameter 44,717
CostAdd 2,354
CostMultiply 0
CostMultiplyAdd 43,932
CostDivision 785
CostExp 785
CostIf 0

Training Procedure : Optimizer

Optimize network “Main” using “Training” dataset.

  • Batch size : 64
  • Solver : Adam
    • Learning rate(Alpha) : 0.001
    • Beta1 : 0.9
    • Beta2 : 0.999
    • Epsilon : 1e-08
  • Weight decay is not applied.

Experimental Result : Learning Curve

Experimental Result : Evaluation

Evaluate network “MainRuntime” using “Validation” dataset.

Variable : y

  • Accuracy : 0.92
  • Avg.Precision : 0.9202420594262295
  • Avg.Recall : 0.92
  • Avg.F-Measures : 0.9199884783408812

References

  • Sony Corporation. Neural Network Console : Not just train and evaluate. You can design neural networks with fast and intuitive GUI. https://dl.sony.com/
  • Sony Corporation. Neural Network Libraries : An open source software to make research, development and implementation of neural network more efficient. https://nnabla.org/
  • Adam – Kingma and Ba, Adam: A Method for Stochastic Optimization. https://arxiv.org/abs/1412.6980