ligma_recognition
Posted by amarc1994
Copy the project to Neural Network Console Cloud
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan, Andrew Zisserman
https://arxiv.org/abs/1409.1556
Dataset : Training
- Number of data : 10,000
- Variable : x (image)
- Type : Image
- Shape : 1, 112, 112
- Variable : y (label)
- Type : Matrix
- Shape : 35, 7
- Variable : r (region)
- Type : Matrix
- Shape : 35, 28
Examples of variable x in “Training”
Dataset : Validation
- Number of data : 1,500
- Variable : x (image)
- Type : Image
- Shape : 1, 28, 28
- Variable : y (9)
- Type : Scalar
Examples of variable x in “Validation”
Dataset : Dataset
- Number of data : 60,000
- Variable : x (image)
- Type : Image
- Shape : 1, 28, 28
- Variable : y (label)
- Type : Scalar
Examples of variable x in “Dataset”
Dataset : Dataset_1
- Number of data : 50,000
- Variable : x (image)
- Type : Image
- Shape : 3, 32, 32
- Variable : y (label)
- Type : Scalar
Examples of variable x in “Dataset_1”
Dataset : Dataset_2
- Number of data : 20
- Variable : x (in)
- Type : Image
- Shape : 1, 64, 64
- Variable : y (out)
- Type : Image
- Shape : 1, 64, 64
Examples of variable x, y in “Dataset_2”
Network Architecture : VGG13Train
| Type | Value |
|---|---|
| Output | 26,494,416 |
| CostParameter | 133,047,848 |
| CostAdd | 12,403,664 |
| CostMultiply | 158,720 |
| CostMultiplyAdd | 11,308,466,176 |
| CostDivision | 1,000 |
| CostExp | 1,000 |
| CostIf | 18,372,608 |
Network Architecture : Training
| Type | Value |
|---|---|
| Output | 844,298 |
| CostParameter | 133,047,848 |
| CostAdd | 12,403,664 |
| CostMultiply | 158,721 |
| CostMultiplyAdd | 11,308,466,176 |
| CostDivision | 1,000 |
| CostExp | 1,000 |
| CostIf | 18,372,608 |
Network Architecture : Top1Error
| Type | Value |
|---|---|
| Output | 457,728 |
| CostParameter | 0 |
| CostAdd | 0 |
| CostMultiply | 0 |
| CostMultiplyAdd | 0 |
| CostDivision | 0 |
| CostExp | 0 |
| CostIf | 0 |
Network Architecture : Top5Error
| Type | Value |
|---|---|
| Output | 457,728 |
| CostParameter | 0 |
| CostAdd | 0 |
| CostMultiply | 0 |
| CostMultiplyAdd | 0 |
| CostDivision | 0 |
| CostExp | 0 |
| CostIf | 0 |
Network Architecture : Runtime
| Type | Value |
|---|---|
| Output | 457,728 |
| CostParameter | 0 |
| CostAdd | 0 |
| CostMultiply | 0 |
| CostMultiplyAdd | 0 |
| CostDivision | 0 |
| CostExp | 0 |
| CostIf | 0 |
Network Architecture : VGG13
| Type | Value |
|---|---|
| Output | 27,289,040 |
| CostParameter | 133,047,848 |
| CostAdd | 12,403,664 |
| CostMultiply | 150,528 |
| CostMultiplyAdd | 11,308,466,176 |
| CostDivision | 1,000 |
| CostExp | 1,000 |
| CostIf | 18,372,608 |
Training Procedure : Optimizer
Optimize network “Training” using “Training” dataset.
- Batch size : 128
- by accumulating the result of batch-size 32 by 4 times.
- Solver : Momentum
- Learning rate: 1e-05
- decayed every 15 iteration using an exponential rate of 0.1 .
- Momentum : 0.9
- Weight decay : 0.0005
Training Procedure : Optimizer_1
Optimize network “Main” using “Training” dataset.
- Batch size : 32
- Solver : Adam
- Learning rate(Alpha) : 0.001
- Beta1 : 0.9
- Beta2 : 0.999
- Epsilon : 1e-08
- Weight decay is not applied.
References
- Sony Corporation. Neural Network Console : Not just train and evaluate. You can design neural networks with fast and intuitive GUI. https://dl.sony.com/
- Sony Corporation. Neural Network Libraries : An open source software to make research, development and implementation of neural network more efficient. https://nnabla.org/
- Convolution – Chen et al., DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. https://arxiv.org/abs/1606.00915, Yu et al., Multi-Scale Context Aggregation by Dilated Convolutions. https://arxiv.org/abs/1511.07122
- ReLU – Vinod Nair, Geoffrey E. Hinton. Rectified Linear Units Improve Restricted Boltzmann Machines. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.6419&rep=rep1&type=pdf
- Momentum – Ning Qian : On the Momentum Term in Gradient Descent Learning Algorithms. http://www.columbia.edu/~nq6/publications/momentum.pdf
- Adam – Kingma and Ba, Adam: A Method for Stochastic Optimization. https://arxiv.org/abs/1412.6980