The Global Config tab is used to configure and view neural network training settings.
1 Editing the project comment
Enter any comment you like using text in Project Description.
2 Specifying the number of epochs until the completion of training optimization
Enter the number of epochs in Max Epoch.
To simply output an untrained designed network with the parameter properties (e.g., Weight, Bias) specified in all layers that have parameters, set Max Epoch to 0.
3 Specifying the conditions for saving a neural network from the training results
To save the neural network of the last epoch, clear the Save Best check box. To save the neural network of the Epoch with the smallest error in the validation dataset, select the Save Best check box.
4 Specifying the batch size to use when optimizing using a mini-batch
Enter the number of data samples to use for each parameter update in Batch Size.
Notes
The batch size value must be set to a value less than the number of data samples contained in each dataset used for training.
5 Specifying the calculation precision used for evaluation during training
Set Precision to one of the following two options.
| Item | Description |
| Float | Calculation is performed using 32-bit floating-point numbers. |
| Half | Calculation is basically performed using 16-bit floating-point numbers, but a portion of it is performed using 32-bit floating-point numbers (Mixed Precision). Selecting Half is advantageous in that the amount of used memory is reduced, allowing larger networks to be trained, and that the calculation speed is increased when a NVIDIA VOLTA generation GPU or later is used. |
Notes
Half can be specified only when an NVIDIA Pascal generation GPU or later is in use.
Note when Half is specified, training may not progress properly depending on the network structure. In such cases, it may be possible to obtain the same performance as when training is performed using Float by applying Loss Scaling.
To perform loss scaling on Neural Network Console, scale the loss value with a MulScaler layer inserted after the loss function. If Sgd, Momentum, Nesterov, or other Updater that does not automatically adjust the learning rate is in use, set the learning rate to 1/scale.
6 Specifying the frequency of performing accuracy evaluation
Specify the accuracy evaluation interval in Monitor Interval in unit of epoch. If the default value of 10 is specified, accuracy evaluation is performed every 10 epochs. To perform accuracy evaluation every epoch, set this to 1.
Reference
During epoch 1 to 5, accuracy evaluation is performed every epoch regardless of the Monitor Interval value.
7 Setting the automatic optimization of network structure to on or off
Click the Enable check box under Structure Search to set the automatic optimization of network structure to on or off.
When training is executed with Structure Search set to Enable, optimization in which the network structure is automatically varied to find network structures with a higher accuracy and less amount of computation is performed.
Notes
When training is executed with Structure Search set to Enable, training continues indefinitely unless the user explicitly stops it such as by pressing the Stop Training button.
8 Selecting the automatic network structure optimization algorithm
Set Method under Structure Search to one of the following two options.
| Item | Description |
| Random | Automatic network structure search using evolutionary computation is performed. In exchange for trying various network structures, optimization takes more time. |
| Network Feature + Gaussian Process | Automatic network structure search using the network features and Bayesian optimization based on Gaussian process is performed. This allows better networks to be found more quickly. |
9 Selecting what to optimize in the automatic network structure optimization
Set Optimize for under Structure Search to one of the following two options.
| Item | Description |
| Error | A network with less error is searched for. Specify this when you want to search for the highest performance network regardless of the amount of computation. |
| Error and Calculation | A network with less error and less amount of computation is searched for. |
Notes
If Optimize for is set to Error, we recommend that you limit the search range of the number of multiplications and additions using the method explained below. If you do not specify a limit, a network with extremely large amount of computation will be searched for, and optimization may take a long time.
10 Limiting the search range based on the accuracy or the number of multiplications and additions of the validation dataset during automatic network structure optimization
Under Search Range, set the Min (minimum) and Max (maximum) values for Validation (validation dataset error) and Multiply Add (the number of multiplications and additions). To not specify the minimum and maximum limits, delete the values so that the boxes are empty.
For example, if you set Validation Max to 0.05 and Multiply Add Max to 2000000, automatic structure search is performed in a range in which the validation dataset error does not exceed 0.05 and the Multiply Add range does not exceed 2000000. (The validation dataset error may exceed the range in the final result, but training results that exceed the range are not used for searching).
11 Stopping searches that performance improvement cannot be expected during automatic network structure optimization
Select the Early Stopping check box. Then, set Time Limit to the maximum amount of time that will be spent in a single network evaluation in the days:hours:minutes:seconds format (2-digit numbers each). To not specify the time limit, delete the Time Limit values so that the boxes are empty.
If the Early Stopping check box is selected, the learning curve is compared to the result of past training that used a network structure with less multiplications and additions than the network structure being searched for. If the number of errors is twice as large or greater in the same epoch, the training is stopped at that point, and the evaluation of the next structure begins.
If a time limit is specified, when the time in evaluating a network exceeds the specified time, the training is stopped at that point, and the evaluation of the next structure begins.
