Graph Converters

class nnabla.experimental.graph_converters.GraphConverter(modifiers=[])

Convert a graph with the modifiers by traversing from output variables.

convert(o)

Parameters:

o (list of nnabla.Variable) – Output variables.

class nnabla.experimental.graph_converters.FunctionModifier

Base class of modifiers.

The modify method is called for a function with inputs in a graph topological order when you call the GraphConverter(<modifiers>).convert(<root variable>) method.

finish_up()

Finish the very time function modification.

Clean up the internal modifier states.

Parameters:

None –

Returns:

None

get_parameter_scope(v)

Get the parameter name corresponding to v

Parameters:

v (nnabla.Variable) – NNabla Variable Object.

Returns:

Scope name

Return type:

str

modify(f, inputs)

Modify the function.

Implement this method in a sub class to modify a function.

Examples:

class ReLUToLeakyReLUModifier(FunctionModifier):

  def __init__(self):
    super(ReLUToLeakyReLUModifier, self).__init__()

  def modify(self, f, inputs):
    if f.info.type_name == 'ReLU':
      x = inputs[0]
      return F.leaky_relu(x)

This examples is a simple case since the network topological order does not change. In GraphConverter, we expect the modify method is called along the original network topological order not the modified order. In such a complex case, see themodify method of BatchNormalizationFoldingModifierInner as a reference.

Parameters:

• f (nnabla.function.Function) – NNabla function object.

• inputs (list of Variable) – New inputs to f. This may be modified one or the same as f.inputs.

Returns:

Variable or list of Variable.

Function Modifiers

class nnabla.experimental.graph_converters.BatchNormalizationFoldingModifier(opposite=False, channel_last=False)

Single Convolution -> BatchNormalization pass is folded into one Convolution.

If there is a Convolution -> BatchNormalization pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.

Supported folding functions: Convolution, Deconvolution, Affine.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.BatchNormalizationFoldingModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.AddBiasModifier

Add bias to Convolution in BatchNormalization folding case if it doesn’t have bias.

Supported folding functions: Convolution, Deconvolution, Affine.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.AddBiasModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.BatchNormalizationFoldingModifierInner(channel_last=False)

Single Convolution -> BatchNormalization pass is folded into one Convolution.

If there is a Convolution -> BatchNormalization pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.

Supported folding functions: Convolution, Deconvolution, Affine.

class nnabla.experimental.graph_converters.BatchNormalizationFoldingOppositeModifierInner(channel_last=False)

Single BatchNormalization -> Convolution pass is folded into one Convolution.

If there is a BatchNormalization -> Convolution pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.

Supported folding functions: Convolution, Deconvolution, Affine.

class nnabla.experimental.graph_converters.BatchNormalizationSelfFoldingModifier(name='bn-self-folding')

The parameters of the batch normalization replaced simple scale and bias.

Parameters:

name (str) – Prefix of the parameter scope.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.BatchNormalizationSelfFoldingModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.FusedBatchNormalizationModifier

Block BatchNormalization -> Add2 -> Non-Linear pass is fused into one FusedBatchNormalization.

If there is a block BatchNormalization -> Add2 -> Non-Linear pass, remove all the block functions and replace the whole block to FusedBatchNormalization.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.FusedBatchNormalizationModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.UnfusedBatchNormalizationModifier

Unfuse FusedBatchNormalization to BatchNormalization -> Add2 -> Non-Linear block.

If there is a FusedBatchNormalization pass, remove the fused batch normalization and replace it with the block BatchNormalization -> Add2 -> Non-Linear.

Supported Non-Linear functions: relu

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.UnfusedBatchNormalizationModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.ChannelLastModifier(inputs, inputs_cl=None)

Convert graph shape from Channel first (NCHW) to Channel last (NHWC) format.

Supported functions: Convolution, Deconvolution, BatchNormalization, MaxPooling, AveragePooling, SumPooling, Unpooling, Concatenate

Parameters:

• inputs (list of nn.Variable) – Original very beginning inputs (NCHW) of a network.

• inputs_cl (list of nn.Variable) – Channel last version of very beginning inputs (NHWC) of a network. If this is not given, inputs_cl are generated internally and holded.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.ChannelLastModifier(<inputs of pred>)]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.ChannelFirstModifier(inputs, inputs_cf=None)

Convert graph shape from Channel last (NHWC) to Channel first (NCHW) format.

Supported functions: Convolution, Deconvolution, BatchNormalization, MaxPooling, AveragePooling, SumPooling, Unpooling, Concatenate

Parameters:

• inputs (list of nn.Variable) – Original channel last version of very beginning inputs (NHWC) of a network.

• inputs_cf (list of nn.Variable) – Channel first version of very beginning inputs (NCHW) of a network. If this is not given, inputs_cf are generated internally and holded.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.ChannelFirstModifier(<inputs of pred>)]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.RemoveFunctionModifier(rm_funcs=[])

Remove specified function layer(s) from a graph.

A convenient converter when one or more functions in an existing graph needs to be removed. This converter remove specified function(s) without recreating a new graph from scratch.

Parameters:

rm_funcs (list of str) – list of function name

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.RemoveFunctionModifier(rm_funcs=['BatchNormalization', 'MulScalar'])]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.BatchNormBatchStatModifier

Change batch_stat to False. Supported functions: BatchNormalization, FusedBatchNormalization, SyncBatchNormalization.

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.BatchNormBatchStatModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.TestModeModifier(rm_funcs=[])

This converter combines BatNormBatchStateModifier and RemoveFunctionModifer. It changes batch_stat to False. Supported functions: BatchNormalization, FusedBatchNormalization, SyncBatchNormalization.

Functions that specified rm_funcs will be removed from a graph.

Parameters:

rm_funcs (list of str) – list of function name

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.TestModeModifier(rm_funcs=['MulScalar'])]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.IdentityModifier(inputs={}, copy_value=False)

All functions are replaced to the same new function.

Parameters:

inputs (dict) – Input variable mapping from the original input to another input. Default is the empty dictionary, so the new graph shares the original inputs.

Examples:

pred = Model(...)
x = nn.Variable(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.IdentityModifier({x0: x1})]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.NoGradModifier

All functions are replaced to the same new function.

Parameters:

inputs (dict) – Input variable mapping from the original input to another input. Default is the empty dictionary, so the new graph shares the original inputs.

Examples:

pred = Model(...)
x = nn.Variable(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.NoGradModifier()]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

class nnabla.experimental.graph_converters.PruningModifier(pruning_threshold, functions_to_prune=('Convolution', 'Deconvolution', 'DepthwiseConvolution', 'DepthwiseDeconvolution', 'Affine'), channel_last=False)

Use PruningModifier to prune the small weight value to 0. The pruning is channel-wise. Using the channel-wise L2 norm to represent the degree of sparsity. If the L2 norm less than the threshold provided, all the value of this channel will be set to 0.

Supported pruning functions: Convolution, Deconvolution, DepthwiseConvolution, ‘DepthwiseDeconvolution’, ‘Affine’

Examples:

pred = Model(...)

import nnabla.experimental.graph_converters as GC

modifiers = [GC.PruningModifier(pruning_threshold=0.1)]
gc = GC.GraphConverter(modifiers)
pred = gc.convert(pred)

__init__(pruning_threshold, functions_to_prune=('Convolution', 'Deconvolution', 'DepthwiseConvolution', 'DepthwiseDeconvolution', 'Affine'), channel_last=False)

Parameters:

• pruning_threshold (float) – Threshold of the L2 norm.

• functions_to_prune (list) – Functions to pruning.

• channel_last (bool) – If True, the data formnat of network is considered to be NHWC.

class nnabla.experimental.graph_converters.QuantizeNonQNNToRecordingModifier(functions_ranks, config=None, training=True)

class nnabla.experimental.graph_converters.QuantizeRecordingToTrainingModifier(functions_ranks, config=None)