operators¶
Module webdnn.graph.operators
contains WebDNN Operator IR classes.
All classes inherit Operator
base class.
All operator class instances are callable. When it is called, input and output variables are registered with corresponding names. In this document, the registered name of each operator is described in Signature section.
For almost of all users, it is not need to understand this section. If you want to implement your custom converter/generator handler, this section may be helpful.
Abs¶
Acos¶
Acosh¶
Asin¶
Asinh¶
Atan¶
Atanh¶
AveragePooling2D¶
-
class
webdnn.graph.operators.average_pooling_2d.
AveragePooling2D
(name, ksize, stride, padding, cover_all=True)[source]¶ Spatial average pooling operator.
Parameters: - name (str) – Operator name.
- ksize (int or tuple of int) – Kernel size.
- stride (int or tuple of int) – Stride size.
- padding (int or tuple of int) – Padding size.
- cover_all (bool, optional) – If True, all input pixels are pooled into some output pixels.
- divide_without_padding – divides sum of input by the number of elements not of padding. Used in tensorflow.
- https (See) – //github.com/mil-tokyo/webdnn/issues/694
- Signature
y, = op(x)
- x - Input variable.
- y - Output value. Its order is same as
x
.
AxiswiseBias¶
-
class
webdnn.graph.operators.axiswise_bias.
AxiswiseBias
(name, axis)[source]¶ Adds a bias value along to specified axis.
Parameters: - name (str) – Operator name.
- axis (
Axis
) – target axis
- Signature
y, = op(x, b)
- x - Data variable. It must has axis axis, and whose size must be same as
b
. - b - Bias variable. It must be 1D variable and that size must be same as
x.shape_dict[axis]
- y - Output variable. Its order and shape is same as
x
.
- x - Data variable. It must has axis axis, and whose size must be same as
Deprecated since version v1.2: Use
ElementwiseAdd
instead.
AxiswiseScale¶
-
class
webdnn.graph.operators.axiswise_scale.
AxiswiseScale
(name, axis)[source]¶ Multiply a scale value along to specified axis.
Parameters: - name (str) – Operator name.
- axis (
Axis
) – target axis
- Signature
y, = op(x, s)
- x - Data variable. It must has axis axis, and whose size must be same as
s
. - s - Scale variable. It must be 1D variable and that size must be same as
x.shape_dict[axis]
- y - Output variable. Its order and shape is same as
x
.
- x - Data variable. It must has axis axis, and whose size must be same as
Deprecated since version v1.2: Use
ElementwiseMul
instead.
Broadcast¶
-
class
webdnn.graph.operators.broadcast.
Broadcast
(name, out_shape, out_order)[source]¶ Broadcast variable to specified shape. Each elementwise operator supports implicit broadcast feature. Therefore you don’t need to use this operator explicitly.
Parameters: - name (str) – Operator name.
- out_shape (list of int) – target shape
- out_order (
Order
) – target order
- Signature
y, = op(x)
- x - Input variable.
- y - Output variable.
ClippedRelu¶
Col2Im¶
-
class
webdnn.graph.operators.col2im.
Col2Im
(name: typing.Union[str, NoneType], ksize: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]], stride: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]], padding: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]])[source]¶
Concat¶
-
class
webdnn.graph.operators.concat.
Concat
(name, axis)[source]¶ Concatenate multiple variables into one variable along to specified axis.
Parameters: - name (str) – Operator name.
- ( (axis) – obj:~`webdnn.Axis`): target axis
- Signature
y, = op(x0, x1, ...)
- x0, x1, ... - Input variables. All variables has same shape except the specified axis.
- y - Output variable. Its order is same as
x0
.
Convolution2D¶
-
class
webdnn.graph.operators.convolution2d.
Convolution2D
(name, ksize, stride, padding, dilation_rate=1)[source]¶ Spatial convolution operator.
Parameters: - Signature
y, = op(x, w)
- x - Input variables. It must has 4 axes,
N
,C
,H
, andW
. - w - Kernel variable. It must has
N
,C
,H
,W
. Its size ofH
andW
must be same as kernel size. Its size ofC
must be same asx
. - y - Output variable. Its order is same as
x
.
- x - Input variables. It must has 4 axes,
Cos¶
Cosh¶
Deconvolution2D¶
-
class
webdnn.graph.operators.deconvolution2d.
Deconvolution2D
(name, ksize, stride, padding)[source]¶ Spatial deconvolution operator.
Parameters: - Signature
y, = op(x, w)
- x - Input variables. It must has 4 axes,
N
,C
,H
, andW
. - w - Kernel variable. It must has
N
,C
,H
,W
. Its size ofH
andW
must be same as kernel size. Its size ofC
must be same asx
. - y - Output variable. Its order is same as
x
.
- x - Input variables. It must has 4 axes,
Depth2Space¶
Elementwise¶
-
class
webdnn.graph.operators.elementwise.
Elementwise
(name)[source]¶ Elementwise operator base class.
Operation “Elementwise” is defined as follows:
It outputs only one variable.
All input variables and the output variable are same shape (not need to be same order).
Each element of output variable can be computed with only the elements at the same position of input variables.
\[y[p] = f(x_0[p], x_1[p], ..., x_n[p]),\]where \(p\) means the position of the element.
All input variables are registered with name like
x0
,x1
, ...x{index}
, and the output variable is registered with namey
.This operator has
Elementwise
attribute.Parameters: name (str) – Operator name. - Signature
y, = op(x0, x1, ...)
- x0, x1, ... - Input variables.
- y - Output variable. Its shape and order is same as
x0
.
ElementwiseAdd¶
-
class
webdnn.graph.operators.elementwise_add.
ElementwiseAdd
(name)[source]¶ Elementwise add
Parameters: name (str) – Operator name. - Signature
y, = op(x0, x1)
- x0, x1 - Input variable. They must be same shape.
- y - Output variable. Its order and shape is same as
x0
.
This operator also can be called by
+
.y = x0 + x1
ElementwiseDiv¶
-
class
webdnn.graph.operators.elementwise_div.
ElementwiseDiv
(name: typing.Union[str, NoneType])[source]¶ ElementwiseAdd(name)
Elementwise divide
Parameters: name (str) – Operator name. - Signature
y, = op(x0, x1)
- x0, x1 - Input variable. They must be same shape.
- y - Output variable. Its order and shape is same as
x0
.
This operator also can be called by
/
.y = x0 / x1
ElementwiseMul¶
-
class
webdnn.graph.operators.elementwise_mul.
ElementwiseMul
(name)[source]¶ Elementwise multiply
Parameters: name (str) – Operator name. - Signature
y, = op(x0, x1)
- x0, x1 - Input variable. They must be same shape.
- y - Output variable. Its order and shape is same as
x0
.
This operator also can be called by
*
.y = x0 * x1
ElementwisePow¶
-
class
webdnn.graph.operators.elementwise_pow.
ElementwisePow
(name)[source]¶ Elementwise power
Parameters: name (str) – Operator name. - Signature
y, = op(x0, x1)
- x0, x1 - Input variable. They must be same shape.
- y - Output variable. Its order and shape is same as
x0
.
This operator also can be called by
**
.y = x0 ** x1
ElementwiseSum¶
-
class
webdnn.graph.operators.elementwise_sum.
ElementwiseSum
(*args, **kwargs)[source]¶ Deprecated since version v1.2: Use
ElementwiseAdd
instead.
Elu¶
-
class
webdnn.graph.operators.elu.
Elu
(name)[source]¶ Exponential Linear Unit operator.
\[\begin{split}f(x) = \left \{ \begin{array}{ll} x & {\rm if}~ x \ge 0 \\ \exp(x) - 1 & {\rm if}~ x < 0, \end{array} \right.\end{split}\]More detail, please see https://arxiv.org/abs/1511.07289
Parameters: name (str) – Operator name. - Signature
y, = op(x0)
- x0 - Input variable.
- y - Output variable. Its order and shape is same as
x0
.
Embedding¶
-
class
webdnn.graph.operators.embedding.
Embedding
(name)[source]¶ Word embedding operator.
Parameters: name (str) – Operator name. - Signature
y, = op(x, w)
- x - Input variables. It must has 2 axes,
N
,T
. - w - Dictionary variable. It must has
N
, andC
. Its size ofC
must be same as the vocabulary size. Its size ofN
must be same as the embed feature size. - y - Output variable. Its order is
OrderNTC
.
- x - Input variables. It must has 2 axes,
Exp¶
Greater¶
GreaterEqual¶
HardSigmoid¶
-
class
webdnn.graph.operators.hard_sigmoid.
HardSigmoid
(name)[source]¶ Hard sigmoid operator
\[\begin{split}f(x) = \left \{ \begin{array}{ll} 0 & {\rm if}~ x < -2.5 \\ 0.2 x + 0.5 & {\rm if}~ -2.5 < x < 2.5 \\ 1 & {\rm if}~ 2.5 < x. \end{array} \right.\end{split}\]Parameters: name (str) – Operator name. - Signature
y, = op(x0)
- x0 - Input variable.
- y - Output variable. Its order and shape is same as
x0
.
Im2Col¶
-
class
webdnn.graph.operators.im2col.
Im2Col
(name: typing.Union[str, NoneType], ksize: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]], stride: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]], padding: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]], dilation_rate: typing.Union[_ForwardRef('IntOrTuple'), int, typing.Tuple[int, int]])[source]¶ -
WH
¶ Input window height considering dilation. Returns:
-
WW
¶ Input window width considering dilation. Returns:
-
LeakyRelu¶
Linear¶
-
class
webdnn.graph.operators.linear.
Linear
(name)[source]¶ Linear operator a.k.a. Dense, Fully-Connected operator.
This operator compute dot product with input and weight variables. Except
N
, all axes are used for calculation. If x isOrderNC
and w isOrderCN
, this operation is just same asnp.tensor_dot(x, w, (1, 0))
in numpy’stensordot
.Also, 4D variable is acceptable for input and weight variable. If x is
OrderNCHW
and w isOrderNHWC
, this operation is just same asnp.tensor_dot(x, w, ((1,2,3), (3,1,2))
in numpy’stensordot
.Input and weight variables must be same dimension. Otherwise, an assertion error is raised.
Parameters: name (str) – Operator name. - Signature
y, = op(x, w)
- x - Input variable.
- w - Weight variable.
- y - Output variable. Its order is
OrderNC
. ItsN
size is same asx.shape_dict[Axis.N]
, and itsC
size is same asw.shape_dict[Axis.N]
LocalResponseNormalization¶
-
class
webdnn.graph.operators.local_response_normalization.
LocalResponseNormalization
(name, n, k, alpha, beta)[source]¶ Operator same as local response normalization layer in Caffe. Only cross channel mode is supported; normalization is done for channel axis.
For more detail, see: http://caffe.berkeleyvision.org/tutorial/layers/lrn.html
Parameters: - Signature
y, = op(x)
- x - Input variable.
- y - Output variable. Its order and shape is same as
x
.
Log¶
LSTM¶
-
class
webdnn.graph.operators.lstm.
LSTM
(name: typing.Union[str, NoneType], use_bias: bool, return_sequences: bool, use_initial_c: bool, use_initial_h: bool, activation: str, recurrent_activation: str)[source]¶ Long-short term memory layer.
Currently, outputs value of hidden layer after final input is consumed. Details are corresponding to Keras’s implementation (layers/recurrent.py)
Parameters: name (str) – Operator name. -
__call__
(x: webdnn.graph.variable.Variable, w_input: webdnn.graph.variable.Variable, w_hidden: webdnn.graph.variable.Variable, b: typing.Union[webdnn.graph.variable.Variable, NoneType] = None, initial_c: typing.Union[webdnn.graph.variable.Variable, NoneType] = None, initial_h: typing.Union[webdnn.graph.variable.Variable, NoneType] = None)[source]¶ Parameters: Returns: Output (OrderNC) final_c (
Variable
): Last cell state (OrderNC)Return type: y (
Variable
)
-
Max¶
MaxPooling2D¶
Pooling2D¶
Prod¶
Reduce¶
-
class
webdnn.graph.operators.reduce.
Reduce
(name, axis)[source]¶ This operator reduces an axis into single element. This operator does NOT consider about the direction of reduction. For all axis without reduction axis, both input and output variables have same size.
Parameters: - name (str) – Operator name.
- axis (
Axis
) –
- Signature
y, = op(x)
- x - Input variables.
- y - Output variable.
ReinterpretAxis¶
-
class
webdnn.graph.operators.reinterpret_axis.
ReinterpretAxis
(name, in_order, out_order)[source]¶ Re-interpret an axis as another semantics. Shape is not changed.
In case of
in_order
isOrderNC
andout_order
isOrderNT
, ifOrderCN
variable is input, ~webdnn.graph.order.OrderTN variable is output.Parameters: - name (str) – Operator name.
- in_order (
Order
) – Input order - out_order (
Order
) – Output order
- Signature
y, = op(x)
- x - Input variable.
- y - Output variable. Its shape is same as
x
.
Relu¶
Reshape¶
-
class
webdnn.graph.operators.reshape.
Reshape
(name, in_order, out_order, out_shape)[source]¶ Reshape array assuming it is C-order. Removing / inserting axis with length 1
When in_order: NHWC, out_order: NTC, out_shape: (2, 6, 10) and input variable is (2, 3, 4, 5), the semantic procedure is as follows. 1. Interpret input variable as NTHWC (2, 1, 3, 4, 5) with inserting axis with length 1 2. Reshape it with assuming C-order and length of axis being removed is 1; NTHWC (2, 6, 1, 1, 10) 3. Remove axes; NTC (2, 6, 10)
Swapping axes is prohibited because it is ambiguous. If in_order and out_order match the actual input / output variable order, kernel does not have to do anything.
Parameters: - name (str) – Operator name.
- in_order (
Order
) – input order - out_order (
Order
) – output order - out_shape (list of int or
Placeholder
) – output shape
- Signature
y, = op(x)
- x - Input variable.
- y - Output variable.
Rsqrt¶
ScalarAdd¶
ScalarAffine¶
ScalarMul¶
ScalarPow¶
Select¶
Sigmoid¶
Sin¶
Sinh¶
Slice¶
Softmax¶
Softplus¶
Softsign¶
-
class
webdnn.graph.operators.softsign.
Softsign
(name)[source]¶ Softsign operator.
For more detail, see https://www.tensorflow.org/api_docs/python/tf/nn/softsign
\[f(x) = \frac{x}{abs(x) + 1}\]Parameters: name (str) – Operator name. - Signature
y, = op(x0)
- x0 - Input variable.
- y - Output variable. Its order and shape is same as
x0
.
Space2Depth¶
Tan¶
Tanh¶
Tensordot¶
ThresholdRelu¶
Tile¶
Transpose¶
-
class
webdnn.graph.operators.transpose.
Transpose
(name: typing.Union[str, NoneType])[source]¶ Transposition. Doing nothing in frontend level, and do memory transposition in backend if input / output variable order differs. This layer is inserted in optimizer to support layers which accepts certain order.
Parameters: name (str) – Operator name.
Unpooling2D¶
-
class
webdnn.graph.operators.unpooling_2d.
Unpooling2D
(name, ksize, stride, padding, outsize)[source]¶ Inverse operation of pooling for 2d array. This function acts similarly to
Deconvolution2D
, but it spreads input 2d array’s value without any parameter instead of computing the inner products.Parameters: - Signature
y, = op(x)
- x - Input variable.
- y - Output value. Its order is same as
x
.