He Normal initializer

tefla.core.initializers.he_normal (seed=None, scale=1.0, dtype=tf.float32) Kaiming He et al. (2015): Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852.

Args

• scale: float Scaling factor for the weights. Set this to 1.0 for linear and sigmoid units, to sqrt(2) for rectified linear units, and to sqrt(2/(1+alpha**2)) for leaky rectified linear units with leakiness alpha. Other transfer functions may need different factors.

---

He Uniform initializer

tefla.core.initializers.he_uniform (seed=None, scale=1.0, dtype=tf.float32)

Args

• scale: float Scaling factor for the weights. Set this to 1.0 for linear and sigmoid units, to sqrt(2) for rectified linear units, and to sqrt(2/(1+alpha**2)) for leaky rectified linear units with leakiness alpha. Other transfer functions may need different factors.

---

Random Normal initializer

tefla.core.initializers.random_normal (seed=None, mean=0.0, stddev=1.0, dtype=tf.float32, name=None)

Args

• mean: a float
• stddev: a float

---

Returns an initializer that generates tensors without scaling variance

tefla.core.initializers.variance_scaling_initializer_v2 (factor=2.0, mode='FAN_IN', uniform=False, seed=None, dtype=tf.float32, mean=0.0, stddev=1.0, normal_type=None, name=None) When initializing a deep network, it is in principle advantageous to keep the scale of the input variance constant, so it does not explode or diminish by reaching the final layer. This initializer use the following formula:

  if mode='FAN_IN': # Count only number of input connections.
n = fan_in
  elif mode='FAN_OUT': # Count only number of output connections.
n = fan_out
  elif mode='FAN_AVG': # Average number of inputs and output connections.
n = (fan_in + fan_out)/2.0
truncated_normal(shape, 0.0, stddev=sqrt(factor / n))

• To get Delving Deep into Rectifiers, use (Default):
  factor=2.0 mode='FAN_IN' uniform=False
• To get Convolutional Architecture for Fast Feature Embedding, use:
  factor=1.0 mode='FAN_IN' uniform=True
• To get Understanding the difficulty of training deep feedforward neural networks, use:
  factor=1.0 mode='FAN_AVG' uniform=True.
• To get xavier_initializer use either:
  factor=1.0 mode='FAN_AVG' uniform=True, or
  factor=1.0 mode='FAN_AVG' uniform=False.

  Args

factor: Float. A multiplicative factor. mode: String. 'FAN_IN', 'FAN_OUT', 'FAN_AVG'. uniform: Whether to use uniform or normal distributed random initialization. seed: A Python integer. Used to create random seeds. See - set_random_seed - for behavior. dtype: The data type. Only floating point types are supported.

Returns

An initializer that generates tensors with unit variance.

Raises

ValueError: if dtype is not a floating point type. TypeError: if mode is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].

Returns

An initializer that generates tensors with unit variance.

Raises

ValueError: if dtype is not a floating point type. TypeError: if mode is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].

---

Bilinear initialization for up sampling operation

tefla.core.initializers.bilinear (f_shape)

Args

• f_shape: shape of the variable

Returns

bilinear initializer

---

Variable initializer that produces a random orthonormal matrix

tefla.core.initializers.random_orthonormal_initializer (shape, dtype=tf.float32, partition_info=None)

Args

• shape: shape of the variable

Returns

random_orthogonal_matrix for initialization.

---