Device chooser for variables

tefla.utils.util.VariableDeviceChooser (num_parameter_servers=0, ps_device='/job:ps', placement='CPU:0') When using a parameter server it will assign them in a round-robin fashion. When not using a parameter server it allows GPU:0 placement otherwise CPU:0. Initialize VariableDeviceChooser.

Args

  • num_parameter_servers: number of parameter servers.
  • ps_device: string representing the parameter server device.
  • placement: string representing the placement of the variable either CPU:0 or GPU:0. When using parameter servers forced to CPU:0.

Valid types for loss, variables and gradients

tefla.utils.util.valid_dtypes () Subclasses should override to allow other float types.

Returns

Valid types for loss, variables and gradients.

Asserts tensors are all valid types (see _valid_dtypes)

tefla.utils.util.assert_valid_dtypes (tensors)

Args

  • tensors: Tensors to check.

    Raises

  • ValueError: If any tensor is not a valid type.

Returns value if value_or_tensor_or_var has a constant value

tefla.utils.util.constant_value (value_or_tensor_or_var, dtype=None)

Args

  • value_or_tensor_or_var: A value, a Tensor or a Variable.
  • dtype: Optional tf.dtype, if set it would check it has the right
  • dtype.

Returns

The constant value or None if it not constant.

Return either fn1() or fn2() based on the boolean value of pred

tefla.utils.util.static_cond (pred, fn1, fn2)

Same signature as control_flow_ops.cond() but requires pred to be a bool.

Args

  • pred: A value determining whether to return the result of fn1 or fn2.
  • fn1: The callable to be performed if pred is true.
  • fn2: The callable to be performed if pred is false.

Returns

Tensors returned by the call to either fn1 or fn2.

Return either fn1() or fn2() based on the boolean predicate/value pred

tefla.utils.util.smart_cond (pred, fn1, fn2, name=None)

If pred is bool or has a constant value it would use static_cond, otherwise it would use tf.cond.

Args

  • pred: A scalar determining whether to return the result of fn1 or fn2.
  • fn1: The callable to be performed if pred is true.
  • fn2: The callable to be performed if pred is false.

  • name: Optional name prefix when using tf.cond

    Returns

  • Tensors returned by the call to either fn1 or fn2.

Returns

Tensors returned by the call to either fn1 or fn2.

Transform numeric labels into onehot_labels

tefla.utils.util.one_hot (labels, num_classes, name='one_hot')

Args

  • labels: [batch_size] target labels.
  • num_classes: total number of classes.

  • scope: Optional scope for op_scope.

    Returns

  • one hot encoding of the labels.

Returns

one hot encoding of the labels.

Returns a true if its input is a collections.Sequence (except strings)

tefla.utils.util.is_sequence (seq)

Args

  • seq: an input sequence.

Returns

True if the sequence is a not a string and is a collections.Sequence.

Returns a flat sequence from a given nested structure

tefla.utils.util.flatten_sq (nest_sq) If nest is not a sequence, this returns a single-element list: [nest].

Args

  • nest: an arbitrarily nested structure or a scalar object. Note, numpy arrays are considered scalars.

Returns

A Python list, the flattened version of the input.

Returns the last dimension of shape while checking it has min_rank

tefla.utils.util.last_dimension (shape, min_rank=1)

Args

  • shape: A TensorShape.
  • min_rank: Integer, minimum rank of shape.

Returns

The value of the last dimension.

Load Graph from frozen weights and model

tefla.utils.util.load_frozen_graph (frozen_graph)

Args

  • frozen_graph: binary pb file

Returns

loaded graph

Normalize a input layer

tefla.utils.util.normalize (input_layer)

Args

  • inmput_layer: input layer tp normalize

Returns

normalized layer

DeNormalize a input layer

tefla.utils.util.denormalize (input_layer)

Args

  • input_layer: input layer to de normalize

Returns

denormalized layer

Computes the squared pairwise Euclidean distances between x and y

tefla.utils.util.compute_pairwise_distances (x, y)

Args

  • x: a tensor of shape [num_x_samples, num_features]
  • y: a tensor of shape [num_y_samples, num_features]

Returns

a distance matrix of dimensions [num_x_samples, num_y_samples].

Computes a Guassian Radial Basis Kernel between the samples of x and y

tefla.utils.util.gaussian_kernel_matrix (x, y, sigmas) We create a sum of multiple gaussian kernels each having a width sigma_i.

Args

  • x: a tensor of shape [num_samples, num_features]
  • y: a tensor of shape [num_samples, num_features]
  • sigmas: a tensor of floats which denote the widths of each of the gaussians in the kernel.

Returns

A tensor of shape [num_samples{x}, num_samples{y}] with the RBF kernel.

compute the length of a sequence. 0 are masked

tefla.utils.util.retrieve_seq_length (data)

Args

  • data: input sequence

Returns

a int, length of the sequence

Advanced Indexing for Sequences

tefla.utils.util.advanced_indexing (inp, index)

Args

  • inp: input sequence
  • index: input index for indexing

Returns

a indexed sequence

pad_sequences

tefla.utils.util.pad_sequences (sequences, maxlen=None, dtype='int32', padding='post', truncating='post', value=0.0) Pad each sequence to the same length: the length of the longest sequence. If maxlen is provided, any sequence longer than maxlen is truncated to maxlen. Truncation happens off either the beginning or the end (default) of the sequence. Supports pre-padding and post-padding (default).

Args

  • sequences: list of lists where each element is a sequence.
  • maxlen: a int, maximum length.
  • dtype: type to cast the resulting sequence.
  • padding: 'pre' or 'post', pad either before or after each sequence.
  • truncating: 'pre' or 'post', remove values from sequences larger than maxlen either in the beginning or in the end of the sequence
  • value: float, value to pad the sequences to the desired value.

Returns

x

numpy array with dimensions (number_of_sequences, maxlen)

Creates a dictionary char:integer for each unique character

tefla.utils.util.chars_to_dictionary (string)

Args

  • string: a string input

Returns

dictionary of chars

string_to_semi_redundant_sequences

tefla.utils.util.string_to_semi_redundant_sequences (string, seq_maxlen=25, redun_step=3, char_idx=None) Vectorize a string and returns parsed sequences and targets, along with the associated dictionary.

Args

  • string: str. Lower-case text from input text file.
  • seq_maxlen: int. Maximum length of a sequence. Default: 25.
  • redun_step: int. Redundancy step. Default: 3.
  • char_idx: 'dict'. A dictionary to convert chars to positions. Will be automatically generated if None

Returns

A tuple: (inputs, targets, dictionary)

Vectorize Text file

tefla.utils.util.textfile_to_semi_redundant_sequences (path, seq_maxlen=25, redun_step=3, to_lower_case=False, pre_defined_char_idx=None) textfile_to_semi_redundant_sequences. Vectorize a string from a textfile and returns parsed sequences and targets, along with the associated dictionary.

Args

  • path: str. path of the input text file.
  • seq_maxlen: int. Maximum length of a sequence. Default: 25.
  • redun_step: int. Redundancy step. Default: 3.
  • to_lower_case: a bool, if true, convert to lowercase
  • pre_defined_char_idx: 'dict'. A dictionary to convert chars to positions. Will be automatically generated if None

Returns

A tuple: (inputs, targets, dictionary)

Computes log probabilities using numerically stable trick

tefla.utils.util.logits_to_log_prob (logits) This uses two numerical stability tricks: 1) softmax(x) = softmax(x - c) where c is a constant applied to all arguments. If we set c = max(x) then the softmax is more numerically stable. 2) log softmax(x) is not numerically stable, but we can stabilize it by using the identity log softmax(x) = x - log sum exp(x)

Args

  • logits: Tensor of arbitrary shape whose last dimension contains logits.

Returns

A tensor of the same shape as the input, but with corresponding log probabilities.

Get the name of the op that created a tensor

tefla.utils.util.GetTensorOpName (x) Useful for naming related tensors, as ':' in name field of op is not permitted

Args

x

the input tensor.

Returns

the name of the op.

Returns the union of two lists

tefla.utils.util.ListUnion (list_1, list_2) Python sets can have a non-deterministic iteration order. In some contexts, this could lead to TensorFlow producing two different programs when the same Python script is run twice. In these contexts we use lists instead of sets. This function is not designed to be especially fast and should only be used with small lists.

Args

  • list_1: A list
  • list_2: Another list

Returns

A new list containing one copy of each unique element of list_1 and list_2. Uniqueness is determined by "x in union" logic; e.g. two ` string of that value appearing in the union.

Maps xs to consumers

tefla.utils.util.Interface (ys, xs) Returns a dict mapping each element of xs to any of its consumers that are indirectly consumed by ys.

Args

ys: The outputs xs: The inputs

Returns

out: Dict mapping each member x of xs to a list of all Tensors that are direct consumers of x and are eventually consumed by a member of ys.

Clip an array of tensors by L2 norm

tefla.utils.util.BatchClipByL2norm (t, upper_bound, name=None) Shrink each dimension-0 slice of tensor (for matrix it is each row) such that the l2 norm is at most upper_bound. Here we clip each row as it corresponds to each example in the batch.

Args

t: the input tensor. upper_bound: the upperbound of the L2 norm. name: optional name.

Returns

the clipped tensor.

Add i.i.d. Gaussian noise (0, sigma^2) to every entry of t

tefla.utils.util.AddGaussianNoise (t, sigma, name=None)

Args

t: the input tensor. sigma: the stddev of the Gaussian noise. name: optional name.

Returns

the noisy tensor.