Skip to content

graph

NetworkDescriptor

A class describing the neural architecture for neural network kernel. It only record the width of convolutional and dense layers, and the skip-connection types and positions.

add_skip_connection

Add a skip-connection to the descriptor.

Args
  • u: Number of convolutional layers before the starting point.

  • v: Number of convolutional layers before the ending point.

  • connection_type: Must be either CONCAT_CONNECT or ADD_CONNECT.

Node

A class for intermediate output tensor (node) in the Graph.

Attributes
  • shape: A tuple describing the shape of the tensor.

Graph

A class representing the neural architecture graph of a Keras model. Graph extracts the neural architecture graph from a Keras model. Each node in the graph is a intermediate tensor between layers. Each layer is an edge in the graph. Notably, multiple edges may refer to the same layer. (e.g. Add layer is adding two tensor into one tensor. So it is related to two edges.)

Attributes
  • input_shape: A tuple describing the input tensor shape, not including the number of instances.

  • weighted: A boolean marking if there are actual values in the weights of the layers. Sometime we only need the neural architecture information with a graph. In that case, we do not save the weights to save memory and time.

  • node_list: A list of integers. The indices of the list are the identifiers.

  • layer_list: A list of stub layers. The indices of the list are the identifiers.

  • node_to_id: A dict instance mapping from node integers to their identifiers.

  • layer_to_id: A dict instance mapping from stub layers to their identifiers.

  • layer_id_to_input_node_ids: A dict instance mapping from layer identifiers to their input nodes identifiers.

  • layer_id_to_output_node_ids: A dict instance mapping from layer identifiers to their output nodes identifiers.

  • adj_list: A two dimensional list. The adjacency list of the graph. The first dimension is identified by tensor identifiers. In each edge list, the elements are two-element tuples of (tensor identifier, layer identifier).

  • reverse_adj_list: A reverse adjacent list in the same format as adj_list.

  • operation_history: A list saving all the network morphism operations.

  • n_dim: An integer. If it uses Conv1d, n_dim should be 1.

  • vis: A dictionary of temporary storage for whether an local operation has been done during the network morphism.

init

Initializer for Graph.

Args
  • input_shape: A tuple describing the input tensor shape, not including the number of instances.

  • weighted: A boolean marking if there are actual values in the weights of the layers. Sometime we only need the neural architecture information with a graph. In that case, we do not save the weights to save memory and time.

add_layer

Add a layer to the Graph.

Args
  • layer: An instance of the subclasses of StubLayer in layers.py.

  • input_node_id: An integer. The ID of the input node of the layer.

Returns
  • output_node_id: An integer. The ID of the output node of the layer.

n_nodes

Return the number of nodes in the model.

n_layers

Return the number of layers in the model.

_add_node

Add a new node to node_list and give the node an ID.

Args
  • node: An instance of Node.
Returns
  • node_id: An integer.

_add_edge

Add a new layer to the graph. The nodes should be created in advance.

_redirect_edge

Redirect the layer to a new node. Change the edge originally from u_id to v_id into an edge from u_id to new_v_id while keeping all other property of the edge the same.

_replace_layer

Replace the layer with a new layer.

topological_order

Return the topological order of the node IDs from the input node to the output node.

_get_pooling_layers

Given two node IDs, return all the pooling layers between them.

Search for all the layers and nodes down the path. A recursive function to search all the layers and nodes between the node in the node_list and the node with target_id.

Search the graph for all the layers to be widened caused by an operation. It is an recursive function with duplication check to avoid deadlock. It searches from a starting node u until the corresponding layers has been widened.

Args
  • u: The starting node ID.

  • start_dim: The position to insert the additional dimensions.

  • total_dim: The total number of dimensions the layer has before widening.

  • n_add: The number of dimensions to add.

to_deeper_model

Insert a relu-conv-bn block after the target block.

Args
  • target_id: A convolutional layer ID. The new block should be inserted after the block.

  • new_layer: An instance of StubLayer subclasses.

to_wider_model

Widen the last dimension of the output of the pre_layer.

Args
  • pre_layer_id: The ID of a convolutional layer or dense layer.

  • n_add: The number of dimensions to add.

_insert_new_layers

Insert the new_layers after the node with start_node_id.

to_add_skip_model

Add a weighted add skip-connection from after start node to end node.

Args
  • start_id: The convolutional layer ID, after which to start the skip-connection.

  • end_id: The convolutional layer ID, after which to end the skip-connection.

to_concat_skip_model

Add a weighted add concatenate connection from after start node to end node.

Args
  • start_id: The convolutional layer ID, after which to start the skip-connection.

  • end_id: The convolutional layer ID, after which to end the skip-connection.

extract_descriptor

Extract the the description of the Graph as an instance of NetworkDescriptor.

produce_model

Build a new torch model based on the current graph.

produce_keras_model

Build a new keras model based on the current graph.

get_main_chain_layers

Return a list of layer IDs in the main chain.

get_main_chain

Returns the main chain node ID list.