Layers¶
Deep learning models are often said to be made up of “layers”. Intuitively, a “layer” is a function which transforms some tensor into another tensor. DeepChem maintains an extensive collection of layers which perform various useful scientific transformations. For now, most layers are Keras only but over time we expect this support to expand to other types of models and layers.

class
InteratomicL2Distances
(*args, **kwargs)[source]¶ Compute (squared) L2 Distances between atoms given neighbors.
This class computes pairwise distances between its inputs.
Examples
>>> import numpy as np >>> import deepchem as dc >>> atoms = 5 >>> neighbors = 2 >>> coords = np.random.rand(atoms, 3) >>> neighbor_list = np.random.randint(0, atoms, size=(atoms, neighbors)) >>> layer = InteratomicL2Distances(atoms, neighbors, 3) >>> result = np.array(layer([coords, neighbor_list])) >>> result.shape (5, 2)

class
GraphConv
(*args, **kwargs)[source]¶ Graph Convolutional Layers
This layer implements the graph convolution introduced in [1]_. The graph convolution combines pernode feature vectures in a nonlinear fashion with the feature vectors for neighboring nodes. This “blends” information in local neighborhoods of a graph.
References
 1
Duvenaud, David K., et al. “Convolutional networks on graphs for learning molecular fingerprints.” Advances in neural information processing systems. 2015. https://arxiv.org/abs/1509.09292

__init__
(out_channel: int, min_deg: int = 0, max_deg: int = 10, activation_fn: Optional[Callable] = None, **kwargs)[source]¶ Initialize a graph convolutional layer.
 Parameters
out_channel (int) – The number of output channels per graph node.
min_deg (int, optional (default 0)) – The minimum allowed degree for each graph node.
max_deg (int, optional (default 10)) – The maximum allowed degree for each graph node. Note that this is set to 10 to handle complex molecules (some organometallic compounds have strange structures). If you’re using this for nonmolecular applications, you may need to set this much higher depending on your dataset.
activation_fn (function) – A nonlinear activation function to apply. If you’re not sure, tf.nn.relu is probably a good default for your application.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.

class
GraphPool
(*args, **kwargs)[source]¶ A GraphPool gathers data from local neighborhoods of a graph.
This layer does a maxpooling over the feature vectors of atoms in a neighborhood. You can think of this layer as analogous to a maxpooling layer for 2D convolutions but which operates on graphs instead. This technique is described in [1]_.
References
 1
Duvenaud, David K., et al. “Convolutional networks on graphs for
learning molecular fingerprints.” Advances in neural information processing systems. 2015. https://arxiv.org/abs/1509.09292

__init__
(min_degree=0, max_degree=10, **kwargs)[source]¶ Initialize this layer
 Parameters
min_deg (int, optional (default 0)) – The minimum allowed degree for each graph node.
max_deg (int, optional (default 10)) – The maximum allowed degree for each graph node. Note that this is set to 10 to handle complex molecules (some organometallic compounds have strange structures). If you’re using this for nonmolecular applications, you may need to set this much higher depending on your dataset.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.

class
GraphGather
(*args, **kwargs)[source]¶ A GraphGather layer pools nodelevel feature vectors to create a graph feature vector.
Many graph convolutional networks manipulate feature vectors per graphnode. For a molecule for example, each node might represent an atom, and the network would manipulate atomic feature vectors that summarize the local chemistry of the atom. However, at the end of the application, we will likely want to work with a molecule level feature representation. The GraphGather layer creates a graph level feature vector by combining all the nodelevel feature vectors.
One subtlety about this layer is that it depends on the batch_size. This is done for internal implementation reasons. The GraphConv, and GraphPool layers pool all nodes from all graphs in a batch that’s being processed. The GraphGather reassembles these jumbled node feature vectors into pergraph feature vectors.
References
 1
Duvenaud, David K., et al. “Convolutional networks on graphs for
learning molecular fingerprints.” Advances in neural information processing systems. 2015. https://arxiv.org/abs/1509.09292

__init__
(batch_size, activation_fn=None, **kwargs)[source]¶ Initialize this layer.
 Parameters
batch_size (int) – The batch size for this layer. Note that the layer’s behavior changes depending on the batch size.
activation_fn (function) – A nonlinear activation function to apply. If you’re not sure, tf.nn.relu is probably a good default for your application.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

class
MolGANConvolutionLayer
(*args, **kwargs)[source]¶ Graph convolution layer used in MolGAN model. MolGAN is a WGAN type model for generation of small molecules. Not used directly, higher level layers like MolGANMultiConvolutionLayer use it. This layer performs basic convolution on onehot encoded matrices containing atom and bond information. This layer also accepts three inputs for the case when convolution is performed more than once and results of previous convolution need to used. It was done in such a way to avoid creating another layer that accepts three inputs rather than two. The last input layer is socalled hidden_layer and it hold results of the convolution while first two are unchanged input tensors.
Example
See: MolGANMultiConvolutionLayer for using in layers.
>>> from tensorflow.keras import Model >>> from tensorflow.keras.layers import Input >>> vertices = 9 >>> nodes = 5 >>> edges = 5 >>> units = 128
>>> layer1 = MolGANConvolutionLayer(units=units,edges=edges) >>> layer2 = MolGANConvolutionLayer(units=units,edges=edges) >>> adjacency_tensor= Input(shape=(vertices, vertices, edges)) >>> node_tensor = Input(shape=(vertices,nodes)) >>> hidden1 = layer1([adjacency_tensor,node_tensor]) >>> output = layer2(hidden1) >>> model = Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
References
 1
Nicola De Cao et al. “MolGAN: An implicit generative model
for small molecular graphs”, https://arxiv.org/abs/1805.11973

__init__
(units: int, activation: Callable = <function tanh>, dropout_rate: float = 0.0, edges: int = 5, name: str = '', **kwargs)[source]¶ Initialize this layer.
 Parameters
units (int) – Dimesion of dense layers used for convolution
activation (function, optional (default=Tanh)) – activation function used across model, default is Tanh
dropout_rate (float, optional (default=0.0)) – Dropout rate used by dropout layer
edges (int, optional (default=5)) – How many dense layers to use in convolution. Typically equal to number of bond types used in the model.
name (string, optional (default="")) – Name of the layer

call
(inputs, training=False)[source]¶ Invoke this layer
 Parameters
inputs (list) – List of two input matrices, adjacency tensor and node features tensors in onehot encoding format.
training (bool) – Should this layer be run in training mode. Typically decided by main model, influences things like dropout.
 Returns
First and second are original input tensors Third is the result of convolution
 Return type
tuple(tf.Tensor,tf.Tensor,tf.Tensor)

class
MolGANAggregationLayer
(*args, **kwargs)[source]¶ Graph Aggregation layer used in MolGAN model. MolGAN is a WGAN type model for generation of small molecules. Performs aggregation on tensor resulting from convolution layers. Given its simple nature it might be removed in future and moved to MolGANEncoderLayer.
Example
>>> from tensorflow.keras import Model >>> from tensorflow.keras.layers import Input >>> vertices = 9 >>> nodes = 5 >>> edges = 5 >>> units = 128
>>> layer_1 = MolGANConvolutionLayer(units=units,edges=edges) >>> layer_2 = MolGANConvolutionLayer(units=units,edges=edges) >>> layer_3 = MolGANAggregationLayer(units=128) >>> adjacency_tensor= Input(shape=(vertices, vertices, edges)) >>> node_tensor = Input(shape=(vertices,nodes)) >>> hidden_1 = layer_1([adjacency_tensor,node_tensor]) >>> hidden_2 = layer_2(hidden_1) >>> output = layer_3(hidden_2[2]) >>> model = Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
Example
vertices = 9 nodes = 5 edges = 5 units = 128
layer_1 = MolGANConvolutionLayer(units=units,edges=edges) layer_2 = MolGANConvolutionLayer(units=units,edges=edges) layer_3 = MolGANAggregationLayer(units=128) adjacency_tensor= layers.Input(shape=(vertices, vertices, edges)) node_tensor = layers.Input(shape=(vertices,nodes)) hidden_1 = layer_1([adjacency_tensor,node_tensor]) hidden_2 = layer_2(hidden_1) output = layer_3(hidden_2[2]) model = keras.Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
References
 1
Nicola De Cao et al. “MolGAN: An implicit generative model
for small molecular graphs”, https://arxiv.org/abs/1805.11973

__init__
(units: int = 128, activation: Callable = <function tanh>, dropout_rate: float = 0.0, name: str = '', **kwargs)[source]¶ Initialize the layer
 Parameters
units (int, optional (default=128)) – Dimesion of dense layers used for aggregation
activation (function, optional (default=Tanh)) – activation function used across model, default is Tanh
dropout_rate (float, optional (default=0.0)) – Used by dropout layer
name (string, optional (default="")) – Name of the layer

call
(inputs, training=False)[source]¶ Invoke this layer
 Parameters
inputs (List) – Single tensor resulting from graph convolution layer
training (bool) – Should this layer be run in training mode. Typically decided by main model, influences things like dropout.
 Returns
aggregation tensor – Result of aggregation function on input convolution tensor.
 Return type
tf.Tensor

class
MolGANMultiConvolutionLayer
(*args, **kwargs)[source]¶ Multiple pass convolution layer used in MolGAN model. MolGAN is a WGAN type model for generation of small molecules. It takes outputs of previous convolution layer and uses them as inputs for the next one. It simplifies the overall framework, but might be moved to MolGANEncoderLayer in the future in order to reduce number of layers.
Example
>>> from tensorflow.keras import Model >>> from tensorflow.keras.layers import Input >>> vertices = 9 >>> nodes = 5 >>> edges = 5 >>> units = 128
>>> layer_1 = MolGANMultiConvolutionLayer(units=(128,64)) >>> layer_2 = MolGANAggregationLayer(units=128) >>> adjacency_tensor= Input(shape=(vertices, vertices, edges)) >>> node_tensor = Input(shape=(vertices,nodes)) >>> hidden = layer_1([adjacency_tensor,node_tensor]) >>> output = layer_2(hidden) >>> model = Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
Example
vertices = 9 nodes = 5 edges = 5 units = 128
layer_1 = MolGANMultiConvolutionLayer(units=(128,64)) layer_2 = MolGANAggregationLayer(units=128) adjacency_tensor= layers.Input(shape=(vertices, vertices, edges)) node_tensor = layers.Input(shape=(vertices,nodes)) hidden = layer_1([adjacency_tensor,node_tensor]) output = layer_2(hidden) model = keras.Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
References
 1
Nicola De Cao et al. “MolGAN: An implicit generative model
for small molecular graphs”, https://arxiv.org/abs/1805.11973

__init__
(units: Tuple = (128, 64), activation: Callable = <function tanh>, dropout_rate: float = 0.0, edges: int = 5, name: str = '', **kwargs)[source]¶ Initialize the layer
 Parameters
units (Tuple, optional (default=(128,64)), min_length=2) – List of dimensions used by consecutive convolution layers. The more values the more convolution layers invoked.
activation (function, optional (default=tanh)) – activation function used across model, default is Tanh
dropout_rate (float, optional (default=0.0)) – Used by dropout layer
edges (int, optional (default=0)) – Controls how many dense layers use for single convolution unit. Typically matches number of bond types used in the molecule.
name (string, optional (default="")) – Name of the layer

call
(inputs, training=False)[source]¶ Invoke this layer
 Parameters
inputs (list) – List of two input matrices, adjacency tensor and node features tensors in onehot encoding format.
training (bool) – Should this layer be run in training mode. Typically decided by main model, influences things like dropout.
 Returns
convolution tensor – Result of input tensors going through convolution a number of times.
 Return type
tf.Tensor

class
MolGANEncoderLayer
(*args, **kwargs)[source]¶ Main learning layer used by MolGAN model. MolGAN is a WGAN type model for generation of small molecules. It role is to further simplify model. This layer can be manually built by stacking graph convolution layers followed by graph aggregation.
Example
>>> from tensorflow.keras import Model >>> from tensorflow.keras.layers import Input, Dropout,Dense >>> vertices = 9 >>> edges = 5 >>> nodes = 5 >>> dropout_rate = .0 >>> adjacency_tensor= Input(shape=(vertices, vertices, edges)) >>> node_tensor = Input(shape=(vertices, nodes))
>>> graph = MolGANEncoderLayer(units = [(128,64),128], dropout_rate= dropout_rate, edges=edges)([adjacency_tensor,node_tensor]) >>> dense = Dense(units=128, activation='tanh')(graph) >>> dense = Dropout(dropout_rate)(dense) >>> dense = Dense(units=64, activation='tanh')(dense) >>> dense = Dropout(dropout_rate)(dense) >>> output = Dense(units=1)(dense)
>>> model = Model(inputs=[adjacency_tensor,node_tensor], outputs=[output])
References
 1
Nicola De Cao et al. “MolGAN: An implicit generative model
for small molecular graphs”, https://arxiv.org/abs/1805.11973

__init__
(units: List = [(128, 64), 128], activation: Callable = <function tanh>, dropout_rate: float = 0.0, edges: int = 5, name: str = '', **kwargs)[source]¶ Initialize the layer.
 Parameters
units (List, optional (default=[(128, 64), 128])) – List of units for MolGANMultiConvolutionLayer and GraphAggregationLayer i.e. [(128,64),128] means two convolution layers dims = [128,64] followed by aggregation layer dims=128
activation (function, optional (default=Tanh)) – activation function used across model, default is Tanh
dropout_rate (float, optional (default=0.0)) – Used by dropout layer
edges (int, optional (default=0)) – Controls how many dense layers use for single convolution unit. Typically matches number of bond types used in the molecule.
name (string, optional (default="")) – Name of the layer

call
(inputs, training=False)[source]¶ Invoke this layer
 Parameters
inputs (list) – List of two input matrices, adjacency tensor and node features tensors in onehot encoding format.
training (bool) – Should this layer be run in training mode. Typically decided by main model, influences things like dropout.
 Returns
encoder tensor – Tensor that been through number of convolutions followed by aggregation.
 Return type
tf.Tensor

class
LSTMStep
(*args, **kwargs)[source]¶ Layer that performs a single step LSTM update.
This layer performs a single step LSTM update. Note that it is not a full LSTM recurrent network. The LSTMStep layer is useful as a primitive for designing layers such as the AttnLSTMEmbedding or the IterRefLSTMEmbedding below.

__init__
(output_dim, input_dim, init_fn='glorot_uniform', inner_init_fn='orthogonal', activation_fn='tanh', inner_activation_fn='hard_sigmoid', **kwargs)[source]¶  Parameters
output_dim (int) – Dimensionality of output vectors.
input_dim (int) – Dimensionality of input vectors.
init_fn (str) – TensorFlow nitialization to use for W.
inner_init_fn (str) – TensorFlow initialization to use for U.
activation_fn (str) – TensorFlow activation to use for output.
inner_activation_fn (str) – TensorFlow activation to use for inner steps.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.


class
AttnLSTMEmbedding
(*args, **kwargs)[source]¶ Implements AttnLSTM as in matching networks paper.
The AttnLSTM embedding adjusts two sets of vectors, the “test” and “support” sets. The “support” consists of a set of evidence vectors. Think of these as the small training set for lowdata machine learning. The “test” consists of the queries we wish to answer with the small amounts of available data. The AttnLSTMEmbdding allows us to modify the embedding of the “test” set depending on the contents of the “support”. The AttnLSTMEmbedding is thus a type of learnable metric that allows a network to modify its internal notion of distance.
See references [1]_ 2 for more details.
References
 1
Vinyals, Oriol, et al. “Matching networks for one shot learning.” Advances in neural information processing systems. 2016.
 2
Vinyals, Oriol, Samy Bengio, and Manjunath Kudlur. “Order matters: Sequence to sequence for sets.” arXiv preprint arXiv:1511.06391 (2015).

__init__
(n_test, n_support, n_feat, max_depth, **kwargs)[source]¶  Parameters
n_support (int) – Size of support set.
n_test (int) – Size of test set.
n_feat (int) – Number of features per atom
max_depth (int) – Number of “processing steps” used by sequencetosequence for sets model.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ Execute this layer on input tensors.
 Parameters
inputs (list) – List of two tensors (X, Xp). X should be of shape (n_test, n_feat) and Xp should be of shape (n_support, n_feat) where n_test is the size of the test set, n_support that of the support set, and n_feat is the number of peratom features.
 Returns
Returns two tensors of same shape as input. Namely the output shape will be [(n_test, n_feat), (n_support, n_feat)]
 Return type
list

class
IterRefLSTMEmbedding
(*args, **kwargs)[source]¶ Implements the Iterative Refinement LSTM.
Much like AttnLSTMEmbedding, the IterRefLSTMEmbedding is another type of learnable metric which adjusts “test” and “support.” Recall that “support” is the small amount of data available in a low data machine learning problem, and that “test” is the query. The AttnLSTMEmbedding only modifies the “test” based on the contents of the support. However, the IterRefLSTM modifies both the “support” and “test” based on each other. This allows the learnable metric to be more malleable than that from AttnLSTMEmbeding.

__init__
(n_test, n_support, n_feat, max_depth, **kwargs)[source]¶ Unlike the AttnLSTM model which only modifies the test vectors additively, this model allows for an additive update to be performed to both test and support using information from each other.
 Parameters
n_support (int) – Size of support set.
n_test (int) – Size of test set.
n_feat (int) – Number of input atom features
max_depth (int) – Number of LSTM Embedding layers.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ Execute this layer on input tensors.
 Parameters
inputs (list) – List of two tensors (X, Xp). X should be of shape (n_test, n_feat) and Xp should be of shape (n_support, n_feat) where n_test is the size of the test set, n_support that of the support set, and n_feat is the number of peratom features.
 Returns
Returns two tensors of same shape as input. Namely the output
shape will be [(n_test, n_feat), (n_support, n_feat)]


class
SwitchedDropout
(*args, **kwargs)[source]¶ Apply dropout based on an input.
This is required for uncertainty prediction. The standard Keras Dropout layer only performs dropout during training, but we sometimes need to do it during prediction. The second input to this layer should be a scalar equal to 0 or 1, indicating whether to perform dropout.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
WeightedLinearCombo
(*args, **kwargs)[source]¶ Computes a weighted linear combination of input layers, with the weights defined by trainable variables.

__init__
(std=0.3, **kwargs)[source]¶ Initialize this layer.
 Parameters
std (float, optional (default 0.3)) – The standard deviation to use when randomly initializing weights.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
CombineMeanStd
(*args, **kwargs)[source]¶ Generate Gaussian nose.

__init__
(training_only=False, noise_epsilon=1.0, **kwargs)[source]¶ Create a CombineMeanStd layer.
This layer should have two inputs with the same shape, and its output also has the same shape. Each element of the output is a Gaussian distributed random number whose mean is the corresponding element of the first input, and whose standard deviation is the corresponding element of the second input.
 Parameters
training_only (bool) – if True, noise is only generated during training. During prediction, the output is simply equal to the first input (that is, the mean of the distribution used during training).
noise_epsilon (float) – The noise is scaled by this factor

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs, training=True)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
Stack
(*args, **kwargs)[source]¶ Stack the inputs along a new axis.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
VinaFreeEnergy
(*args, **kwargs)[source]¶ Computes freeenergy as defined by Autodock Vina.
TODO(rbharath): Make this layer support batching.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
NeighborList
(*args, **kwargs)[source]¶ Computes a neighborlist in Tensorflow.
Neighborlists (also called Verlet Lists) are a tool for grouping atoms which are close to each other spatially. This layer computes a Neighbor List from a provided tensor of atomic coordinates. You can think of this as a general “kmeans” layer, but optimized for the case k==3.
TODO(rbharath): Make this layer support batching.

__init__
(N_atoms, M_nbrs, ndim, nbr_cutoff, start, stop, **kwargs)[source]¶  Parameters
N_atoms (int) – Maximum number of atoms this layer will neighborlist.
M_nbrs (int) – Maximum number of spatial neighbors possible for atom.
ndim (int) – Dimensionality of space atoms live in. (Typically 3D, but sometimes will want to use higher dimensional descriptors for atoms).
nbr_cutoff (float) – Length in Angstroms (?) at which atom boxes are gridded.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.

compute_nbr_list
(coords)[source]¶ Get closest neighbors for atoms.
Needs to handle padding for atoms with no neighbors.
 Parameters
coords (tf.Tensor) – Shape (N_atoms, ndim)
 Returns
nbr_list – Shape (N_atoms, M_nbrs) of atom indices
 Return type
tf.Tensor

get_atoms_in_nbrs
(coords, cells)[source]¶ Get the atoms in neighboring cells for each cells.
 Returns
 Return type
atoms_in_nbrs = (N_atoms, n_nbr_cells, M_nbrs)

get_closest_atoms
(coords, cells)[source]¶ For each cell, find M_nbrs closest atoms.
Let N_atoms be the number of atoms.
 Parameters
coords (tf.Tensor) – (N_atoms, ndim) shape.
cells (tf.Tensor) – (n_cells, ndim) shape.
 Returns
closest_inds – Of shape (n_cells, M_nbrs)
 Return type
tf.Tensor

get_cells_for_atoms
(coords, cells)[source]¶ Compute the cells each atom belongs to.
 Parameters
coords (tf.Tensor) – Shape (N_atoms, ndim)
cells (tf.Tensor) – (n_cells, ndim) shape.
 Returns
cells_for_atoms – Shape (N_atoms, 1)
 Return type
tf.Tensor

get_neighbor_cells
(cells)[source]¶ Compute neighbors of cells in grid.
# TODO(rbharath): Do we need to handle periodic boundary conditions properly here? # TODO(rbharath): This doesn’t handle boundaries well. We hardcode # looking for n_nbr_cells neighbors, which isn’t right for boundary cells in # the cube.
 Parameters
cells (tf.Tensor) – (n_cells, ndim) shape.
 Returns
nbr_cells – (n_cells, n_nbr_cells)
 Return type
tf.Tensor

get_cells
()[source]¶ Returns the locations of all grid points in box.
Suppose start is 10 Angstrom, stop is 10 Angstrom, nbr_cutoff is 1. Then would return a list of length 20^3 whose entries would be [(10, 10, 10), (10, 10, 9), …, (9, 9, 9)]
 Returns
cells – (n_cells, ndim) shape.
 Return type
tf.Tensor


class
AtomicConvolution
(*args, **kwargs)[source]¶ Implements the atomic convolutional transform introduced in
Gomes, Joseph, et al. “Atomic convolutional networks for predicting proteinligand binding affinity.” arXiv preprint arXiv:1703.10603 (2017).
At a high level, this transform performs a graph convolution on the nearest neighbors graph in 3D space.

__init__
(atom_types=None, radial_params=[], boxsize=None, **kwargs)[source]¶ Atomic convolution layer
N = max_num_atoms, M = max_num_neighbors, B = batch_size, d = num_features l = num_radial_filters * num_atom_types
 Parameters
atom_types (list or None) – Of length a, where a is number of atom types for filtering.
radial_params (list) – Of length l, where l is number of radial filters learned.
boxsize (float or None) – Simulation box length [Angstrom].

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶  Parameters
X (tf.Tensor of shape (B, N, d)) – Coordinates/features.
Nbrs (tf.Tensor of shape (B, N, M)) – Neighbor list.
Nbrs_Z (tf.Tensor of shape (B, N, M)) – Atomic numbers of neighbor atoms.
 Returns
layer – A new tensor representing the output of the atomic conv layer
 Return type
tf.Tensor of shape (B, N, l)

radial_symmetry_function
(R, rc, rs, e)[source]¶ Calculates radial symmetry function.
B = batch_size, N = max_num_atoms, M = max_num_neighbors, d = num_filters
 Parameters
R (tf.Tensor of shape (B, N, M)) – Distance matrix.
rc (float) – Interaction cutoff [Angstrom].
rs (float) – Gaussian distance matrix mean.
e (float) – Gaussian distance matrix width.
 Returns
retval – Radial symmetry function (before summation)
 Return type
tf.Tensor of shape (B, N, M)

radial_cutoff
(R, rc)[source]¶ Calculates radial cutoff matrix.
B = batch_size, N = max_num_atoms, M = max_num_neighbors
 Parameters
[B (R) – Distance matrix.
N (tf.Tensor) – Distance matrix.
M] (tf.Tensor) – Distance matrix.
rc (tf.Variable) – Interaction cutoff [Angstrom].
 Returns
FC [B, N, M] – Radial cutoff matrix.
 Return type
tf.Tensor

gaussian_distance_matrix
(R, rs, e)[source]¶ Calculates gaussian distance matrix.
B = batch_size, N = max_num_atoms, M = max_num_neighbors
 Parameters
[B (R) – Distance matrix.
N (tf.Tensor) – Distance matrix.
M] (tf.Tensor) – Distance matrix.
rs (tf.Variable) – Gaussian distance matrix mean.
e (tf.Variable) – Gaussian distance matrix width (e = .5/std**2).
 Returns
retval [B, N, M] – Gaussian distance matrix.
 Return type
tf.Tensor

distance_tensor
(X, Nbrs, boxsize, B, N, M, d)[source]¶ Calculates distance tensor for batch of molecules.
B = batch_size, N = max_num_atoms, M = max_num_neighbors, d = num_features
 Parameters
X (tf.Tensor of shape (B, N, d)) – Coordinates/features tensor.
Nbrs (tf.Tensor of shape (B, N, M)) – Neighbor list tensor.
boxsize (float or None) – Simulation box length [Angstrom].
 Returns
D – Coordinates/features distance tensor.
 Return type
tf.Tensor of shape (B, N, M, d)

Part of a sluice network. Adds alpha parameters to control sharing between the main and auxillary tasks
Factory method AlphaShare should be used for construction
 Parameters
in_layers (list of Layers or tensors) – tensors in list must be the same size and list must include two or more tensors
 Returns
out_tensor (a tensor with shape [len(in_layers), x, y] where x, y were the original layer dimensions)
Distance matrix.
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.
Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).
This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.

class
SluiceLoss
(*args, **kwargs)[source]¶ Calculates the loss in a Sluice Network Every input into an AlphaShare should be used in SluiceLoss

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.

Part of a sluice network. Adds beta params to control which layer outputs are used for prediction
 Parameters
in_layers (list of Layers or tensors) – tensors in list must be the same size and list must include two or more tensors
 Returns
output_layers – Distance matrix.
 Return type
list of Layers or tensors with same size as in_layers
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.
Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).
Size of input layers must all be the same

class
ANIFeat
(*args, **kwargs)[source]¶ Performs transform from 3D coordinates to ANI symmetry functions

__init__
(max_atoms=23, radial_cutoff=4.6, angular_cutoff=3.1, radial_length=32, angular_length=8, atom_cases=[1, 6, 7, 8, 16], atomic_number_differentiated=True, coordinates_in_bohr=True, **kwargs)[source]¶ Only X can be transformed

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.


class
GraphEmbedPoolLayer
(*args, **kwargs)[source]¶ GraphCNNPool Layer from Robust Spatial Filtering with Graph Convolutional Neural Networks https://arxiv.org/abs/1703.00792
This is a learnable pool operation It constructs a new adjacency matrix for a graph of specified number of nodes.
This differs from our other pool operations which set vertices to a function value without altering the adjacency matrix.
..math:: V_{emb} = SpatialGraphCNN({V_{in}}) ..math:: V_{out} = sigma(V_{emb})^{T} * V_{in} ..math:: A_{out} = V_{emb}^{T} * A_{in} * V_{emb}

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶  Parameters
num_filters (int) – Number of filters to have in the output
in_layers (list of Layers or tensors) –
[V, A, mask] V are the vertex features must be of shape (batch, vertex, channel)
 A are the adjacency matrixes for each graph
Shape (batch, from_vertex, adj_matrix, to_vertex)
mask is optional, to be used when not every graph has the same number of vertices
 Returns
Returns a tf.tensor with a graph convolution applied
The shape will be (batch, vertex, self.num_filters).


class
GraphCNN
(*args, **kwargs)[source]¶ GraphCNN Layer from Robust Spatial Filtering with Graph Convolutional Neural Networks https://arxiv.org/abs/1703.00792
Spatialdomain convolutions can be defined as H = h_0I + h_1A + h_2A^2 + … + hkAk, H ∈ R**(N×N)
We approximate it by H ≈ h_0I + h_1A
We can define a convolution as applying multiple these linear filters over edges of different types (think up, down, left, right, diagonal in images) Where each edge type has its own adjacency matrix H ≈ h_0I + h_1A_1 + h_2A_2 + … h_(L−1)A_(L−1)
V_out = sum_{c=1}^{C} H^{c} V^{c} + b

__init__
(num_filters, **kwargs)[source]¶  Parameters
num_filters (int) – Number of filters to have in the output
in_layers (list of Layers or tensors) –
[V, A, mask] V are the vertex features must be of shape (batch, vertex, channel)
 A are the adjacency matrixes for each graph
Shape (batch, from_vertex, adj_matrix, to_vertex)
mask is optional, to be used when not every graph has the same number of vertices
Returns (tf.tensor) –
a tf.tensor with a graph convolution applied (Returns) –
shape will be (batch (The) –
vertex –
self.num_filters) –

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
Highway
(*args, **kwargs)[source]¶ Create a highway layer. y = H(x) * T(x) + x * (1  T(x))
H(x) = activation_fn(matmul(W_H, x) + b_H) is the nonlinear transformed output T(x) = sigmoid(matmul(W_T, x) + b_T) is the transform gate
Implementation based on paper
Srivastava, Rupesh Kumar, Klaus Greff, and Jürgen Schmidhuber. “Highway networks.” arXiv preprint arXiv:1505.00387 (2015).
This layer expects its input to be a two dimensional tensor of shape (batch size, # input features). Outputs will be in the same shape.

__init__
(activation_fn='relu', biases_initializer='zeros', weights_initializer=None, **kwargs)[source]¶  Parameters
activation_fn (object) – the Tensorflow activation function to apply to the output
biases_initializer (callable object) – the initializer for bias values. This may be None, in which case the layer will not include biases.
weights_initializer (callable object) – the initializer for weight values

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
WeaveLayer
(*args, **kwargs)[source]¶ This class implements the core Weave convolution from the Google graph convolution paper [1]_
This model contains atom features and bond features separately.Here, bond features are also called pair features. There are 2 types of transformation, atom>atom, atom>pair, pair>atom, pair>pair that this model implements.
Examples
This layer expects 4 inputs in a list of the form [atom_features, pair_features, pair_split, atom_to_pair]. We’ll walk through the structure of these inputs. Let’s start with some basic definitions.
>>> import deepchem as dc >>> import numpy as np
Suppose you have a batch of molecules
>>> smiles = ["CCC", "C"]
Note that there are 4 atoms in total in this system. This layer expects its input molecules to be batched together.
>>> total_n_atoms = 4
Let’s suppose that we have a featurizer that computes n_atom_feat features per atom.
>>> n_atom_feat = 75
Then conceptually, atom_feat is the array of shape (total_n_atoms, n_atom_feat) of atomic features. For simplicity, let’s just go with a random such matrix.
>>> atom_feat = np.random.rand(total_n_atoms, n_atom_feat)
Let’s suppose we have n_pair_feat pairwise features
>>> n_pair_feat = 14
For each molecule, we compute a matrix of shape (n_atoms*n_atoms, n_pair_feat) of pairwise features for each pair of atoms in the molecule. Let’s construct this conceptually for our example.
>>> pair_feat = [np.random.rand(3*3, n_pair_feat), np.random.rand(1*1, n_pair_feat)] >>> pair_feat = np.concatenate(pair_feat, axis=0) >>> pair_feat.shape (10, 14)
pair_split is an index into pair_feat which tells us which atom each row belongs to. In our case, we hve
>>> pair_split = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2, 3])
That is, the first 9 entries belong to “CCC” and the last entry to “C”. The final entry atom_to_pair goes in a little more indepth than pair_split and tells us the precise pair each pair feature belongs to. In our case
>>> atom_to_pair = np.array([[0, 0], ... [0, 1], ... [0, 2], ... [1, 0], ... [1, 1], ... [1, 2], ... [2, 0], ... [2, 1], ... [2, 2], ... [3, 3]])
Let’s now define the actual layer
>>> layer = WeaveLayer()
And invoke it
>>> [A, P] = layer([atom_feat, pair_feat, pair_split, atom_to_pair])
The weave layer produces new atom/pair features. Let’s check their shapes
>>> A = np.array(A) >>> A.shape (4, 50) >>> P = np.array(P) >>> P.shape (10, 50)
The 4 is total_num_atoms and the 10 is the total number of pairs. Where does 50 come from? It’s from the default arguments n_atom_input_feat and n_pair_input_feat.
References
 1
Kearnes, Steven, et al. “Molecular graph convolutions: moving beyond
fingerprints.” Journal of computeraided molecular design 30.8 (2016): 595608.

__init__
(n_atom_input_feat: int = 75, n_pair_input_feat: int = 14, n_atom_output_feat: int = 50, n_pair_output_feat: int = 50, n_hidden_AA: int = 50, n_hidden_PA: int = 50, n_hidden_AP: int = 50, n_hidden_PP: int = 50, update_pair: bool = True, init: str = 'glorot_uniform', activation: str = 'relu', batch_normalize: bool = True, batch_normalize_kwargs: Dict = {'renorm': True}, **kwargs)[source]¶  Parameters
n_atom_input_feat (int, optional (default 75)) – Number of features for each atom in input.
n_pair_input_feat (int, optional (default 14)) – Number of features for each pair of atoms in input.
n_atom_output_feat (int, optional (default 50)) – Number of features for each atom in output.
n_pair_output_feat (int, optional (default 50)) – Number of features for each pair of atoms in output.
n_hidden_AA (int, optional (default 50)) – Number of units(convolution depths) in corresponding hidden layer
n_hidden_PA (int, optional (default 50)) – Number of units(convolution depths) in corresponding hidden layer
n_hidden_AP (int, optional (default 50)) – Number of units(convolution depths) in corresponding hidden layer
n_hidden_PP (int, optional (default 50)) – Number of units(convolution depths) in corresponding hidden layer
update_pair (bool, optional (default True)) – Whether to calculate for pair features, could be turned off for last layer
init (str, optional (default 'glorot_uniform')) – Weight initialization for filters.
activation (str, optional (default 'relu')) – Activation function applied
batch_normalize (bool, optional (default True)) – If this is turned on, apply batch normalization before applying activation functions on convolutional layers.
batch_normalize_kwargs (Dict, optional (default {renorm=True})) – Batch normalization is a complex layer which has many potential argumentswhich change behavior. This layer accepts userdefined parameters which are passed to all BatchNormalization layers in WeaveModel, WeaveLayer, and WeaveGather.

class
WeaveGather
(*args, **kwargs)[source]¶ Implements the weavegathering section of weave convolutions.
Implements the gathering layer from [1]_. The weave gathering layer gathers peratom features to create a moleculelevel fingerprint in a weave convolutional network. This layer can also performs Gaussian histogram expansion as detailed in [1]_. Note that the gathering function here is simply addition as in [1]_>
Examples
This layer expects 2 inputs in a list of the form [atom_features, pair_features]. We’ll walk through the structure of these inputs. Let’s start with some basic definitions.
>>> import deepchem as dc >>> import numpy as np
Suppose you have a batch of molecules
>>> smiles = ["CCC", "C"]
Note that there are 4 atoms in total in this system. This layer expects its input molecules to be batched together.
>>> total_n_atoms = 4
Let’s suppose that we have n_atom_feat features per atom.
>>> n_atom_feat = 75
Then conceptually, atom_feat is the array of shape (total_n_atoms, n_atom_feat) of atomic features. For simplicity, let’s just go with a random such matrix.
>>> atom_feat = np.random.rand(total_n_atoms, n_atom_feat)
We then need to provide a mapping of indices to the atoms they belong to. In ours case this would be
>>> atom_split = np.array([0, 0, 0, 1])
Let’s now define the actual layer
>>> gather = WeaveGather(batch_size=2, n_input=n_atom_feat) >>> output_molecules = gather([atom_feat, atom_split]) >>> len(output_molecules) 2
References
 1
Kearnes, Steven, et al. “Molecular graph convolutions: moving beyond
fingerprints.” Journal of computeraided molecular design 30.8 (2016): 595608.
Note
This class requires tensorflow_probability to be installed.

__init__
(batch_size: int, n_input: int = 128, gaussian_expand: bool = True, compress_post_gaussian_expansion: bool = False, init: str = 'glorot_uniform', activation: str = 'tanh', **kwargs)[source]¶  Parameters
batch_size (int) – number of molecules in a batch
n_input (int, optional (default 128)) – number of features for each input molecule
gaussian_expand (boolean, optional (default True)) – Whether to expand each dimension of atomic features by gaussian histogram
compress_post_gaussian_expansion (bool, optional (default False)) – If True, compress the results of the Gaussian expansion back to the original dimensions of the input by using a linear layer with specified activation function. Note that this compression was not in the original paper, but was present in the original DeepChem implementation so is left present for backwards compatibility.
init (str, optional (default 'glorot_uniform')) – Weight initialization for filters if compress_post_gaussian_expansion is True.
activation (str, optional (default 'tanh')) – Activation function applied for filters if compress_post_gaussian_expansion is True. Should be recognizable by tf.keras.activations.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs: List) → List[source]¶ Creates weave tensors.
 Parameters
inputs (List) – Should contain 2 tensors [atom_features, atom_split]
 Returns
output_molecules – Each entry in this list is of shape (self.n_inputs,)
 Return type
List

gaussian_histogram
(x)[source]¶ Expands input into a set of gaussian histogram bins.
 Parameters
x (tf.Tensor) – Of shape (N, n_feat)
Examples
This method uses 11 bins spanning portions of a Gaussian with zero mean and unit standard deviation.
>>> gaussian_memberships = [(1.645, 0.283), (1.080, 0.170), ... (0.739, 0.134), (0.468, 0.118), ... (0.228, 0.114), (0., 0.114), ... (0.228, 0.114), (0.468, 0.118), ... (0.739, 0.134), (1.080, 0.170), ... (1.645, 0.283)]
We construct a Gaussian at gaussian_memberships[i][0] with standard deviation gaussian_memberships[i][1]. Each feature in x is assigned the probability of falling in each Gaussian, and probabilities are normalized across the 11 different Gaussians.
 Returns
outputs – Of shape (N, 11*n_feat)
 Return type
tf.Tensor

class
DTNNEmbedding
(*args, **kwargs)[source]¶ 
__init__
(n_embedding=30, periodic_table_length=30, init='glorot_uniform', **kwargs)[source]¶  Parameters
n_embedding (int, optional) – Number of features for each atom
periodic_table_length (int, optional) – Length of embedding, 83=Bi
init (str, optional) – Weight initialization for filters.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
DTNNStep
(*args, **kwargs)[source]¶ 
__init__
(n_embedding=30, n_distance=100, n_hidden=60, init='glorot_uniform', activation='tanh', **kwargs)[source]¶  Parameters
n_embedding (int, optional) – Number of features for each atom
n_distance (int, optional) – granularity of distance matrix
n_hidden (int, optional) – Number of nodes in hidden layer
init (str, optional) – Weight initialization for filters.
activation (str, optional) – Activation function applied

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
DTNNGather
(*args, **kwargs)[source]¶ 
__init__
(n_embedding=30, n_outputs=100, layer_sizes=[100], output_activation=True, init='glorot_uniform', activation='tanh', **kwargs)[source]¶  Parameters
n_embedding (int, optional) – Number of features for each atom
n_outputs (int, optional) – Number of features for each molecule(output)
layer_sizes (list of int, optional(default=[1000])) – Structure of hidden layer(s)
init (str, optional) – Weight initialization for filters.
activation (str, optional) – Activation function applied

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
DAGLayer
(*args, **kwargs)[source]¶ DAG computation layer.
This layer generates a directed acyclic graph for each atom in a molecule. This layer is based on the algorithm from the following paper:
Lusci, Alessandro, Gianluca Pollastri, and Pierre Baldi. “Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for druglike molecules.” Journal of chemical information and modeling 53.7 (2013): 15631575.
This layer performs a sort of inward sweep. Recall that for each atom, a DAG is generated that “points inward” to that atom from the undirected molecule graph. Picture this as “picking up” the atom as the vertex and using the natural tree structure that forms from gravity. The layer “sweeps inwards” from the leaf nodes of the DAG upwards to the atom. This is batched so the transformation is done for each atom.

__init__
(n_graph_feat=30, n_atom_feat=75, max_atoms=50, layer_sizes=[100], init='glorot_uniform', activation='relu', dropout=None, batch_size=64, **kwargs)[source]¶  Parameters
n_graph_feat (int, optional) – Number of features for each node(and the whole grah).
n_atom_feat (int, optional) – Number of features listed per atom.
max_atoms (int, optional) – Maximum number of atoms in molecules.
layer_sizes (list of int, optional(default=[100])) – List of hidden layer size(s): length of this list represents the number of hidden layers, and each element is the width of corresponding hidden layer.
init (str, optional) – Weight initialization for filters.
activation (str, optional) – Activation function applied.
dropout (float, optional) – Dropout probability in hidden layer(s).
batch_size (int, optional) – number of molecules in a batch.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.


class
DAGGather
(*args, **kwargs)[source]¶ 
__init__
(n_graph_feat=30, n_outputs=30, max_atoms=50, layer_sizes=[100], init='glorot_uniform', activation='relu', dropout=None, **kwargs)[source]¶ DAG vector gathering layer
 Parameters
n_graph_feat (int, optional) – Number of features for each atom.
n_outputs (int, optional) – Number of features for each molecule.
max_atoms (int, optional) – Maximum number of atoms in molecules.
layer_sizes (list of int, optional) – List of hidden layer size(s): length of this list represents the number of hidden layers, and each element is the width of corresponding hidden layer.
init (str, optional) – Weight initialization for filters.
activation (str, optional) – Activation function applied.
dropout (float, optional) – Dropout probability in the hidden layer(s).

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
MessagePassing
(*args, **kwargs)[source]¶ General class for MPNN default structures built according to https://arxiv.org/abs/1511.06391

__init__
(T, message_fn='enn', update_fn='gru', n_hidden=100, **kwargs)[source]¶  Parameters
T (int) – Number of message passing steps
message_fn (str, optional) – message function in the model
update_fn (str, optional) – update function in the model
n_hidden (int, optional) – number of hidden units in the passing phase

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).


class
EdgeNetwork
(*args, **kwargs)[source]¶ Submodule for Message Passing

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
GatedRecurrentUnit
(*args, **kwargs)[source]¶ Submodule for Message Passing

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ This is where the layer’s logic lives.
Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.
 Parameters
inputs – Input tensor, or list/tuple of input tensors.
**kwargs – Additional keyword arguments. Currently unused.
 Returns
A tensor or list/tuple of tensors.


class
SetGather
(*args, **kwargs)[source]¶ set2set gather layer for graphbased model
Models using this layer must set pad_batches=True.

__init__
(M, batch_size, n_hidden=100, init='orthogonal', **kwargs)[source]¶  Parameters
M (int) – Number of LSTM steps
batch_size (int) – Number of samples in a batch(all batches must have same size)
n_hidden (int, optional) – number of hidden units in the passing phase

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns
Python dictionary.

build
(input_shape)[source]¶ Creates the variables of the layer (optional, for subclass implementers).
This is a method that implementers of subclasses of Layer or Model can override if they need a statecreation step inbetween layer instantiation and layer call.
This is typically used to create the weights of Layer subclasses.
 Parameters
input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call
(inputs)[source]¶ Perform M steps of set2set gather,
Detailed descriptions in: https://arxiv.org/abs/1511.06391


cosine_dist
(x, y)[source]¶ Computes the inner product (cosine similarity) between two tensors.
This assumes that the two input tensors contain rows of vectors where each column represents a different feature. The output tensor will have elements that represent the inner product between pairs of normalized vectors in the rows of x and y. The two tensors need to have the same number of columns, because one cannot take the dot product between vectors of different lengths. For example, in sentence similarity and sentence classification tasks, the number of columns is the embedding size. In these tasks, the rows of the input tensors would be different test vectors or sentences. The input tensors themselves could be different batches. Using vectors or tensors of all 0s should be avoided.

The vectors in the input tensors are first l2normalized such that each vector

has length or magnitude of 1. The inner product (dot product) is then taken

between corresponding pairs of row vectors in the input tensors and returned.
Examples
The cosine similarity between two equivalent vectors will be 1. The cosine similarity between two equivalent tensors (tensors where all the elements are the same) will be a tensor of 1s. In this scenario, if the input tensors x and y are each of shape (n,p), where each element in x and y is the same, then the output tensor would be a tensor of shape (n,n) with 1 in every entry.
>>> import tensorflow as tf >>> import deepchem.models.layers as layers >>> x = tf.ones((6, 4), dtype=tf.dtypes.float32, name=None) >>> y_same = tf.ones((6, 4), dtype=tf.dtypes.float32, name=None) >>> cos_sim_same = layers.cosine_dist(x,y_same)
x and y_same are the same tensor (equivalent at every element, in this case 1). As such, the pairwise inner product of the rows in x and y will always be 1. The output tensor will be of shape (6,6).
>>> diff = cos_sim_same  tf.ones((6, 6), dtype=tf.dtypes.float32, name=None) >>> tf.reduce_sum(diff) == 0 # True <tf.Tensor: shape=(), dtype=bool, numpy=True> >>> cos_sim_same.shape TensorShape([6, 6])
The cosine similarity between two orthogonal vectors will be 0 (by definition). If every row in x is orthogonal to every row in y, then the output will be a tensor of 0s. In the following example, each row in the tensor x1 is orthogonal to each row in x2 because they are halves of an identity matrix.
>>> identity_tensor = tf.eye(512, dtype=tf.dtypes.float32) >>> x1 = identity_tensor[0:256,:] >>> x2 = identity_tensor[256:512,:] >>> cos_sim_orth = layers.cosine_dist(x1,x2)
Each row in x1 is orthogonal to each row in x2. As such, the pairwise inner product of the rows in x1`and `x2 will always be 0. Furthermore, because the shape of the input tensors are both of shape (256,512), the output tensor will be of shape (256,256).
>>> tf.reduce_sum(cos_sim_orth) == 0 # True <tf.Tensor: shape=(), dtype=bool, numpy=True> >>> cos_sim_orth.shape TensorShape([256, 256])
 Parameters
x (tf.Tensor) – Input Tensor of shape (n, p). The shape of this input tensor should be n rows by p columns. Note that n need not equal m (the number of rows in y).
y (tf.Tensor) – Input Tensor of shape (m, p) The shape of this input tensor should be m rows by p columns. Note that m need not equal n (the number of rows in x).
 Returns
Returns a tensor of shape (n, m), that is, n rows by m columns. Each i,jth entry of this output tensor is the inner product between the l2normalized ith row of the input tensor x and the the l2normalized jth row of the output tensor y.
 Return type
tf.Tensor
