Deeper Look at Sequential Model Building in TensorFlow

Let's revisit the model building command in TensorFlow in our "hello world" equivalent example.

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])

So Sequential lets you build up a model in layers (see Layer class, or tf.keras.Layer, that inherits from Operation). 

Layers are callable objects. In Python, a callable is any object that can be called using parentheses (optionally with arguments). Read the implementation here (in keras/source/layers/layer.py).

But what does the Flatten method/Layer do?

Dense creates a densely-connected NN Layer (convolutional neural network architecture). 

• The first argument is the positive integer units, representing the dimensionality of the output space
• The second argument is the activation function to use (if this is missing, no activation is applied which is actually linear activation a(x) = x)

Activation functions in a neural net introduces non-linearity into the network.  

• Essentially these functions work with neurons and transform neural computations into output signals
• ReLU (rectified linear unit) is one of the most widely used activation functions in neural networks (f(x) = max(0,x)).