1. Wrap-up
In this course, I've tried to focus on designing network topologies that can be used to solve interesting problems.
2. So far...
You learned how to build functional keras models, including advanced topics such as shared layers, categorical embeddings, multiple inputs, and multiple outputs.
You can now build a model capable of solving a regression and a classification problem simultaneously.
3. Shared layers
In this final video, I'd like to discuss some real-world uses cases of what you've learned.
Shared layers are incredibly useful when you want to compare two things.
For example, we compared two basketball teams to decide how different they were in their ability to score points in a basketball game, using a shared embedding model.
In academic research, shared models are known as "siamese networks", which are used to calculate things like document similarity, using shared embedding layer and a shared long-short-term memory layer (or LSTM layer), and then comparing the LSTM outputs.
Since both documents are encoded with the same embedding layer and the same LSTM layer, the model learns a representation of the documents that can be used to compare them.
Here are some links for further reading.
4. Multiple inputs
Multiple input networks are especially useful when you want to process different types of data within your model.
For example, in our basketball models, we processed team IDs separately, using an embedding layer. For numeric data such as home vs away, we skipped the embedding step and passed it directly to the output.
You can extend this concept to build a network that, for example, uses an LSTM to process text, a standard Dense layer to process numerics, and a convolutional layer (or CNN) to process images.
I didn't cover LSTMs or CNNs in this course, but once you understand them, you can use this concept to make networks that understand both text and images!
5. Multiple outputs
The multiple output network you built in chapter 4 is the coolest model of the course. It can do both classification AND regression!
I like this model a lot because the regression problem turns out to be a lot easier than the classification problem. In the regression problem, the neural network gets penalized less for random chance. For example, a team that wins by 1 point got really lucky, and the model can use that information. However, in the classification model, winning by 1 point is the same as winning by 10 points.
I think it's pretty cool that we can use the output from the easier regression problem to help solve the more difficult classification problem.
6. Skip connections
Here's a final example that we didn't explicitly cover in the class, but that is trivially easy now that you know how to use a concatenate layer.
In a paper called "Visualizing the Loss Landscape of Neural Nets", Li et al. propose a method called "skip connections" to simplify the optimization of neural networks.
To summarize a very long and interesting paper, skip connections make it a lot easier for the adam optimizer to find the global minimum of the network's loss function.
In Keras, implementing a skip connection is as simple as using the "Concatenate" layer to concatenate the inputs to the deep network's outputs, right before the final output layer.
7. Best of luck!
I hope you enjoyed taking a deep dive into the keras functional API with me, and that the flexibility of these neural networks will give you a lot of creative ways to solve your data science problems.
Thank you for taking this course and best of luck building deep neural networks!