Designing from scratch

1. Designing from scratch

person: In this section, we'll apply what we've learned to a new use case. Let's pretend the head of a municipal transit system has contacted you to build a system that predicts the traffic levels on roads. As part of your preparation for this task, you're trying to thoroughly understand the business constraints in order to make the appropriate system design tradeoffs. The data available consists of sensors deployed all over the city, which record whenever a car passes by. For each sensor, we know where it is. We also know the characteristics of the road that it's on. What sort of training architecture is appropriate? Well, you should ask yourself, what is the relationship between the features and the labels? Is it more like physics or fashion? In this case, it's more like fashion trends. Cities are very complex systems. If a train stops service, people will still need to get home. Technology's also always changing. On demand taxi services have reshaped urban transit in ways we didn't anticipate a decade ago. And there are also episodic changes, like sports events and parades, for example. For dynamic relationships, we need to use dynamic training. Which sort of serving architecture is appropriate? Well, you should ask yourself, is the distribution of prediction requests likely to be more peaked or less peaked? In this case, it's likely to be more peaked. The distribution of demand is peaked, because it's likely to be dominated by the request for the most heavily trafficked roads. Is the cardinality of the set of all prediction requests likely to be low, moderate, high, or perhaps need more info? In this case, you'll need more info. Why is that? What does it depend upon? Consider historical traffic data, problem framing, or the variance of traffic levels. In this case, the answers are both historical traffic data and the problem framing, but not the variance of traffic levels. The reason that the cardinality depends upon the framing of the problem is that we don't know whether the task is to make predictions for every minute, hour, or day. And similarly, we don't know how big a region of space each prediction should correspond to. It could be anything from a few feet to a few blocks. As we learned in the first specialization, machine learning is all about generalization-- the leap of faith into unseen input. And what we don't know is whether our users wanna generalize in space, i.e. by making predictions far away from the sensors, in time, by making predictions in the future with finer granularity than the historical data, or both. In all likelihood, you'd start conservatively, which corresponds to a lower, rather than a higher, cardinality. Variance of traffic levels wouldn't matter, because that's a label and not a feature.

2. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.