Deep Learning is not a new concept and it’s been around for decades. However, now we seem to be in another stage of hype surrounding it. Yet each time we seem to grow our expectations, there also seems to be a plateau that follows suit.
Whether we will return to a plateau is yet to be seen but we could expect it. After Geoff Hinton’s work on backpropagation became widely circulated in 1986, we saw a surge of interest surrounding the topic leading to buzzwords that surround us today like “A.I. - Path to the future”. Yet even after such interest back then, we saw that the industry failed to apply much of the techniques explained to effect and bring about actual value.
After all, business is business and if the results don’t tie themselves to revenue, then it doesn’t really matter that a machine recognized a cat instead of a dog. Personally, I think this is when Industrial Engineering techniques shone through. I like to call it the golden age of Industrial Engineering (aka 1970s-2000s). Because ultimately, the ideas behind both fields are quite similar in how they start out.
They start out with Math. In Industrial Engineering, the main aim is to create revenue. This I think enabled it to stand out over Machine Learning as the go-to tool for corporations who just weren’t willing to be patient with Machine Learning research. Industrial Engineering uses a lot of math to identify weak-chains in the system (bottlenecks as we like to call them), increase their possible output to improve overall output. In Neural networks, we see a similar technique of finding nodes that maybe hindering the network to produce the desired output of a recognition. So we change their properties. The difference is that while admittedly, there are a lot of differences between both fields besides a specific example laid out here, Industrial Enigneering techniques lead to quicker real-world results whereas Deep Learning was still trying to figure out how to walk back then.
That isn’t to say there isn’t great potential in Deep Learning. With today’s computers being able to process terabytes of information quickly, we are closer to an A.I. future as cliche as that sounds. Maybe the combination of these two fields could even bring out further new discoveries. Let’s hope so.