A really good video. President Obama seems very knowledgable about the state of modern artificial intelligence.
Today, the programmable chips that Burger and Lu believed would transform the world—called field programmable gate arrays—are here. FPGAs already underpin Bing, and in the coming weeks, they will drive new search algorithms based on deep neural networks—artificial intelligence modeled on the structure of the human brain—executing this AI several orders of magnitude faster than ordinary chips could. As in, 23 milliseconds instead of four seconds of nothing on your screen. FPGAs also drive Azure, the company’s cloud computing service. And in the coming years, almost every new Microsoft server will include an FPGA. That’s millions of machines across the globe. “This gives us massive capacity and enormous flexibility, and the economics work,” Burger says. “This is now Microsoft’s standard, worldwide architecture.”
Machine learning is a field of computer science that encompasses a vast amount of techniques, algorithms and disciplines. Studying machine learning can be daunting at first, mainly due to the sheer amount of different topics on offer. But don’t let that deter you! Stick with it and you’ll discover how to do some really amazing stuff.
In this post, I’ll share some tutorials, videos and online courses to help you get started.
Machine learning algorithms can be thought of as programs that produce other programs. These generate programs are expressed as a bunch of numbers, so some mathematics & statistics knowledge is required. When I forget the details of some equation or statistical concept, I usually refer to the OpenIntro Statistics book (the newest edition is available as a free PDF). It answers those questions that you always had but were to embarrassed to ask at school, and has a bunch of real-world problems and solutions instead of the silly ones found in most other books of its ilk.
What is machine learning?
Before learning about different types of algorithms, it’s a good idea to know what machine learning is, and what it isn’t. There are many excellent intro videos online, but of the ones I’ve watched, I’d recommend Frank Chen’s 45-minute video AI, Deep Learning, and Machine Learning: A Primer. It explains not only what machine learning is, but also gives a good overview of its history.
Another video I’d like to mention is Android Authority’s What is machine learning? It’s 11 minutes long and chock-full of examples. Some of these may not make sense to you is you are just starting to study machine learning, but don’t worry if there are some things you don’t understand.
Sounds good, where do I start?
There are several good ways to start learning about machine learning. Personally, I’d recommend starting with basic supervised learning techniques such as linear and logistic regression before moving on to more advanced algorithms. After that, I’d recommended delving into unsupervised learning.
The same approach is taken in Stanford’s Machine Learning course on Coursera. The lecturer is Andrew Ng (@AndrewYNg), formerly of Google Brain and now Chief Scientist at Baidu. Mr. Ng also happens to be the co-founder of Coursera itself, so it isn’t surprising that the pedagogical quality of the course is excellent. You’ll learn about stuff like linear regression, logistic regression, neural networks, SVMs, anomaly detection algorithms, and much more! The programming language used throughout the course is Octave, which some people love and some people hate (I don’t care for it). Either way, I highly recommend that you try to solve all the assignments. In the real world, there are excellent libraries that let you use learning algorithms successfully without implementing them from scratch; that said, implementing everything from scratch at least once will really help you understand what’s going on behind the scenes. It’ll make you more knowledgeable than many people working as data scientists. And even if you don’t correctly implement an algorithm from start to finish, trying to do so will at least make you appreciate the brilliant minds that came up with this stuff in the first place.
Where do I go from here?
Once you have are familiar with some of the basic machine learning algorithms out there, I’d recommended specialising in something you are particularly interested in. Trying to learn every single algorithm, technique and framework will only make you head spin. Focusing on a few algorithms will allow you to become expert in them; since many learning algorithms are driven my the same basic principles, you can easily adapt your knowledge to new topics when needed.
I’m personally interested in neural networks and Bayesian inference. For advanced neural networks, I recommended reading Quoc V. Le’s series on Deep Learning (part 1, part 2), which covers autoencoders, convolutional neural networks and recurrent neural networks including its amazing LSTM variant. It’s a great resource that explains things in plain English in addition to equations — something that precious few pieces of literature do.
For Bayesian inference, I’ve found the Bayesian for Hackers book to be invaluable. It’s a code-first resource, which is great if you have a programming background. The book is written as an interactive Jupyter Notebook, so you can mess around with the code whilst you read.
Another book I’ve heard great things about is Introduction to Bayesian Statistics.
A couple of tips
Machine leaning algorithms are exciting because the fundamentals—the way they learn—doesn’t change when they are used to solve different problems. If you want to classify a pieces of fruit based on weight, colour, surface texture et cetera or recognise handwritten digits from images, you can use the same core learning algorithms and simply tweak some stuff around it. For that reason, I highly recommend saving every bit of code you write to a repository of some sort. There’s no reason to re-invent the wheel; if you want to use a learning algorithm to solve some task and have previously done some work for a similar type of problem, chances are you’ll be able to reuse a lot of your old code.
Another tip I’d like to share is to keep a personal glossary. Machine learning as a field is full of jargon, acronyms and extremely poorly chosen names. Some things have several different names; others have highly misleading or confusing ones. Every time you stumble upon a term/name/acronym you’ve never heard of before, do a quick Google search and jot down a “for idiots” explanation. This will reduce the cognitive load as you study. It’ll also prove a valuable reference going forward, especially if you are like me an have a brain that isn’t always as coöperative as you would like.
Things are clearly progressing rapidly when it comes to machine intelligence. But how did we get here, after not one but multiple “A.I. winters”? What’s the breakthrough? And why is Silicon Valley buzzing about artificial intelligence again?
A great presentation by Frank Chen.
We’ve been building the best AI team and tools for years, and recent breakthroughs will allow us to do even more. This past March, DeepMind’s AlphaGo took on Lee Sedol, a legendary Go master, becoming the first program to beat a professional at the most complex game mankind ever devised. The implications for this victory are, literally, game changing—and the ultimate winner is humanity. This is another important step toward creating artificial intelligence that can help us in everything from accomplishing our daily tasks and travels, to eventually tackling even bigger challenges like climate change and cancer diagnosis.