First intro: Hacker News andÂ Udacity
My interest in ml stems back to 2014 when I started reading articles about it on Hacker News. I simply found the idea of teaching machines stuff by looking at data appealing. At the time I wasnâ€™t even a professional developer, but a hobby coder whoâ€™d done a couple of small projects.
So I began watching the first few chapters of Udacityâ€™s Supervised Learning course, while also reading all articles I came across on the subject.
This gave me a little bit of conceptual understanding, though no practical skills. I also didnâ€™t finish it, as I rarely do with MOOCâ€™s.
Failing Courseraâ€™s MLÂ Course
In January 2015 I joined the Founders and Coders (FAC) bootcamp in London in order to become a developer. A few weeks in, I wanted to learn how to actually code machine learning algorithms, so I started a study group with a few of my peers. Every Tuesday evening, weâ€™d watch lectures from Courseraâ€™s Machine Learning course.
Itâ€™s a fantastic course, and I learned a hell of a lot. But itâ€™s tough for a beginner. I had to watch the lectures over and over again before grasping the concepts. The Octave coding task are challenging as well, especially if you donâ€™t know Octave. As a result of the difficulty, one by one fell off the study group as the weeks passed. Eventually, I fell off it myself as well.
I hindsight, I should have started with a course that either used ml libraries for the coding tasksâ€Šâ€”â€Šas opposed to building the algorithms from scratchâ€Šâ€”â€Šor at least used a programming language I knew.
Learning a new language while also trying to code ml algorithms is too hard for aÂ newbie.
If I could go back in time, Iâ€™d choose Udacityâ€™s Intro to Machine Learning, as itâ€™s easier and uses Python and Scikit Learn. This way, we would have gotten our hands dirty as soon as possible, gained confidence, and had more fun.
Lesson learned: Start with something easy and practical rather than difficult and theoretical.
Machine Learning in aÂ Week
One of the last things I did at FAC was the ml week stunt. My goal was to be able to apply machine learning to actual problems at the end of the week, which I managed to do.
Throughout the week I did the following:
- got to know Scikit Learn
- tried ml on a real world dataset
- coded a linear regression algorithm from scratch (in Python)
- did a tiny bit of nlp
Itâ€™s by far the steepest ml learning curve Iâ€™ve ever experienced. Go ahead and read the article if you want a more detailed overview.
Lesson learned: Setting off a week solely to immerse yourself into a new subject is extremely effective.
Failing neuralÂ networks
After I finished FAC in London and moved back to Norway, I tried to repeat the success from the ml week, but for neural networks instead.
There were simply too many distractions to spend 10 hours of coding and learning every day. I had underestimated how important it was to be surrounded by peers at FAC.
Lesson learned: Find an thriving environment to surround yourself with when doing these kinds of learning stunts.
However, I got started with neural nets at least, and slowly started to grasp the concept. By July I managed to code my first net. Itâ€™s probably the crappiest implementation ever created, and I actually find it embarrassing to show off. But it did the trick; I proved to myself that I understood concepts like backpropagation and gradient descent.
In the second half of the year, my progression slowed down, as I started a new job. The most important takeaway from this period was the leap from non-vectorized to vectorized implementations of neural networks, which involved repeating linear algebra from university.
By the end of the year I wrote an article as a summary of how I learned this:
Testing out KaggleÂ Contests
During the christmas vacation of 2015, I got a motivational boost again and decided try out Kaggle. So I spent quite some time experimenting with various algorithms for their Homesite Quote Conversion, Otto Group Product Classification and Bike Sharing Demand contests.
The main takeaway from this was the experience of iteratively improving the results by experimenting with the algorithms and the data.
I learned to trust my logic when doing machine learning.
If tweaking a parameter or engineering a new feature seems like a good idea logically, itâ€™s quite likely that it actually will help.
Setting up a learning routine atÂ work
Back at work in January 2016 I wanted to continue in the flow Iâ€™d gotten into during Christmas. So I asked my manager if I could spend some time learning stuff during my work hours as well, which he happily approved.
Having gotten a basic understanding of neural networks at this point, I wanted to move on to deep learning.
Udacityâ€™s DeepÂ Learning
My first attempt was Udacityâ€™s Deep Learning course, which ended up as a big disappointment. The contents of the video lectures are good, but they are too short and shallow to me.
And the IPython Notebook assignments ended up being too frustrating, as I spent most of my time debugging code errors, which is the most effective way to kill motivation. So after doing that for a couple of sessions at work, I simply gave up.
To their defense, Iâ€™m a total noob when it comes to IPython Notebooks, so it might not be as bad for you as it was for me. So it might be that I simply wasnâ€™t ready for the course.
Stanfordâ€Šâ€”â€ŠDeep Learning forÂ NLP
Luckily, I then discovered Stanfordâ€™s CS224D and decided to give it a shot. It is a fantastic course. And though itâ€™s difficult, I never end up debugging when doing the problem sets.
Secondly, they actually give you the solution code as well, which I often look at when Iâ€™m stuck, so that I can work my way backwards to understand the steps needed to reach a solution.
Though Iâ€™ve havenâ€™t finished it yet, it has significantly boosted my knowledge in nlp and neural networks so far.
However itâ€™s been tough. Really tough. At one point, I realized I needed help from someone better than me, so I came in touch with a Ph.D student who was willing to help me out for 40 USD per hour, both with the problem sets as well as the overall understanding. This has been critical for me in order to move on, as he has uncovered a lot of black holes in my knowledge.
Lesson learned: Itâ€™s possible to get a good machine learning teacher for around 50 USD per hour. If you can afford it, itâ€™s definitely worth it.
In addition to this, Xeneta also hired a data scientist recently. Heâ€™s got a masters degree in math, so I often ask him for help when Iâ€™m stuck with various linear algebra an calculus tasks, or ml in general. So be sure to check out which resources you have internally in your company as well.
Boosting Sales atÂ Xeneta
After doing all this, I finally felt ready to do a ml project at work. It basically involved training an algorithm to qualify sales leads by reading company descriptions, and has actually proven to be a big time saver for the sales guys using the tool.
Check out out article I wrote about it below or head over to GitHub to dive straight into the code.
Getting to this point has surely been a long journey. But also a fast one; when I started my machine learning in a week project, I certainly didnâ€™t have any hopes of actually using it professionally within a year.
But itâ€™s 100 percent possible. And if I can do it, so can anybody else.