HN Gopher Feed (2017-12-14) - page 1 of 10
Machine Learning 101 slidedeck: 2 years of headbanging, so you
don't have to
1055 points by flor1shttps://docs.google.com/presentation/d/1kSuQyW5DTnkVaZEjGYCkfOxv...0
candiodari - 10 hours ago
Cool presentation ... but there's a million ones like this. We
don't need yet another basic introduction to machine learning, we
need detailed practical studies of real problems.
gregorymichael - 9 hours ago
Easier to curse the darkness than to light a match.
mr_toad - 10 hours ago
Animats - 11 hours ago
Loading...Google Slides is really slow. That's why this needs two
hours.Most of the real content is in linked videos.
hprotagonist - 13 hours ago
I was absolutely convinced by the title that this would be a link
to a research blog post about an analysis of hair motion at metal
erk__ - 12 hours ago
Yeah like how you should do it if you don't want to make damage
to your neck.
elicox - 13 hours ago
Me too; After I read Perceptron I fast forward lol
Jaruzel - 2 hours ago
Reading normally, and skipping the videos, the whole deck takes
about 15 minutes. The last 3rd of the slides are basically
promotional material for the various Cloud ML services that are out
there.It's nice deck, but I'd hoped the blue slides went more
technical without dropping out to various videos. If wanted videos,
I'd go to youtube directly. Not everyone wants to learn through
watching people talk. I learn best when I read, it's unfortunate
that youngsters these days think that the written word is now a
poor cousin to flashy video.In the same way that new clothes
are no longer for me, and new music is no longer for me, and all
good TV shows and films are full of people half my age, I also now
feel that I'm being aged off the internet.I was here first, you
young whippersnappers! It's MY lawn.
rkagerer - 11 hours ago
Not bad but toward the end it basically just becomes a big pitch
for Google's ML products. It links to 3Blue1Brown's videos which
XR0CSWV3h3kZWg - 13 hours ago
I really wish that they hadn't decided on having moving images
behind the text you are supposed to be reading.
fredley - 2 hours ago
> 2 years of headbangingShame the backgrounds gave me a headache
yathern - 13 hours ago
Agreed - but that's only a small portion in the beginning. The
bulk of it is fine.
carlmr - 7 hours ago
Give us your undivided attention.Does everything to distract us.
vowelless - 7 hours ago
Slide 64: A whole tonne of stuff going on in robotics right now.
Just take a look at Boston Dynamics YT channel for some mind
bloding research, most of which is driven by ML..I highly doubt
that BD is doing any ML work right now ... Can the author link to
specific research that they are doing using ML?
iamwil - 6 hours ago
As I remember, they don't use any deep learning ML. I think their
stuff is based on something about funnels.
fnl - 7 hours ago
A bit extreme, but yes, saying that robotics is driven my ML is
natch - 7 hours ago
You mean public work perhaps? I imagine they are doing a lot with
vision, gait learning, object manipulation, task planning,
autonomy, multi-robot coordination, etc. all of which can be
enabled by or at least helped along by machine learning, no? Your
request for links is valid, I just am surprised anyone would
doubt that they are doing ML research unless you are thinking of
a strangely narrow definition of ML.
charlysl - 9 hours ago
"There is no golden road to geometry"If you really want to
understand you would be much better off starting
iamwil - 6 hours ago
What was he headbanging about in these last 2 years? Just a
kbart - 7 hours ago
Information is great, but it would be much more readable in simple
text form or pdf. It's strange that senior creative engineer at
Google doesn't know presentation making basics.
quacker - 6 hours ago
It's not surprising that a Google engineer would use Google docs.
It's at least easily shareable and there are complementary
embedded videos that aren't suitable for text/PDF anyway.Though,
the options to export as a PDF didn't work for me (either via
download or as an export to Google Drive). I'm assuming the
presentation is too big.
JepZ - 1 hours ago
Same here, just tried to download the PDF multiple times but it
always failed (didn't even start; tested Chrome and Firefox)...
donkeyd - 5 hours ago
I created nearly the same presentation this week. It's good to see
that I didn't miss much, though this one goes deeper, which I don't
do on purpose. I'll probably send the attendee this presentation
afterwards for those who want to go deeper, very cool!
mrweasel - 2 hours ago
I honestly don't really see the value in a slidedeck, without the
accompanying talk. It's the same as when someone proclaim: "Slides
from this talk is available online", yeah that's not really any
good without video or audio.
minikomi - 2 hours ago
Idea: Neural network generated infinite sidedeck which appears to
follow a coherent narrative, which is ever so slightly out of
nerdponx - 2 hours ago
Disagree. Slides, even out of context slides, can be an excellent
source of information for people new to a field: they still give
a sense of the structure of the talk (what is related to what),
and they give you keywords to start searching for. And for
experienced readers, sometimes they just contain nice ideas or
tips you had not been aware of.
cyberpunk0 - 1 hours ago
No they aren't. Slides are a skeleton. Just bullet points that
99% of the time offer no usable information
pruthvishetty - 13 hours ago
AdReVice - 6 hours ago
Why is the file protected.. how to access it
minimaxir - 13 hours ago
The presentation goes straight from from linear regression and
classification to computer vision and reinforcement learning.The
practical value of ML/AI is what?s in between and is something that
isn?t often discussed between all the hype. ML/AI can be used to
build models which work well with nontabular data (e.g. text and
images), and can solve such regression/classification problems more
cleanly. (and with tools like Keras, they?re as easy to train and
deploy as a normal model)
flor1s - 13 hours ago
I think slide 12 touches on this. Even in the case of an image we
can process it pixel by pixel, but that would be lunacy!For text
great results have been achieved using automatons, but they only
work for structured strings and break if you add only a little
bit of noise.I feel like ML should be considered whenever you
feel like programming something requires you to deal with many
different cases, you have a lot of example data available, and
having some false positives / true negatives is not a big
ForFreedom - 7 hours ago
How can I download this slide?
trishmapow2 - 3 hours ago
Going to the /export/pdf link shows access denied. My quick 3min
workaround was to use FF dev tools, settings enable screenshot,
click through and save each page.
natch - 7 hours ago
Just use the download button under the gear icon. /sUnfortunately
it doesn't work I guess.Oh, and he says he is watching you...
Maybe he really means this? Maybe that's why he disabled
ForFreedom - 6 hours ago
I clicked the gear I clicked the PDF/ PPTX nothing seems to be
happening.. I think, I just became "Person of Interest"
natch - 5 hours ago
Yep same here.
elephant_burger - 4 hours ago
Thank you for this. This is an excellent slide deck
nblavoie - 12 hours ago
The document is awesome, but the animated backgrounds are
ausjke - 12 hours ago
exactly, I stopped at the second slide because of that. "I ask
for your undivided attention for two hours" is what it says, the
background animation seems not helping that goal, quite the
kmax12 - 11 hours ago
As someone who works with a lot of people new to machine learning,
I appreciate guides like this. I especially like the early slides
that help frame AI vs ML vs DL so that people can have a realistic
understanding of what these technologies are for.For my part, one
of the biggest realization I had after many years of applying
machine learning was that I got too caught up in the machine
learning algorithms themselves. I was often way too eager to guess
and check across different algorithms and parameters in search of
higher accuracy. Fortunately, there are new automated tools today
that can do that automatically.However, the key piece of advice I'd
give someone new to machine learning is not to get caught up in the
different machine learning techniques (SVM vs random forrest vs
neural network, etc). Instead (1) spend more time on translating
your problem into terms a machine can understand (i.e how are you
defining and generating your labels) and (2) how do you perform
feature engineering so the the right variables are available for
machine learning to use. Focusing on these two things helped me
build more accurate models that were more likely to be deployed in
the real world.Feature engineering in particular has become a bit
of a passion of mine since that realization. I currently work on an
open source project called Featuretools
(https://github.com/featuretools/featuretools/) that aims help
people apply feature engineering to transactional or relational
datasets. We just put out a tutorial on building models to predict
what product a customer will buy next, which is a good hands on
example to learn from
xzzherculeszzx - 10 hours ago
What are best resources for "defining and generating" labels?
kmax12 - 10 hours ago
I don't know of a definitive public resource for this. I
published a paper in IEEE's Data Science and Advanced Analytics
conference on it back in 2016. You can find that here:
content/uploads/2017/10/Pred_eng...Additionally, my company
(link in profile) builds a commercial product to help people
define and iterate on prediction problems in a structured way
based off of the ideas in that paper.
nerdponx - 2 hours ago
You need a good random sample and lots of manpower to manually
label them. Mechanical Turk  is one place to go for that man
power if you don't have a "grunt work" team and are not willing
to spend a few days doing it yourself.There are also some
methodologies out there that can help you label data sets more
efficiently. I don't often see them used, but they exist. Look
up "active learning" and "semi-supervised learning".:
bitL - 8 hours ago
Deep Learning frees you from the need to do "feature engineering"
and usually works much better than methods which require such
process. I'd instead recommend everyone to "dive deep" into deep
learning and once they master it, get acquainted with classical
methods that still might get used here and there. I understand
it's difficult to let go of what you worked very hard to
understand when you were studying ML, but such is life, "sunk
cost fallacy" should not you blind from seeing 95% success rate
of DL while observing paltry 60% success rate with SVM/HMM on the
same problem. Just let it go.
peterhunt - 7 hours ago
This might be true for CV and speech recognition and synthesis,
but there are huge categories of problems (dare I say, the
majority of industry use of ML) that are either working with
time series data (which DL hasn?t had great success with) or
must be highly explainable and tunable.
fnl - 7 hours ago
Or you don't have millions of annotated examples to learn
from, and no similar problem to transfer from...
agibsonccc - 7 hours ago
I both make most of my money from time series data and use
deep learning and work with data with no labels. Here's a
recent presentation I did on some of this work and a
companion presentation I encourage people to read on how to
use this effectively in production.While you are right that
some feature engineering is needed, there's no reason DL
can't be a part of your
-design-pa...For more of the basics, my book on deep
learning might help as well (minimal math vs the standard
384028345 - 3 hours ago
Thanks for the info! The book looks interesting.Do you
have an opinion on the fast.ai and deeplearning.ai
courses? I finally have some time to work through these
and since the deeplearning.ai series starts on December
18th, I'm wondering which one to dive into since I can't
tell from the outside how they compare.
agibsonccc - 2 hours ago
I would take both. deeplearning.ai focuses more on math
fundamentals, fast.ai takes a more coding oriented
approach. It also has 2 classes: a beginner and advanced
one. I personally prefer the fast.ai approach.
QuasiAlon - 4 hours ago
> Fortunately, there are new automated tools today that can do
that automatically.can you please elaborate?
graycat - 2 hours ago
One very old tool for such things was called "stepwise
regression". IIRC J. Tukey was partially involved in that. It
appears that the AI/ML work is close to the regression and
curve fitting going back strongly to the early days of
computers in the 1960s and a lot in the social sciences back to
the 1940s and even about 1900.A lot is known. E.g., there's
the now classic Draper and Smith, Applied Regression Analysis.
Software IBM Scientific Subroutine Package (SSP), SPSS
(Statistical Package for the Social Sciences), SAS (Statistical
Analysis System), etc. does the arithmetic for texts such as
Draper and Smith. For some decades some of the best users of
such applied math were the empirical macro economic model
builders. E.g., once at a hearing in Congress I heard a guy,
IIRC, Adams talking about that.Lesson: If are going to do
curve fitting for model building, then a lot is known. Maybe
what is new is working with millions of independent variables
and trillions of bytes of data. But it stands to reason that
there will also be problems with 1, 2, 1 dozen, 2 dozen
variables and some thousands or millions of bytes of data, and
people have been doing a lot of work like that for over half a
century. Sometimes they did good work. If want to do model
building on that more modest and common scale, my guess is that
should look mostly at the old very well done work. Here is just
a really short sampling of some of that old work:Stephen E.
Fienberg, The Analysis of Cross-Classified Data, ISBN
0-262-06063-9, MIT Press, Cambridge, Massachusetts, 1979.Yvonne
M. M. Bishop, Stephen E. Fienberg, Paul W. Holland, Discrete
Multivariate Analysis: Theory and Practice, ISBN 0-262-52040-0,
MIT Press, Cambridge, Massachusetts, 1979.Shelby J. Haberman,
Analysis of Qualitative Data, Volume 1, Introductory Topics,
ISBN 0-12-312501-4, Academic-Press, 1978.Shelby J. Haberman,
Analysis of Qualitative Data, Volume 2, New Developments, ISBN
0-12-312502-2, Academic-Press, 1979.Henry Scheffe, Analysis of
Variance, John Wiley and Sons, New York, 1967.C. Radhakrishna
Rao, Linear Statistical Inference and Its Applications: Second
Edition, ISBN 0-471-70823-2, John Wiley and Sons, New York,
1967.N. R. Draper and H. Smith, Applied Regression Analysis,
John Wiley and Sons, New York, 1968.Leo Breiman, Jerome H.
Friedman, Richard A. Olshen, Charles J. Stone, Classification
and Regression Trees, ISBN 0-534-98054-6, Wadsworth &
Brooks/Cole, Pacific Grove, California, 1984.There is a lesson
about curve fitting: There was the ancient Greek Ptolemy who
took data on the motions of the planets and fitted circles and
circles inside circles, etc. and supposedly, except for some
use of Kelly's Variable Constant and Finkel's Fudge Factor, got
good fits. The problem, his circles had next to nothing to do
with planetary motion; instead, that's based on ellipses and
that was from more observations, Kepler, and Newton. Lesson:
Empirical curve fitting is not the only approach.Actually the
more mathematical statistics texts, e.g, the ones with
theorems and proofs, say, "We KNOW that our system is linear
and has just these variables and we KNOW about the statistical
properties of our data, e.g., Gaussian errors, independent and
identically distributed, and ALL we want to do is just get some
good estimates of the coefficients with confidence intervals
and t-tests and confidence intervals on predicted values.
Then, can go through all that statistics and see how to do
that. But notice the assumptions at the beginning: We KNOW
the system is linear, etc. and are ONLY trying to estimate the
coefficients that we KNOW exist. That's long been a bit
distant from practice and is apparently still farther from
current ML practice.Okay, ML for image processing. Okay. I am
unsure about how much image processing there is to do where
there is enough good data for the ML techniques to do
well.Generally there is much, much more to what can be done
with applied math, applied probability, and statistics than
curve fitting. My view is that the real opportunities are in
this much larger area and not in the recent comparatively small
area of ML.E.g., my startup has some original work in applied
probability. Some of that work does some things some people in
statistics said could not be done. No, it's doable: But it's
not in the books. What is in the books is asking too much from
my data. So, the books are trying for too much, and with my
data that's impossible. But I'm asking for less than is in the
books, and that is possible and from my data. I can't go into
details in public, but my lesson is this:There a lot in applied
math and applications that is really powerful and not currently
popular, canned, etc.
wenc - 59 minutes ago
Stepwise regression is no longer recommended because it's
very easy to fool oneself.http://www.sascommunity.org/mwiki/i
amigoingtodie - 2 hours ago
Thank you for the list of resources.Are you able to go into
more detail about your startup (problems it is solving)?
nl - 8 hours ago
This tool is pretty interesting.I've been playing around with a
similar idea of text. Do you already do that?
Fiahil - 4 hours ago
Don't you think people are, sometimes, just applying ML to their
problem "because of hype" ?One example I have in mind, was a
contest where participants were given a series of satellite
pictures and asked to write a classifier to detect icebergs and
cargo ships (the two are quite similar). As someone else pointed
out, trying to use classical computer vision and machine learning
on these images will always have some error rate during
identification. However, if we were able to extract speed and
trajectory of all objects in the picture and mixing them with AIS
data, finding which ones are ships, which ones are giant pieces
of ice, and which one are non-moving structures to be avoided,
becomes easy.So, you have to choose between a black box that will
give you potential results with a given error-rate, and a
predictable algorithm that anyone can audit. Seems like a no-
brainer situation to me. For what other reason would you choose
the first solution, except hype-related decisions ?
radarsat1 - 1 hours ago
Your comparison seems like a false dichotomy, and I think you
are agreeing with OP. OP says, spend less time worrying about
the algorithm and more time worrying about what data you are
feeding the algorithm. You are saying, what if you had to
choose between dataset A with algorithm A and dataset B with
algorithm B.You claim, (probably correctly) that dataset B,
which includes velocity and trajectory, is more correct for the
problem at hand, and given dataset B, I would suggest that
either algorithm A or B would probably do just fine.You also
claim that algorithm A has "some error rate during
identification." But so will algorithm B, and so will either
algorithm on dataset A and B!The question you should ask is,
how much do I care about "black box" vs. "white box", and is
there are trade-off? If the black-box solution (algorithm A,
the "ML" solution) gives you 10% higher accuracy, and that
accuracy is going to save lives, you bet I'd choose it. Or
maybe I decide that interpretability is really important due to
external audit reasons, so I need the white-box solution. But
maybe I'd choose both, the interpretable one, and use the
uninterpretable one as a flag for "a human should look at
this." Or maybe I'd combine the results of both algorithms to
get even higher accuracy.There are just so many ways to
configure a solution to the problem you propose, and you are
only distinguishing between only two of them.
nerdponx - 2 hours ago
You wouldn't, and any data scientist worth their salt would
recommend that the business choose the latter option.
cm2187 - 2 hours ago
The thing is if you know exactly what you are looking for, like
in your example, or a QR code, or a barcode, it makes sense to
tailor an algorithm. But you may not want to have to maintain a
complex algorithm every time a small change happens (say new
kind of ships appear). Or you might want a generic approach
(recognise any objects, including objects that did not even
exist at the time the code is written, but will appear in the
data). In such case I can see ML being a good choice.
bllguo - 10 hours ago
For sure, usually the algorithms aren't the interesting part, but
rather how you frame the problem and most importantly what data
you have.I wish I could say I was passionate about feature
engineering. I enjoy where deep learning is heading right now -
where that kind of finicky, more-art-than-science approach
becomes unnecessary, and the model does a better job detecting
features than humans.
jmmcd - 5 hours ago
> I especially like the early slides that help frame AI vs ML vs
DL so that people can have a realistic understanding of what
these technologies are for.But they're wrong! I read "Deep
learning drives machine learning which drives artificial
intelligence." This is very wrong. I stopped reading.
acdanger - 1 hours ago
How is it wrong? What's the correct hierarchy ?
60654 - 45 minutes ago
AI is the overall field and the superset of all of the
various approaches.ML is one family of approaches for
knowledge acquisition in AI, but far from the only one (eg.
logic based inference is another big one).DL is a family of
approaches in supervised ML. As the author points out, it's a
subset of a subset.But saying that this sub-subset "drives"
AI is like saying endocrinology "drives" medicine: not the
right mental model at all.
kovek - 1 hours ago
My understanding is that Deep Learning is a type of Machine
Learning. Artificial Intelligence is the idea that a machine
performs similarly or better than a human for a specific
task. Artificial General Intelligence is when a machine
performs similarly to a human in many different kinds of
dmurthy - 7 hours ago
Really thanks for this. I've recently dived into ML & DL and have
slowly but surely realized the importance of feature
engineering(FE). Though I've taken a few MOOCs, I haven't found
one that truly focuses on FE and still looking.
rubenfiszel - 13 hours ago
If I understand correctly those are slides from a Googler (Not sure
if those slides have corporate approval), that probably have as a
side goal to showcase that Google is a fun place to do ML.Not that
I am judging or anything but, the author's personal website
http://www.jasonmayes.com/ whose link is displayed multiple times
is a giant ad to get hired elsewhere and show at least some desire
for other career opportunities. Not sure if that reflects greatly
on the company.
npgatech - 8 hours ago
Checking his website, it reeks of narcissism. There are better
ways to assert yourself than to do all the corny things he has
done on his self promotion website.
jasonkester - 8 hours ago
Are you honestly slagging a guy off for talking about himself
on his resume???I mean yeah, we computer folk are supposed to
be all self deprecating and all. But if there is one place we
should stop mumbling and talking ourselves down for a second,
that is it.At some point if you want people to know what you
do, you're going to have to tell them.
cormacrelf - 5 hours ago
I sorta draw the line at autoplaying music. Apart from that,
he's done a good job. How many of us are bold enough to put
long list of glowing reviews on our resume?
thenomad - 2 hours ago
This is a deeply unfair, unreasonable and arguably abusive
comment.It's entirely reasonable to talk about yourself and
your achievements on your resume, and Mr Mayes' site is rather
a good example of doing so.
zo7 - 13 hours ago
Given the number of plugs for Google products/projects/research
(especially near the end) it's probably intended to be more of an
ad for Google.
auntienomen - 11 hours ago
Maybe. OTOH, this guy's resume says he's a web programmer.
I'd think if google were recruiting people interested in
machine learning, they'd get one of their machine learning
specialists to write this.
carlmr - 6 hours ago
Specialists are usually the worst teachers, because they
assume that you know trivial things. What appears trivial to
them is not trivial to the audience
mmanfrin - 9 hours ago
Good slides, got me back in to the fever of wanting to learn;
although a lot of the credit goes to the linked 3Blue1Brown videos
(whose Linear Calculus series is excellent) which were a lot more
technical but no less approachable.Question to those versed in ML:
I want to work on an AI that plays a video game (aspirations of
playing something like Rocket League, but I know I need to start
smaller with something like an old NES game). I understand these
are usually done with Recurrent Neural Networks, but I'm a little
lost as to how to get data in to the RNN -- will I need to make
another AI or CNN to read the screen and interpret (including the
score?) My 30k ft view is that if I can define a 'score', give it a
'reset' button, and define 'inputs (decision targets)', then I just
need to give it the screen and let it do its thing. But getting the
'score' is the part I can't figure out short of adding another
layer to the classifier.
allenguo - 8 hours ago
You should check out Berkeley's deep reinforcement learning
course. There's lecture videos, slides, and homework
assignments, and it's all very up-to-date.
catnaroek - 13 hours ago
The slide titled ?A note on dimensionality? reminded me of this
xkcd: https://www.xkcd.com/547/?That would be (very) bad.?
TheNewLab - 3 hours ago
Nice introduction, but I really don't see how "2 years of
headbanging, so you don't have to" applies.
pimlottc - 2 hours ago
I think the author meant "banging my head against the wall" while
getting it all working but didn't realize the term has a very
levesque - 1 hours ago
There ain't no quick way to get a good grasp on ML. You just need
to spent the time needed to get there, reading and working on
simple problems, carefully validating that you understand
concepts as you go. It's like asking for a way to learn
mathematics or computer programming in an hour. Hint: there