HN Gopher Feed (2017-06-30) - page 1 of 10 ___________________________________________________________________
I ask 100 information questions to four digital assistants
206 points by forgot-my-pw
https://vlad.d2dx.com/the-great-assistant-skills-comparison-goog...___________________________________________________________________
nmstoker - 5 hours ago
I'm surprised by a number of the failures, such that I did wonder
if the failure might be happening on the speech recognition side
rather than the response generation side.Both Google Home and Alexa
have very little problem recognising my speech whilst friends often
struggle even when they seem to say the precise same phrase, to the
point it's mildly entertaining. With Google I suspect they've
tailored to my voice (I've used voice commands extensively for
several years) but I've only had a Dot briefly and it worked well
from the start. Another surprise is that they cope well with my
perculiarly English English phrasing and pronunciation, but I'm
sure there are lots of less widely spoken dialects that would throw
them.
ghaff - 3 hours ago
Alexa is the first thing I've owned that really does quite a
solid job in the voice recognition department (whatever its
failings to return something useful based on that recognition).
Siri on my phone is rather hit or miss by contrast.I suspect that
the microphone array has a lot to do with it. Anecdotally, I've
read pieces by people saying that homebrew "Echos" together with
the Alexa APIs aren't as good as an actual Alexa.
51Cards - 6 hours ago
I notice some different answers on my devices. For example on the
"Are tomatoes vegetables?" question my Google Home states that they
are definitely a fruit. (quoting Oxford Dictionary)Edit: And
"What's the height in meters of the Empire State Building?" gets me
"381 meters, 443 meters to tip"
IanCal - 6 hours ago
I tried the conversational weather one on google assistant on my
phone."Should I take an umbrella tomorrow?""No..." and shows me
tomorrows forecast."What about the day after?""No..." and shows a
forecast that when I look more closely at I notice is for
today.Neither of these also spotted that I'm heading to another
city tomorrow, which is in my calendar. If I changed it to "do I
need to take an umbrella for my trip tomorrow" it just searches
google and gives me a search result suggesting I take a small
folding umbrella... for a trip to Thailand.
visarga - 6 hours ago
It might be so, but remember that experts were predicting
computer will beat top humans in Go in 10 years. Maybe next month
there will be a breakthrough with NLP. The amount of research and
compute going into this problem is amazing, and we don't know
what's around the corner.Also, it might be possible to create a
much better assistant today, but it would be too expensive to
offer to the public for free. What if it requires 100 TPUs to
run?
opportune - 3 hours ago
This isn't an NLP problem, it's a coder problem. These
solutions already exist, Google/ other assistant providers just
need to dedicate the man hours to make it happen.
IanCal - 5 hours ago
The thing for me in that example is it's not a particularly
complex set of options. It knows I'm asking for the weather, it
knows that it should give me the weather in a particular
location, and google knows where I'll be tomorrow.The other
main mistake was (despite it getting the conversational part
right) thinking "the day after" means today.I don't think these
things require TPUs or new NLP.
marcosdumay - 5 hours ago
This one problem of NLP is kinda solved? for more than 10 years
already.What you are seeing here is a much simpler CS problem,
but much harder social problem. It is "why can't my
applications talk to each other?". I really doubt it will be
solved in 10 years.1 - Humans don't have a perfect solution for
it, and machines are still worse, but not that worse.
jetpacktuxedo - 5 hours ago
I use google assistant for controlling smart lights sometimes
with "ok google, turn on/off the lights". At one point I tried
"Ok google, turn off the lights in ten minutes" and it just
searched it. That seems super simple and like something it
shouldn't have had any trouble with, but here we are :/
codekilla - 6 hours ago
more people are starting to appreciate that AGIish stuff is
actually really, really hard.
snarf21 - 6 hours ago
Agreed, determining human intent is really, really, really hard.
There are some many context clues we use in everyday life that
the voice interface will never have access to like where am I
standing, what's my expression, etc.
forgot-my-pw - 4 hours ago
Makes me wonder if Watson will do a better contextual search and
single result answer. IBM should release a voice assistant too.
ghaff - 5 hours ago
The ability to function as a virtual assistant, even at the level
of a not-so-sharp intern [1], would be a killer app.Give it some
parameters for a trip you're taking. It comes back with some
options and follow-up questions. We are a long way from that
point. Even that not-so-sharp intern has a huge amount of
internalized knowledge about general preferences, cities,
airports, etc. and probably knows questions to ask to narrow
things down.I strongly suspect there are other domains where a
lot of people are assuming we're 90% there and we're not.[1] Not
to insult interns or any other group. I just mean you don't need
to be at experienced executive assistant level to be really
useful.
dahart - 6 hours ago
What's interesting to me reading this is the expectation that voice
search should do things that text search currently doesn't do. If I
ask Google about Bill Murray running for president, I presumably am
looking for articles that mention the words "Bill", "Murray", and
"President". I would expect to return articles that best match the
query, and I would never expect Google to be able to tell me the
difference between real and fake articles. For a server to answer
the author's question it has to understand the question so well it
can change the question into "What is the list of people that have
run for U.S. president?" and then check if Bill Murray is in that
list. That's a tall order.We are moving past expecting the most
word matches into expecting the server to understand what we really
want. Voice search is more like the "I'm feeling lucky" button,
because it takes longer; you only have time for one answer and the
first answer has to be right. It comes without the expectation that
you're lucky if the answer happens to be right, now we need the
first result to be the rightest result there is.So the glass is
half empty. I personally prefer to see it half-full, but it's also
true and the critique is more valuable and interesting than
optimism.
freehunter - 5 hours ago
I do expect voice to do something text doesn't do, because text
allows for interactions that voice doesn't. I can't skim a page
of search results via voice. I'm not going to sit there and have
Alexa read 10 page titles and URLs to me. I ask a question, I
expect a concise voice response. Anything else is a complete
failure of the UI.Currently there is one way for a computer to
interact with humans via voice: direct and unquestioning answers.
hk__2 - 5 hours ago
> I can't skim a page of search results via voice. I'm not
going to sit there and have Alexa read 10 page titles and URLs
to me.I think you can. Alexa can read you the titles and you
can ask it more information about a specific title. It?s like
asking the waiter which desserts they have then interrupting
him because you don?t know what a pannacotta is.
[deleted]
freehunter - 5 hours ago
You can, but I don't want to. It takes longer than reading,
makes me sit and actually listen, and it just feels awkward
to me.
acdha - 3 hours ago
That's certainly true for most people sitting in front of a
computer but I really liked the inclusive design guidelines
from Microsoft[1] reminding us that there are many people
for whom any particular assumption is untrue, often only
temporarily or in a specific
situation:https://www.microsoft.com/en-
us/design/inclusiveAs an example, a coworker mentioned that
his use of Alexa went from casual to heavy when they had a
child and the ability to do things while carrying a baby
suddenly became really important. I suspect there are more
situations like that than we might think at first.1. I
know, 90s me is still getting used to saying that too
ghaff - 3 hours ago
And I know a couple with a relatively young child and
they love their Echo. All the tell a joke and other
things along those lines that are kinda dumb to me. Or
questions that I'd just as soon type on my phone or a
computer but which are more natural to just speak in a
family conversational setting.
ghaff - 4 hours ago
I think a good model to imagine is that you don't have any
computers on you and you're talking to an assistant over
the phone who has access to Google and other online
resources. The types of responses you'd expect from that at
least modestly intelligent assistant are probably not all
that different from what you'd like to hear from a digital
assistant.If I were to ask a question that had a long list
of potential responses, I'd expect them to ask me to
clarify or narrow down what I'm looking for or at least
explicitly ask me if I really wanted them to read the whole
list.
bluGill - 1 hours ago
I'd expect my assistant to know a fair amount about me
and the current context. Using those clues a human can
pick out what I really care about, at least in most
cases. Even when there is a list of responses I'd expect
an assistant to give a better summary when asking for
clarification.
ghaff - 37 minutes ago
Certainly learning my preferences is an important
component of a personal assistant. e.g. I almost always
go to the airport using a particular service.That said,
for those of us who don't have personal admins, there's a
lot of opportunity for digital services that fall between
purely self-service travel booking as it exists today and
and having an assistant.
skywhopper - 23 minutes ago
Voice search will not be actually useful until it can respond
naturally to naturally voiced questions. The whole point of voice
UI is to make things more natural for human interaction. If it's
just a matter of how well computers can interpret speech, well,
that's a fun parlor game, but it's not what is implicitly
promised by a voice UI, and it's definitely not what's explicitly
promised by the marketing for these services. And ultimately
until these things can interact naturally, they are doomed to
being a novelty.
amelius - 5 hours ago
But text searches are expected to return multiple results,
whereas a voice search is expected to return a single result.
seiferteric - 6 hours ago
Because it was sold as something better. When siri first came out
it was billed as something revolutionary and that it would
continuously improve as more people used it. Did that even
happen? Outside of the "happy path" sort of questions, I find it
rather disappointing with pre-canned responses, or just showing
me search results most of the time. Now I really only use it for
setting reminders or alarms.
dahart - 6 hours ago
Totally. Me too, my use of Google Voice and Siri and Alexa has
declined because I don't usually get what I want the first
time. Reminders and alarms it always gets right, it's faster to
set a reminder by voice than by typing. But I think you and I
are illustrating how we expect more from voice search than text
search. My use of text search hasn't declined like my use of
voice search, and text search is just as fundamentally bad as
voice search. I think it's because I can see & sift many
results, and because I can easily iterate on my query when it's
not quite right. Voice search can't do either easily.Perhaps
Amazon, Apple, Google and Microsoft all initially thought that
the revolutionary part was being able to speak a query and have
the query match what you said, and that the search part was
already good enough.
ncallaway - 2 hours ago
> But I think you and I are illustrating how we expect more
from voice search than text searchI think @seiferteric's
point was that the expectation may not be there because it is
a voice search. That expectation is there because that's how
it was marketed.If the marketing for these things was: "Ask a
question, and get search results by voice" I don't think I'd
have the expectation that it find and deliver the correct
answer to me.But the marketing for all of these devices is:
"It's a personal assistant! Ask it a question and you'll get
an answer!"I'm personally not convinced that the high
expectations are because it's a voice interaction, but rather
that the technology simply can't live up to the marketing
pitch.
bluGill - 1 hours ago
I think part of it is desire. Many people would love to
have an assistant like that. There is a vague memory of the
days of personal secretaries, a girl (those were sexist
days) who could looks things up for you so that you can
spend your efforts are other tasks. There are a lot of
times when everybody could use help, but they don't have
it.
brians - 5 hours ago
Try a reminder including the word "play". The choice to play
an album or open an app dominates, so it tells you it doesn't
have an app with some nonsense name.List decoding was
invented by 1955. This is a set of hard problems, but very
well studied ones.
comex - 3 hours ago
Which assistant? It works for me with Siri:
http://imgur.com/a/GiuMP
ghaff - 5 hours ago
>I think it's because I can see & sift many results, and
because I can easily iterate on my queryAbsolutely. In fact
sometimes I'm searching for something, whether in a search
engine, at an ecommerce site, or whatever and I'm not getting
what I want immediately. If I'm on a phone or tablet, I'll
often grab a nearby laptop because it's just faster and
easier to do a lot of typing and clicking on. (Less true with
more recent tablets but my basic point is that there's
sometimes a lot of fast iteration when I'm trying to find the
answer to something non-obvious.)
acdha - 6 hours ago
It seems like there's something akin to the uncanny valley effect
going on here where a voice UI invites people to think about the
other end of the conversation as a person and then be
disappointed when they hit the edges of what it's designed to
do.There's a really interesting discussion to be had about how UI
decisions can make that process smoother ? I really liked
https://bigmedium.com/speaking/design-in-the-era-of-the-algo...
as a call for how you can make the failure modes of the system
more graceful. I think a lot of the success in the next decade or
so is going to come from the places which figure out good answers
for not making a system which seems to promise more than it can
deliver.
dahart - 5 hours ago
Yes! Voice is sort-of "tactile" if you will. The process of
speaking instead of typing may well cause us to expect a more
human interaction. I bet they're already studying this effect
and changing search results for voice searches accordingly, but
it will be fun to watch how it unfolds.Very nice article, I
only skimmed so far, but I think I agree with all of it. It has
a definitely pro-consumer bent that I wish would come true, but
seems like trends are in the other direction. I suspect there's
too much money in search and improving query understanding at
scale for companies that get there to be as transparent and
open and sharing as this author is asking.
makmanalp - 1 hours ago
> a voice UI invites people to think about the other end of the
conversation as a personIt's not the voice UI that does this,
it's the marketing.If they sold it as "speak your google search
terms", it'd work a lot better. It'd also be a lot less sexy,
but that's still mostly what it is IMHO. Not to say that it
isn't impressive stuff, it is! But it's highly oversold, still.
redler - 3 hours ago
The fact that we interact by voice leads toward a sort of
inadvertent theory-of-mind about the other party, which makes
the pulling away of the curtain with so many of the answers
much more jarring. Voice interaction seems to recruit a much
deeper evolutionary expectation than the much more recent
phenomenon of typing and reading.
acdha - 2 hours ago
Yes! I wonder whether that's an argument in favor of things
like deliberately using quasi-robotic styles to help people
recognize the limitations faster. It'll be interesting to see
what product designers come up with and how the market adapts
to this.
7952 - 6 hours ago
You see this issue with Google Map searches. Over time it seems
to have relied more and more on structured data and less on
algorithmic results. But it still returns bad results when the
software obviously lacks the data. Better to just say "nothing
found" sometimes.
scrooched_moose - 6 hours ago
I find it interesting that Google couldn't get the Bill Murray
one. No matter what combination of "did bill murray run for
president" I search for, I get a rich snippet on the results page
from snopes.com which says"Claim: Comedian Bill Murray is running
for president and proclaimed religion to be "the worst enemy of
mankind." Claimed by: Internet Fact check by Snopes.com: FALSE"It
would seem they are reasonably close but this is more of a
product integration failure than a recognition failure.
dahart - 6 hours ago
Great point! I get the same from Google. Looking back in the
article, he only criticized Siri and Cortana on this question.
He claimed none of them got it right, but didn't say
specifically what Google did with it, and it's entirely
possible it was a different answer before now.Bigger picture
though, Bill Murray is famous making it easier to answer
questions like this. In general, does the wording of the
author's question truly imply he's searching for a fact check,
and do you expect Google to know that even if there are no
articles that match the wording of the question? The snopes
articles does contain the terms "did", and "Bill Murray", and
"run for president", so we don't have any evidence that Google
understands the question, we just have some content that
matches the query.The issue I see is that the computational
question of search has long been trying to measure relevance by
matching the query against the corpus. This Bill Murray
question is an example of how that can break down. I might
actually want the fake articles... and I might not. There's no
way for the search engine to know without making an inference,
and the expectation that mass market search engines make
inferences seems pretty new to me - and I don't expect that
when I do text searching. I guess I just expect voice search to
push the need for question understanding and inference making
even faster than text search has.
ancalimon - 6 hours ago
Heya,My first Hacker News inclusion. I feel like there should
be some rite of passage. Well, other than the sudden and
unanticipated login attempts.My testing device for Google was
the Google Home speaker, which appears to have a different
tolerance for reading search results. I've had it rattle off
several sentences from web pages for other keywords in the
list (see, for example, the boiling point of water), but for
the Bill Murray question there seems to be some kind of
limiter. I just re-checked, using the exact phrasing I had
before, and it still says that it doesn't know, but it's
learning all the time.I'm guessing there is some kind of a
relevance check for the speaker version compared to the phone
version. The phone is probably happier to return any result
(a la Siri), whereas the speaker appears to be making some
attempt to understand what I'm asking for before reading
search results.This particular question appears to trigger
the speaker not to read the search results. We can only
speculate as to why: does it not find it relevant enough? Is
there a reserved path on "Did xyz" questions when sent to
Google Home? Am I unknowingly in the A/B testing group that
doesn't get the answer? There's few ways of knowing black-box
without massive data testing, but it is curious.
dahart - 5 hours ago
Welcome! I don't know of any rite of passage, but maybe I
gave you your first upvote? ;)> I'm guessing there is some
kind of a relevance check for the speaker version compared
to the phone version.I would bet on that & expect it too...
I'm sure all these voice search products are experimenting
with how voice search needs to be tuned differently than
text search.
ewanm89 - 6 hours ago
What does "Google Home" speaker usually do when there is no
clear result? On the phone google assistant just displays
google search results in such a case, obviously that can't
work on the "Google Home" speaker.
ancalimon - 5 hours ago
Sometimes it'll read a page, e.g. it read a passage from
Wikipedia a few times. But if it really can't decide, it'll
say something like "I'm sorry, I don't know that one".
[deleted]
kurthr - 39 minutes ago
Since the voice interface can only really give one Answer, it
needs to be more certain that the Answer is a good or common
Answer rather than the best Answer. Variation in quality needs to
be reduced rather than just optimization of PageRank.It's a bit
like pressing I'm Feeling Lucky for your result. I'd hope that it
was more optimized for always good results rather than often
great, but occasionally lousy.
Swizec - 5 hours ago
That's the thing. Google can do a lot of those crazy things with
their little popup boxes. I've seen cases where I ask a really
weird question and it summarizes an entire stackoverflow thread
into a neat paragraph that answers my question. Clickthrough and
the exact answer that Google showed me isn't anywhere on the
page.Also the query "did Bill Murray run for president" returns a
Snopes article debunking the myth as the first result. This
should totally be something a Siri thing could parse and tell you
about.https://www.google.com/search?q=did+bill+murray+run+for+pre
s...
batbomb - 4 hours ago
What you are talking about knowledge base construction. Google
does do some of that, in fact. Apple wants to get better, they
just bought lattice.io
rojobuffalo - 5 hours ago
I keep coming back to the idea that progress towards AGI might be
made by someone working on a "coordinator" agent. We might have
several narrowly focused agents with deep knowledge in particular
domains: a mathematician, a fact-checker, a botanist, a structural
engineer, etc.; then have an agent that broadly understands how to
route requests to the right vertical. Maybe that's already
descriptive of the underlying architecture for some of these
agents. The alternative might be that we interface with several
different conversational agents, and like interfacing with people,
we use our judgement to decide which specialist to ask.
Bjartr - 1 hours ago
That's kind of what Watson did, but that level of architecture
hasn't made it into personal assistants yet
csomar - 6 hours ago
It's interesting that while many fails, there is still one that
wins. That is if you combine the efforts of these 4 digital
assistants, you'll get a much smarter one. Do they have an API? Can
you query siri, cortana, etc..?
giobox - 6 hours ago
There are APIs for both Amazon and Google's voice assistant
services. Not surprisingly Siri doesn't expose a public one, I've
no idea about Cortana. I've messed around a little with them on
the Raspberry Pi.This idea, while simple in principle, might be
kinda annoying in practice. You're still left with similar issues
- how do you decide which talking cylinder service answered the
question best? Do you play all of the answers? For me I'm fairly
sure listening to all of them in a row would frustrate me even
further - just waiting for Alexa to finish telling me the news
headlines is sometimes kinda annoying, especially when that
information in visual form can be grokked almost instantly. Many
of these devices, especially the Google one, are getting better
at context based followup questions - managing who to send your
follow up question to could be kinda crappy as well. I suppose
you could do one device that could ask each service individually
("Alexa...", "Ok Google..."), but in my experience as soon as I
get one bad answer, I inevitably just use google.com to find what
I need rather than risk wasting my time on another failed
conversation.The main part that I've found hard to do in home
rolled voice assistants is microphone arrays. Almost all these
devices use pretty sophisticated microphone technologies for
things like noise cancelling, subject isolation etc, which so far
has been non-trivial to do to a similar standard in homemade
versions of them. It also certainly used to be the case that
creating your own "hotword" system to call the Alexa API was
technically against the ToS (it allowed you to use a button press
to call Alexa instead), as naturally Amazon would rather you buy
a real Echo. No idea if this is still the case, and at any rate
Amazon can't really enforce this either, but worth mentioning.
forgot-my-pw - 3 hours ago
A funny popular experiment is the seebotschat Twitch account
who livestreamed 2 google homes running Cleverbot API talking
to each others.Here's a short highlight:
https://www.youtube.com/watch?v=WoI6_z2mfdY Some
implementation details in AMA: https://redd.it/5nz3eb
majani - 5 hours ago
I'm curious about how accurate the assistants were at listening.
That used to be the most pressing issue with voice commands that
had relegated the technology to a running joke. It appears that's
understandably what the companies have been focusing on so far, but
there's still work to be done to get to 100% accuracy of listening,
especially when you take into account exotic names and
interchanging between languages and slang.
kelchm - 5 hours ago
One of the most 'magical' experiences I've ever had using Google
Assistant was the following exchange:Me: "Okay Google, What's the
latest album by Death Cab For Cutie?" Google: "The latest ablum by
Death Cab For Cutie is Kintsugi"Me: "Okay Google, Play the album
Kintsugi on Spotify" Google: "Okay, asking to play Kintsugi. [album
starts playing]"
sib - 4 hours ago
Wouldn't it have been a lot more magical if it had simply asked,
"Would you like me to play it?" (knowing that you have a Spotify
subscription) at the end of answering your question?Or, at least,
for you to be able to say "Play it!" rather than the unnatural
"Okay Google, play the album Kintsugi on Spotify"...
forgot-my-pw - 4 hours ago
The context searching might work with: "Play it". I haven't
tried myself.
notadoc - 6 hours ago
I find Google Assistant to be very good at answering most
questions.Also, I can accurately get the weather from Siri most of
the time.
davidw - 6 hours ago
Google's thing can't even figure out my wife's Italian name most of
the time. It's quite frustrating.
octalmage - 6 hours ago
My girlfriend has the name Taryn and Siri really struggles with
it, usually correcting it to Karen or Terell (both names in my
address book). Alexa does better but probably because it doesn't
know about the other names.
evilduck - 4 hours ago
Not that it excuses Siri's shortcomings or helps if you're
referring to her by name mid-sentence texting to someone else,
but you can give assign nicknames in your Contacts app. It
might make it less frustrating to dictate texts or start calls.
So instead of saying "Call Taryn" and getting it misheard you
could say "Call my girlfriend".
boznz - 3 hours ago
English is a terrible language for this, unfortunately it's the
only one I speak.Not sure how other languages cope, I suspect the
simpler ones cope much better. We almost need a spoken equivalent
of SQL
thaumasiotes - 2 hours ago
English is best known for being simpler than average, not more
complex. It's one of the flagships (along with Latin / Mandarin
Chinese / Swahili) for the theory "languages which are widely
learned by adults become simplified over time".
paradite - 6 hours ago
Ensemble for the win, I suspect that they can perform better than
the combined individual best when sharing training resources and
models.
GCA10 - 6 hours ago
As much fun as this test is, it dodges the most interesting
question of all: "Are these machines supposed to be talking search
engines?"I'm increasingly believing that the answer is: "No." These
machines (especially Alexa) are rapidly gaining popularity while
still providing pretty ragged answers to search queries. So we
should start asking: "Are they taking on a different function that
didn't match our early expectations?"In a word, yeah. Alexa is a
really nifty jukebox for those of us that don't have the good sense
to create formal playlists. It's a handy kitchen timer, especially
if you've got multiple pots doing different things. It's a better
alarm clock and a better purveyor of soothing bedtime sounds. (If
you're asking: Good god, how many people really want or need that,
think: Fussing infants.)Smartphones already provide pretty
excellent search results on the fly. I'm not sure voice-powered
assistants will re-solve that problem with great success. But there
are a surprising number of rudimentary needs around the house for
which a voice-enabled device becomes quite handy.
bkohlmann - 5 hours ago
This is a really interesting point - particularly the kitchen
timer thing. Right now, I'm the kitchen timer for my wife.
She'll say, "set a timer for 8 minutes." I'll interrupt what I'm
doing to comply. I may buy Alexa just for that...
amelius - 5 hours ago
Yeah, but why does it need to phone home to Alexa servers all
the time, just for setting a timer?
ghaff - 4 hours ago
Because it doesn't have the local intelligence to understand
your unique waveforms that are saying something along the
lines of "set a timer for five minutes." Now I'm sure someone
could design a specialized device that could act as a voice
activated timer--I suspect such exists--but Alexa is a lot
more general purpose.
wcummings - 3 hours ago
>If you're asking: Good god, how many people really want or need
that, think: Fussing infants.TIL I am a fussing infant. I love
that "sounds of the rain forest" bs (though I don't use an echo).
GCA10 - 2 hours ago
We're all fussing infants, to be truthful about it.
Splines - 5 hours ago
Interesting - maybe it means that voice interfaces are better for
tasks of a certain shape: Those that are typically multi-step,
specific, and "deep" in an app. Things like setting a kitchen
timer, saving a reminder for yourself, replying to a text, or
setting a travel destination.Tasks that require a high amount of
breadth, like search, don't scale well to a voice interface.
ghaff - 6 hours ago
I expect there are a lot of questions that lend themselves to
concise answers. BUT if sensible informally phrased questions
don't get answered properly a decent percentage of the time, we
learn not to bother.I agree that voice interfaces aren't good for
a lot of things. How do I cook XYZ? probably isn't suited. But
overall performance just isn't that great.
bluGill - 5 hours ago
> How do I cook XYZ? probably isn't suitedThat is perfectly
suited to voice if voice worked. When I call my mom for the
recipe for cake it would be a whole lot easier if my mom would
say "beat the eggs for 1 minute", listen for the beater to
start and then say stop after one minute. My mom has better
things to do with her time than walk me through the recipe, but
an assistant should be able to do this.Of course I have just
transformed the problem into something that technology isn't
able to do. However the problem isn't with the voice interface
it is our AI isn't yet up to all that. (poor AI, every time
they do something useful we rename it and move the goal posts)
ghaff - 5 hours ago
Fair enough. I was thinking of it as a one time answer. But
you're absolutely right that a good interface could maybe
show you a recipe on a screen somewhere and then walk you
through the process step by step.
stephengillie - 6 hours ago
These digital assistants are just begging for an app store.
Search is just the first app, jokes and weather are other useful
apps. These could easily follow a similar product life cycle
pattern as smartphones.
rrdharan - 6 hours ago
They have app stores: https://www.amazon.com/b?node=13727921011
coryfklein - 1 hours ago
* One has an app store
ghaff - 6 hours ago
Well, that's kinda what skills are in the case of Alexa. Part
of the issue though is discoverability. I forget what I've
installed or I forget what the right wizard's incantation is to
access some skill/app.
bootloop - 5 hours ago
Actions on Google. Played around with the developer tools a bit
and it looked promising to me.
LesZedCB - 4 hours ago
i wonder if there would be any use in services that don't respond
in real-time.I think these digital assistants are nerfed by the
real-time response requirement. I'd be happy to ask some of those
questions, and get a pop up in a few minutes. And they could be of
much higher quality as they can be processed and better researched.
colinbartlett - 5 hours ago
Forget about questions, I cannot even get Siri on Apple TV to
recognize what I am saying. I have often wanted to keep a kind of
journal like this poster but I suspect it would recognize the
correct words about 30% of the time.My wife who, unlike me, is not
a native English speaker has probably a 10% success rate. This is
why any kind of forthcoming voice-response Apple device is
completely a nonstarter to me.
forgot-my-pw - 4 hours ago
Google voice recognition has 5% word error rate now:
https://venturebeat.com/2017/05/17/googles-speech-recognitio...It
might actually be better of Apple were to license the speech
recognition or use the Cloud Speech API.
Domenic_S - 5 hours ago
What a weird conclusion, that future -- and presumably better --
tech would be a nonstarter because current tech doesn't work for
you.
michaelmrose - 5 hours ago
The parent poster is dubious that improvements in features will
coincide with improvements in recognizing his voice. Voice
assist functionality could be 200% more awesome but if it
specifically doesn't seem good at just recognizing what he has
said such functionality is useless to him. This isn't terribly
strange at all.
maerF0x0 - 4 hours ago
I'm gonna make a service where you ask my service and it answers
the 4 answers given :D
netvarun - 3 hours ago
Shameless Plug: I work at Semantics3 [https://semantics3.com/] - an
API for product and pricing data.These 4 digital assistants should
partner with us to help their users find the prices for a pack of
Lays chips, iphone, etc. ;)
lowbloodsugar - 2 hours ago
Man asks Amazon digital assistant about the price of Lay's chips,
is disappointed when it "has little interest in having a
conversation about it" and wants to sell it to him instead. o_0
myrandomcomment - 1 hours ago
So I decided to ask Siri some of these that he listed as giving a
Bing search answer that I felt Wolfram would have answered
correctly for Siri. I my case I did get the correct answer, not a
Bing result.Where does the Jackfruit grow?What is the boiling point
of water at an altitude of 1km?What is 1km in feet?How far away is
Disneyland?For "km" I said "kilometer" and not "km".
contingencies - 3 hours ago
"Okay Google, spend my money."
glitcher - 2 hours ago
The Alexa version :) https://xkcd.com/1807/
elicash - 6 hours ago
I got 22 out of the "40 verbose Assistant questions" correct. Not
bad! I beat them all (as a percentage).Maybe not a bad idea for a
gameshow.
[deleted]
chris_overseas - 6 hours ago
That's similar to how The Chase[1] works, except contestants go
head to head with a professional quiz master instead of a digital
assistant.[1]
https://en.wikipedia.org/wiki/The_Chase_(UK_game_show)
coldcode - 5 hours ago
I would like to know how much the pack of lies costs?
anentropic - 4 hours ago
"It?s right there on their website. These numbers do not
accurately represent the price you will pay"
jandrese - 1 hours ago
> Siri took the crown on factual questions, but surprisingly did
poorly on reasoning (?Queries?) where I expected the Wolfram Alpha-
backed service to get flying colours.Didn't Apple ditch the
WolframAlpha integration pretty quickly after Siri was released? I
remember a lot of the Wolfram type queries stopped working shortly
after release.
valleyer - 1 hours ago
This very article shows examples of Siri responding with Wolfram
Alpha results.
rojobuffalo - 6 hours ago
Might be interesting to test with Wolfram Alpha as well. It looks
like some of the questions wouldn't fit the WA API, but I'm curious
how it would score.
ksk - 5 hours ago
One problem is that the computing power dedicated to each user is
minuscule. If you could dedicate a super computer for processing
every input, you could have a much more sophisticated system that
could easily deal with all of those queries.
EGreg - 6 hours ago
Why isn't siri as half as smart as Wolfram Alpha's box? Someone
should license them!I want to be able to ask basic factual
questions while driving, get the answers and dig deeper.Until then
I would like an audio service like Google Helpouts used to be, on
demand. Like Magic service.
fooker - 5 hours ago
Siri had wolfram alpha integration at launch. Then Apple got
overconfident and removed it.Google "Siri getting dumber" for
reference.
freeone3000 - 5 hours ago
Wolfram Alpha is slow. Even if it's right, it's only good for
knowledge questions out of its database - things like public
figures ("how many children does barak obama have"), physical
statistics ("what is the melting point of tungsten"), and so on
work fine. However, topical ("what about aluminium?"), temporal
("what's the weather?"), and location-based ("show me restaurants
nearby") are outside of scope for wolfram alpha entirely - so a
given app must aggregate.Why don't apps aggregate? The "can you
handle this?" api endpoint is frequently returns false positives,
and the proper API is really slow (multiple seconds) for
negatives . If we get a false positive, or something hard to
detect as a negative, that's the only answer we can show. And
since a voice assistant is expected to return one answer quickly,
this is straight out.